926741 - Not running B2G emulator builds and tests on b2g18 anymore

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Ryan VanderMeulen [:RyanVM]]

With bug 924503, we are no longer running emulator builds and tests on the b2g18 branch. I'm told this was a purposeful decision. This means that we have *no* visibility into whether a leo+ blocker is passing tests until it's merged to v1.1hd, meaning we're now compounding any bustage that lands there with no visibility as to what caused the regression.

Can we please reconsider this decision and restore emulator builds and tests?

Comment 1

[Ryan VanderMeulen [:RyanVM]]

I will also point out that while the v1.1hd branch does contain the fixes that b2g18 also contains, the two are *not* identical and we shouldn't assume that the hd+ patches that land only on v1.1hd won't affect tests in their own right.

Comment 2

[Ryan VanderMeulen [:RyanVM]]

Faramarz, it sounds like you're the person to ping about this?

Comment 3

[Ryan VanderMeulen [:RyanVM]]

This is a blocker IMO.

Comment 4

[Aki Sasaki (not active)]

Attachment: emulator-b2g18

Not touching this argument/decision, but this is the one-liner that should RESO FIXED this bug, if we decide to.

Comment 5

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

The answer seems to be entangled in different people having different ideas of "what releases ship from what branches". Per b2g meeting this afternoon, dylan will close the loop with clee + faramarz to get a formal answer here. 

ryan/ctalbert: can you also please verify the "remove" builds listed on https://wiki.mozilla.org/User:Asasaki:B2GBuilds in case there are other builds which you feel should not have been disabled?

Comment 6

[Ryan VanderMeulen [:RyanVM]]

(In reply to John O'Duinn [:joduinn] from comment #5)
> ryan/ctalbert: can you also please verify the "remove" builds listed on
> https://wiki.mozilla.org/User:Asasaki:B2GBuilds in case there are other
> builds which you feel should not have been disabled?

The only other concern I have from this list is that the emulator builds and tests were also disabled on v1.0.1, meaning we'd have no test coverage there either in the event of a chemspill release.

While the odds of that actually happening are probably quite low and time is running short for that branch anyway, we should probably leave the capability intact for generating those builds and tests for as long as code might still be changing there. It's not like it's costing us anything since builds are only generated on checkin for that branch anyway.

None of the other REMOVEs affect test coverage, so I have no opinion on the rest of the list :)

Comment 7

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

per email from faramarz, we should turn emulator builds+tests back on for b2g18.

Comment 8

[Aki Sasaki (not active)]

Comment on attachment 817538 [details] [diff] [review]
emulator-b2g18

https://hg.mozilla.org/build/buildbot-configs/rev/a6148c5ad888

Should be fixed on the next reconfig.

Comment 9

[Justin Wood (:Callek)]

reconf happened on thursday