897141 - Schedule full emulator tests on b2g18

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Andrew Halberstadt [:ahal]]

There is a frequent intermittent crash on the b2g18 branch (bug 880285) as well as host of other intermittent problems. One theory is that the emulator snapshot keeps getting out of date. Seeing as b2g18 is going to be around for at least another 6 months, we should enable tests on the full stack builds and see how they run.

Because we don't have b2g18 try, we might need to hide them if there are many failures. If so we'll have to decide whether to attempt to fix or disable again and just live with the current setup awhile longer.

Comment 1

[Aki Sasaki (not active)]

Attachment: emu-b2g18

I have the feeling you're going to tell me to refactor mozilla-tests/b2g_config.py (given bug 880273), but taking a shot anyway.

* splitting linux64_gecko away from the emulator/ics_armv7a_gecko loop for simplicity.
* removing 'emulator' from all non-mc branches except b2g18
* removing ics_armv7a_gecko from all mc branches (keeping it on b2g18)

It's ugly.  I don't like the loops, and it looks like they're not going to go away anytime soon.

I imagine the long term fix involves bug 817046.

Comment 2

[Aki Sasaki (not active)]

Attachment: builder.diff

This is the diff in the builder list.

Comment 3

[Armen [:armenzg]]

Comment on attachment 780131 [details] [diff] [review]
emu-b2g18

Review of attachment 780131 [details] [diff] [review]:
-----------------------------------------------------------------

This is something that is already hacky and not clearly going to be riding the trains as with the other bugs.

lgtm.

Comment 4

[Aki Sasaki (not active)]

Comment on attachment 780131 [details] [diff] [review]
emu-b2g18

https://hg.mozilla.org/build/buildbot-configs/rev/e7e7ce0a59c3

Comment 5

[Rail Aliiev [:rail]]

In production

Comment 6

[Ryan VanderMeulen [:RyanVM]]

I retriggered a full-stack build after the reconfig, but I don't see any tests starting.

Comment 7

[Ryan VanderMeulen [:RyanVM]]

(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #6)
> I retriggered a full-stack build after the reconfig, but I don't see any
> tests starting.

Still no sign of these tests...

Comment 8

[Aki Sasaki (not active)]

It's not sendchanging because we're not building the tests zip.
Gotta figure out how to enable those...

Comment 9

[Aki Sasaki (not active)]

Attachment: emulator-b2g18-config

This adds the package-tests and upload changes from mozilla-central emulator config.json, and updates the l10n/manifest settings to match the mozilla-b2g18 unagi config.json.

Comment 10

[Aki Sasaki (not active)]

This needs to land on b2g18 to get the emulator tests running.

Comment 11

[Preeti Raghunath(:Preeti)]

leo+

Comment 12

[Aki Sasaki (not active)]

Comment on attachment 785921 [details] [diff] [review]
emulator-b2g18-config

https://hg.mozilla.org/releases/mozilla-b2g18/rev/c0d287b8ddf1

Comment 13

[Aki Sasaki (not active)]

Ok, scheduled.  https://tbpl.mozilla.org/?tree=Mozilla-B2g18
They probably need some tlc; they're like autumn.

Comment 14

[Andrew Halberstadt [:ahal]]

Fwiw we only run reftest-sanity on b2g18, so the failures (most likely) don't have anything to do with the fact that it's a full stack build.

Crashtests are what we are most interested in given that that's where the intermittent failure is most common. There seems to be one perma-orange there, but otherwise they seem stable.

Comment 15

[Ryan VanderMeulen [:RyanVM]]

Looking good, thanks! We should be doing this on the v1.1hd branch as well, but I will file a new bug for that.

Comment 16

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-b2g18_v1_1_0_hd/rev/c0d287b8ddf1

(This was from a b2g18 branch merge to v1.1hd)