Schedule mochitests on debug B2G emulators on all trunk trees

RESOLVED FIXED

Status

Release Engineering
General Automation
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: jgriffin, Assigned: jgriffin)

Tracking

(Blocks: 1 bug)

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments, 3 obsolete attachments)

(Assignee)

Description

4 years ago
It looks like we've achieved a < 10% failure rate on mochitests on debug B2G emulators on cedar; we should schedule them across all trunk trees now.
(Assignee)

Comment 1

4 years ago
Created attachment 8338154 [details] [diff] [review]
Schedule debug B2G mochitests on trunk branches,

I think this should do the trick.
Attachment #8338154 - Flags: review?(aki)
(Assignee)

Updated

4 years ago
Assignee: nobody → jgriffin

Comment 2

4 years ago
Created attachment 8338168 [details] [diff] [review]
builderdiff

Diff of a clean/patched linux test master.
This looks good.

Created via:
./setup-master.py -l
./setup-master.py bm51-tests1-linux bm51-tests1-linux
cd bm51-tests1-linux
/src/clean/braindump/buildbot-related/builder_list.py master.cfg > ../../clean.dump
# apply patch to buildbot-configs
/src/clean/braindump/buildbot-related/builder_list.py master.cfg > ../../dirty.dump
cd ../..
diff {clean,dirty}.dump > builderdiff

Comment 3

4 years ago
Comment on attachment 8338154 [details] [diff] [review]
Schedule debug B2G mochitests on trunk branches,

I also see the debug emulator build running sendchanges on fx-team and m-c, so we should see tests running after this patch lands + reconfigs.
Attachment #8338154 - Flags: review?(aki) → review+

Comment 4

4 years ago
Thanks Jonathan!
(Assignee)

Comment 5

4 years ago
Created attachment 8358550 [details] [diff] [review]
Schedule debug B2G mochitests on trunk branches,

This was bitrotted quite a bit; reflagging for review.  Aki, can you run another builder diff on this to make sure I got it right?
Attachment #8358550 - Flags: review?(aki)
(Assignee)

Updated

4 years ago
Attachment #8338154 - Attachment is obsolete: true

Comment 6

4 years ago
Comment on attachment 8358550 [details] [diff] [review]
Schedule debug B2G mochitests on trunk branches,

>+BRANCHES['mozilla-b2g18_v1_0_1']['platforms']['emulator']['enable_debug_unittests'] = False

We deleted the 1.0.1 branch so this line is causing a KeyError:

Traceback (most recent call last):
  File "/src/clean/braindump/buildbot-related/builder_list.py", line 20, in <module>
    execfile(os.path.basename(sys.argv[1]), g)
  File "master.cfg", line 12, in <module>
    import b2g_config
  File "/src/jgriffin/buildbot-configs/bm51-tests1-linux/b2g_config.py", line 1443, in <module>
    BRANCHES['mozilla-b2g18_v1_0_1']['platforms']['emulator']['enable_debug_unittests'] = False
KeyError: 'mozilla-b2g18_v1_0_1'

r=me with that line removed.

Attaching the new builderdiff.
Attachment #8358550 - Flags: review?(aki) → review+

Comment 7

4 years ago
Created attachment 8358586 [details] [diff] [review]
builderdiff
Attachment #8338168 - Attachment is obsolete: true
(Assignee)

Comment 8

4 years ago
Created attachment 8358587 [details] [diff] [review]
Schedule debug B2G mochitests on trunk branches,

1_0_1 line deleted, and MOCHITEST changed to MOCHITEST_EMULATOR_DEBUG, so that we get all 15 chunks, instead of 9.
(Assignee)

Updated

4 years ago
Attachment #8358550 - Attachment is obsolete: true
(Assignee)

Comment 9

4 years ago
Comment on attachment 8358587 [details] [diff] [review]
Schedule debug B2G mochitests on trunk branches,

Review of attachment 8358587 [details] [diff] [review]:
-----------------------------------------------------------------

carry r+ forward
Attachment #8358587 - Flags: review+
(Assignee)

Comment 10

4 years ago
Sheriffs, we're turning on debug B2G emulator mochitests with the next buildbot reconfig.  These are green on pine now; we'll deal aggressively with any frequent intermittents by disabling the tests.
(Assignee)

Comment 11

4 years ago
https://hg.mozilla.org/build/buildbot-configs/rev/29f5bfaea918
Depends on: 958732
(In reply to Jonathan Griffin (:jgriffin) from comment #10)
> Sheriffs, we're turning on debug B2G emulator mochitests with the next
> buildbot reconfig.  These are green on pine now; we'll deal aggressively
> with any frequent intermittents by disabling the tests.

Stop the Presses! Everything is failing right now because of bug 958732.
The patch in bug 958732 fixes this. We just have to make sure it lands first.
in production
All b2g emulator debug mochitest jobs hidden on b2g-inbound for being orange; the other trees will need doing as soon as jobs have completed on them (and thus are able to be hidden in TBPL).
(Assignee)

Updated

4 years ago
Depends on: 958432
(Assignee)

Comment 16

4 years ago
These are running very green again on b2g-inbound; any reason not to unhide them?
Oh they are hidden? Yes please un-hide again.
Unhidden on b2g-inbound, where I'm always happy to unhide (amusingly, permaorange on them had just landed). But let's see some of that comment 10 aggressive disabling before we unhide them elsewhere.
(In reply to Phil Ringnalda (:philor) from comment #18)
> Unhidden on b2g-inbound, where I'm always happy to unhide (amusingly,
> permaorange on them had just landed). But let's see some of that comment 10
> aggressive disabling before we unhide them elsewhere.

Well if we know that changeset that regressed it we shouldn't blame the test :)
Oh, I don't want that permaorange disabled, I already backed out Nuwa for that (though the end result will probably be disabling a bogus assertion); I mean the rest of the constant but not permaorange failures. They are running "very green" if you look at several pushes where they were all green; they are not so green if you are the single person on the planet who looks at every single orange one on every single tree.
(Assignee)

Comment 21

4 years ago
(In reply to Phil Ringnalda (:philor) from comment #20)
> Oh, I don't want that permaorange disabled, I already backed out Nuwa for
> that (though the end result will probably be disabling a bogus assertion); I
> mean the rest of the constant but not permaorange failures. They are running
> "very green" if you look at several pushes where they were all green; they
> are not so green if you are the single person on the planet who looks at
> every single orange one on every single tree.

If you see frequent failures that we haven't noticed, please note them here or in bug 929122 and we'll disable them for now.
(Assignee)

Comment 22

4 years ago
I'm closing this since the tests are running where we want them too; I realize there's some more effort required to green them up, but let's track that against bug 929122 or file new bugs, as needed.
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.