Closed Bug 940161 Opened 9 years ago Closed 9 years ago

Schedule mochitests on debug B2G emulators on all trunk trees

Categories

(Release Engineering :: General, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: jgriffin, Assigned: jgriffin)

References

(Blocks 1 open bug)

Details

Attachments

(2 files, 3 obsolete files)

It looks like we've achieved a < 10% failure rate on mochitests on debug B2G emulators on cedar; we should schedule them across all trunk trees now.
I think this should do the trick.
Attachment #8338154 - Flags: review?(aki)
Assignee: nobody → jgriffin
Attached patch builderdiff (obsolete) — Splinter Review
Diff of a clean/patched linux test master.
This looks good.

Created via:
./setup-master.py -l
./setup-master.py bm51-tests1-linux bm51-tests1-linux
cd bm51-tests1-linux
/src/clean/braindump/buildbot-related/builder_list.py master.cfg > ../../clean.dump
# apply patch to buildbot-configs
/src/clean/braindump/buildbot-related/builder_list.py master.cfg > ../../dirty.dump
cd ../..
diff {clean,dirty}.dump > builderdiff
Comment on attachment 8338154 [details] [diff] [review]
Schedule debug B2G mochitests on trunk branches,

I also see the debug emulator build running sendchanges on fx-team and m-c, so we should see tests running after this patch lands + reconfigs.
Attachment #8338154 - Flags: review?(aki) → review+
Thanks Jonathan!
This was bitrotted quite a bit; reflagging for review.  Aki, can you run another builder diff on this to make sure I got it right?
Attachment #8358550 - Flags: review?(aki)
Attachment #8338154 - Attachment is obsolete: true
Comment on attachment 8358550 [details] [diff] [review]
Schedule debug B2G mochitests on trunk branches,

>+BRANCHES['mozilla-b2g18_v1_0_1']['platforms']['emulator']['enable_debug_unittests'] = False

We deleted the 1.0.1 branch so this line is causing a KeyError:

Traceback (most recent call last):
  File "/src/clean/braindump/buildbot-related/builder_list.py", line 20, in <module>
    execfile(os.path.basename(sys.argv[1]), g)
  File "master.cfg", line 12, in <module>
    import b2g_config
  File "/src/jgriffin/buildbot-configs/bm51-tests1-linux/b2g_config.py", line 1443, in <module>
    BRANCHES['mozilla-b2g18_v1_0_1']['platforms']['emulator']['enable_debug_unittests'] = False
KeyError: 'mozilla-b2g18_v1_0_1'

r=me with that line removed.

Attaching the new builderdiff.
Attachment #8358550 - Flags: review?(aki) → review+
Attached patch builderdiffSplinter Review
Attachment #8338168 - Attachment is obsolete: true
1_0_1 line deleted, and MOCHITEST changed to MOCHITEST_EMULATOR_DEBUG, so that we get all 15 chunks, instead of 9.
Attachment #8358550 - Attachment is obsolete: true
Comment on attachment 8358587 [details] [diff] [review]
Schedule debug B2G mochitests on trunk branches,

Review of attachment 8358587 [details] [diff] [review]:
-----------------------------------------------------------------

carry r+ forward
Attachment #8358587 - Flags: review+
Sheriffs, we're turning on debug B2G emulator mochitests with the next buildbot reconfig.  These are green on pine now; we'll deal aggressively with any frequent intermittents by disabling the tests.
Depends on: 958732
(In reply to Jonathan Griffin (:jgriffin) from comment #10)
> Sheriffs, we're turning on debug B2G emulator mochitests with the next
> buildbot reconfig.  These are green on pine now; we'll deal aggressively
> with any frequent intermittents by disabling the tests.

Stop the Presses! Everything is failing right now because of bug 958732.
The patch in bug 958732 fixes this. We just have to make sure it lands first.
in production
All b2g emulator debug mochitest jobs hidden on b2g-inbound for being orange; the other trees will need doing as soon as jobs have completed on them (and thus are able to be hidden in TBPL).
Depends on: 958432
These are running very green again on b2g-inbound; any reason not to unhide them?
Oh they are hidden? Yes please un-hide again.
Unhidden on b2g-inbound, where I'm always happy to unhide (amusingly, permaorange on them had just landed). But let's see some of that comment 10 aggressive disabling before we unhide them elsewhere.
(In reply to Phil Ringnalda (:philor) from comment #18)
> Unhidden on b2g-inbound, where I'm always happy to unhide (amusingly,
> permaorange on them had just landed). But let's see some of that comment 10
> aggressive disabling before we unhide them elsewhere.

Well if we know that changeset that regressed it we shouldn't blame the test :)
Oh, I don't want that permaorange disabled, I already backed out Nuwa for that (though the end result will probably be disabling a bogus assertion); I mean the rest of the constant but not permaorange failures. They are running "very green" if you look at several pushes where they were all green; they are not so green if you are the single person on the planet who looks at every single orange one on every single tree.
(In reply to Phil Ringnalda (:philor) from comment #20)
> Oh, I don't want that permaorange disabled, I already backed out Nuwa for
> that (though the end result will probably be disabling a bogus assertion); I
> mean the rest of the constant but not permaorange failures. They are running
> "very green" if you look at several pushes where they were all green; they
> are not so green if you are the single person on the planet who looks at
> every single orange one on every single tree.

If you see frequent failures that we haven't noticed, please note them here or in bug 929122 and we'll disable them for now.
I'm closing this since the tests are running where we want them too; I realize there's some more effort required to green them up, but let's track that against bug 929122 or file new bugs, as needed.
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.