Closed
Bug 1264772
Opened 8 years ago
Closed 8 years ago
Schedule android mochitest-media as --chunk-by-runtime
Categories
(Testing :: General, defect)
Tracking
(firefox48 fixed)
RESOLVED
FIXED
mozilla48
Tracking | Status | |
---|---|---|
firefox48 | --- | fixed |
People
(Reporter: dminor, Assigned: dminor)
Details
Attachments
(5 files)
58 bytes,
text/x-review-board-request
|
ahal
:
review+
|
Details |
58 bytes,
text/x-review-board-request
|
ahal
:
review+
|
Details |
58 bytes,
text/x-review-board-request
|
ahal
:
review+
|
Details |
58 bytes,
text/x-review-board-request
|
ahal
:
review+
|
Details |
58 bytes,
text/x-review-board-request
|
drno
:
review+
|
Details |
The mochitest-media tests on Android are currently quite unbalanced. The first chunk takes around ~55 minutes to run, the second ~12 minutes. Using --chunk-by-runtime should help with this.
Assignee | ||
Comment 1•8 years ago
|
||
Initial try run here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=1cef2c4a4a23e6048156b522c721e8690e471cfd&selectedJob=19501270 It's not as balanced as I had hoped, but it has the nice side effect of putting all of the WebRTC tests in one (slow) chunk so I think it's worthwhile going forward with this change.
Comment 2•8 years ago
|
||
you could consider splitting up the webrtc chunks into 2 manifests, then chunk-by-runtime would calculate the weight for each manifest independently and do a better job balancing :)
Comment 3•8 years ago
|
||
Yes chunk-by-runtime will keep manifests of tests as one atomic unit (to avoid test dependency issues). So if the mda tests live in one or two very large manifests, then it won't help very much.
Assignee | ||
Comment 4•8 years ago
|
||
This fixes a few typos in the script and updates the README with instructions on how to run it. Review commit: https://reviewboard.mozilla.org/r/46515/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/46515/
Attachment #8741484 -
Flags: review?(ahalberstadt)
Attachment #8741485 -
Flags: review?(ahalberstadt)
Attachment #8741486 -
Flags: review?(ahalberstadt)
Attachment #8741487 -
Flags: review?(ahalberstadt)
Assignee | ||
Comment 5•8 years ago
|
||
Review commit: https://reviewboard.mozilla.org/r/46517/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/46517/
Assignee | ||
Comment 6•8 years ago
|
||
We currently hard code the runtime name to be mochitest-plain. This changes things to use the subsuite name if available. Review commit: https://reviewboard.mozilla.org/r/46519/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/46519/
Assignee | ||
Comment 7•8 years ago
|
||
Review commit: https://reviewboard.mozilla.org/r/46521/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/46521/
Assignee | ||
Comment 8•8 years ago
|
||
(In reply to Joel Maher (:jmaher) from comment #2) > you could consider splitting up the webrtc chunks into 2 manifests, then > chunk-by-runtime would calculate the weight for each manifest independently > and do a better job balancing :) I'm hoping we'll be able to eventually run these tests on Autophone rather than the emulator. They only take 5 - 10 minutes to run there.
Comment 9•8 years ago
|
||
Comment on attachment 8741484 [details] MozReview Request: Bug 1264772 - Fixup writeruntimes.py; r=ahal https://reviewboard.mozilla.org/r/46515/#review43325
Attachment #8741484 -
Flags: review?(ahalberstadt) → review+
Updated•8 years ago
|
Attachment #8741485 -
Flags: review?(ahalberstadt) → review+
Comment 10•8 years ago
|
||
Comment on attachment 8741485 [details] MozReview Request: Bug 1264772 - Add runtimes file for mochitest-media; r=ahal https://reviewboard.mozilla.org/r/46517/#review43327
Comment 11•8 years ago
|
||
Comment on attachment 8741486 [details] MozReview Request: Bug 1264772 - Use subsuite name for chunk-by-runtime; r=ahal https://reviewboard.mozilla.org/r/46519/#review43331
Attachment #8741486 -
Flags: review?(ahalberstadt) → review+
Comment 12•8 years ago
|
||
Comment on attachment 8741487 [details] MozReview Request: Bug 1264772 - Set chunk-by-runtime option for mochitest-media on android; r=ahal https://reviewboard.mozilla.org/r/46521/#review43333
Attachment #8741487 -
Flags: review?(ahalberstadt) → review+
Assignee | ||
Comment 13•8 years ago
|
||
Thanks for the review! I'm going to hold off landing this until I figure out whether this made the WebRTC mochitests on debug builds more intermittent or if they've always been horrible.
Assignee | ||
Comment 14•8 years ago
|
||
Interestingly, it's the first test that is failing pretty consistently [1]. I wonder if there is some kind of required start up time that is no longer elapsing now that we're running a WebRTC test right off the bat. [1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=d42a554f893dfe404566c13f1f82d00532b28aab
Comment 15•8 years ago
|
||
sounds very likely- or a previous test which ran and set the state of firefox/prefs to something needed.
Assignee | ||
Comment 16•8 years ago
|
||
I added a no-op test and things seem much stabler [1]. [1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=886a309ae28cb8e5cc7f0978cf92a56c3c9b738d
Assignee | ||
Comment 17•8 years ago
|
||
The first test in the group seems to have a disproportionate number of intermittents. Adding a no-op test to the beginning of the group helps things. Review commit: https://reviewboard.mozilla.org/r/47197/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/47197/
Attachment #8741484 -
Attachment description: MozReview Request: Bug 1264772 - Fixup writeruntimes.py; r?ahal → MozReview Request: Bug 1264772 - Fixup writeruntimes.py; r=ahal
Attachment #8741485 -
Attachment description: MozReview Request: Bug 1264772 - Add runtimes file for mochitest-media; r?ahal → MozReview Request: Bug 1264772 - Add runtimes file for mochitest-media; r=ahal
Attachment #8741486 -
Attachment description: MozReview Request: Bug 1264772 - Use subsuite name for chunk-by-runtime; r?ahal → MozReview Request: Bug 1264772 - Use subsuite name for chunk-by-runtime; r=ahal
Attachment #8741487 -
Attachment description: MozReview Request: Bug 1264772 - Set chunk-by-runtime option for mochitest-media on android; r?ahal → MozReview Request: Bug 1264772 - Set chunk-by-runtime option for mochitest-media on android; r=ahal
Attachment #8742407 -
Flags: review?(drno)
Assignee | ||
Comment 18•8 years ago
|
||
Comment on attachment 8741484 [details] MozReview Request: Bug 1264772 - Fixup writeruntimes.py; r=ahal Review request updated; see interdiff: https://reviewboard.mozilla.org/r/46515/diff/1-2/
Assignee | ||
Comment 19•8 years ago
|
||
Comment on attachment 8741485 [details] MozReview Request: Bug 1264772 - Add runtimes file for mochitest-media; r=ahal Review request updated; see interdiff: https://reviewboard.mozilla.org/r/46517/diff/1-2/
Assignee | ||
Comment 20•8 years ago
|
||
Comment on attachment 8741486 [details] MozReview Request: Bug 1264772 - Use subsuite name for chunk-by-runtime; r=ahal Review request updated; see interdiff: https://reviewboard.mozilla.org/r/46519/diff/1-2/
Assignee | ||
Comment 21•8 years ago
|
||
Comment on attachment 8741487 [details] MozReview Request: Bug 1264772 - Set chunk-by-runtime option for mochitest-media on android; r=ahal Review request updated; see interdiff: https://reviewboard.mozilla.org/r/46521/diff/1-2/
Assignee | ||
Comment 22•8 years ago
|
||
Another try run: https://treeherder.mozilla.org/#/jobs?repo=try&revision=d324d10c7a502742f660336eb79639c3d1d040ee
Comment 23•8 years ago
|
||
Comment on attachment 8742407 [details] MozReview Request: Bug 1264772 - Add no-op WebRTC Mochitest; r=drno https://reviewboard.mozilla.org/r/47197/#review43981 ::: dom/media/tests/mochitest/test_a_noOp.html:16 (Diff revision 1) > +</head> > +<body> > +<a target="_blank" href="https://bugzilla.mozilla.org/show_bug.cgi?id=1264772">Mozilla Bug 1264772</a> > +<p id="display"></p> > +<div id="content" style="display: none"> > + Nit: WS
Attachment #8742407 -
Flags: review?(drno) → review+
Assignee | ||
Comment 24•8 years ago
|
||
Comment on attachment 8742407 [details] MozReview Request: Bug 1264772 - Add no-op WebRTC Mochitest; r=drno Review request updated; see interdiff: https://reviewboard.mozilla.org/r/47197/diff/1-2/
Attachment #8742407 -
Attachment description: MozReview Request: Bug 1264772 - Add no-op WebRTC Mochitest; r?drno → MozReview Request: Bug 1264772 - Add no-op WebRTC Mochitest; r=drno
Comment 25•8 years ago
|
||
https://hg.mozilla.org/integration/mozilla-inbound/rev/9bc41e65fdd2 https://hg.mozilla.org/integration/mozilla-inbound/rev/28c25f010f9d https://hg.mozilla.org/integration/mozilla-inbound/rev/27983fa7a03a https://hg.mozilla.org/integration/mozilla-inbound/rev/d805abb38086 https://hg.mozilla.org/integration/mozilla-inbound/rev/6245a6bc7d1c
Comment 26•8 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/9bc41e65fdd2 https://hg.mozilla.org/mozilla-central/rev/28c25f010f9d https://hg.mozilla.org/mozilla-central/rev/27983fa7a03a https://hg.mozilla.org/mozilla-central/rev/d805abb38086 https://hg.mozilla.org/mozilla-central/rev/6245a6bc7d1c
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
status-firefox48:
--- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla48
You need to log in
before you can comment on or make changes to this bug.
Description
•