Closed
Bug 1329819
Opened 7 years ago
Closed 7 years ago
Create 3 chunks for mochitest-media on TC linux64 opt/debug/pgo
Categories
(Testing :: General, defect)
Testing
General
Tracking
(firefox54 fixed)
RESOLVED
FIXED
mozilla54
Tracking | Status | |
---|---|---|
firefox54 | --- | fixed |
People
(Reporter: jgriffin, Assigned: pyang)
References
(Blocks 1 open bug)
Details
Attachments
(2 files)
59 bytes,
text/x-review-board-request
|
jgriffin
:
review+
pyang
:
checked-in?
|
Details |
100.86 KB,
image/jpeg
|
Details |
According to the End to End times dashboard, https://people-mozilla.org/~klahnakoski/MoBuildbotTimings/End-to-End.html, mochitest-media is by far the longest-running suite on TaskCluster linux64 opt/debug, and the one we need to chunk in order to bring down our E2E times. Let's initially split this into 3 chunks.
Assignee | ||
Comment 1•7 years ago
|
||
I ran 3 chunks of mochitest-media in several platforms opt https://treeherder.mozilla.org/#/jobs?repo=try&revision=400fcce8889a7bbc4f44e732dbafe16fd86cc329 debug https://treeherder.mozilla.org/#/jobs?repo=try&revision=9cf4be8ddeb152103b7391a61ea196aa6d9d6490 linux64 debug reduced from 1 hour to max 40 minutes, avg. 33 minutes linux64 opt reduced from 50 minutes to max 35 minutes, avg. 27 minutes
Comment 2•7 years ago
|
||
I would run each of the chunks 10 times each as each run is more variable than one would like to admit. In addition we have linux64-asan (which is different from opt/debug).
Assignee | ||
Comment 3•7 years ago
|
||
Thanks! Here're summary and each of chunks ran 10 time opt https://treeherder.mozilla.org/#/jobs?repo=try&revision=400fcce8889a7bbc4f44e732dbafe16fd86cc329 debug https://treeherder.mozilla.org/#/jobs?repo=try&revision=9cf4be8ddeb152103b7391a61ea196aa6d9d6490 linux64 asan https://treeherder.mozilla.org/#/jobs?repo=try&revision=6dc2684aace7a6ba84f4a7c56f17ce2d02eb5218 For statistics I remove those extremely fast, since majority of each data set were close. linux64 debug reduced from 1 hour to max 42 minutes (chunk1 avg. 32 min., chunk2 avg. 42 min., chunk3 avg. 22 min.) linux64 opt reduced from 50 minutes to max 34 minutes (chunk1 avg. 25 min., chunk2 avg. 34 min., chunk3 avg. 19 min.) Result of other platforms was included in the link.
Comment hidden (mozreview-request) |
Assignee | ||
Comment 5•7 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=251fc5022721a78f44c98daf97d3719888627019&selectedJob=69509104&filter-searchStr=pgo pgo results are close to opt build
Comment 6•7 years ago
|
||
You can see the sample runs as orange, blue (and green) dots in the test durations chart. https://people-mozilla.org/~klahnakoski/MoBuildbotTimings/TaskCluster-Tests-Detail.html#sampleMax=2017-01-18&sampleMin=2017-01-11&showWaitTime=false&test=mochitest-media&product=Firefox&type=OPT&platform=Linux32&branch=Try I attached an image, just in case.
Assignee | ||
Comment 7•7 years ago
|
||
Thanks Kyle for rich dashboard! I can see about 15 to 20 minutes reduced by this change. However if there's a way to split them by running minutes would be nicer. Here is another result using chunks=5 https://treeherder.mozilla.org/#/jobs?repo=try&author=pyang@mozilla.com&selectedJob=74236498 about 10 more minutes eliminated. Considering longest running task after media suite split is 82 minutes (windows2012 64bit pgo build), we might not gain a lot to have 5 chunks. jgriffin, do we need more observation or simply move to next target?
Flags: needinfo?(jgriffin)
Reporter | ||
Comment 8•7 years ago
|
||
We're explicitly excluding PGO from our efforts to reduce end-to-end times, because there isn't a lot we can do to improve them. I think the averages you gave are pretty reasonable, and I think we can green up the tests and finish this.
Flags: needinfo?(jgriffin)
Assignee | ||
Updated•7 years ago
|
Attachment #8827377 -
Flags: review?(jgriffin)
Reporter | ||
Comment 9•7 years ago
|
||
on the try runs in comment #7, there seem to be perma-fails in chunks 3 and 4 on debug. I don't seem them with just three chunks from try runs in comment #3. Have you investigated these yet?
Assignee | ||
Comment 10•7 years ago
|
||
I retry by 5 chunks with latest m-c. https://treeherder.mozilla.org/#/jobs?repo=try&revision=8866c2f74dcebd36d9b6ab9ea775c28c86785c3c We can't see same issue reproduced so possibly a bug fixed. However in mda3 it showed "INFO - [Child 2098] WARNING: Late NotifyFinished after MediaManager shutdown: file /home/worker/workspace/build/src/dom/media/MediaManager.cpp," then test process froze and kept complaining message dropped. Might be a teardown issue but need more investigating. https://treeherder.mozilla.org/#/jobs?repo=try&revision=8ff374e9906144befe59d2d262730bdc32648162 While 3 chunks version kept being green.
Reporter | ||
Comment 11•7 years ago
|
||
mozreview-review |
Comment on attachment 8827377 [details] Bug 1329819 - Create 3 chunks for mochitest-media on TC linux64 opt/debug/pgo https://reviewboard.mozilla.org/r/105074/#review113052
Attachment #8827377 -
Flags: review?(jgriffin) → review+
Reporter | ||
Comment 12•7 years ago
|
||
Great, thanks!
Reporter | ||
Comment 13•7 years ago
|
||
Is this good to land now?
Assignee | ||
Comment 14•7 years ago
|
||
Comment on attachment 8827377 [details] Bug 1329819 - Create 3 chunks for mochitest-media on TC linux64 opt/debug/pgo I think it's good to go.
Attachment #8827377 -
Flags: checked-in?
Updated•7 years ago
|
Keywords: checkin-needed
Comment 15•7 years ago
|
||
Pushed by cbook@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/e5b189f0363b Create 3 chunks for mochitest-media on TC linux64 opt/debug/pgo r=jgriffin
Keywords: checkin-needed
Comment 16•7 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/e5b189f0363b
Status: NEW → RESOLVED
Closed: 7 years ago
status-firefox54:
--- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla54
You need to log in
before you can comment on or make changes to this bug.
Description
•