Create 3 chunks for mochitest-media on TC linux64 opt/debug/pgo

RESOLVED FIXED in Firefox 54

Status

Testing
General
RESOLVED FIXED
a year ago
11 months ago

People

(Reporter: jgriffin, Assigned: pyang)

Tracking

(Blocks: 1 bug)

unspecified
mozilla54
Points:
---

Firefox Tracking Flags

(firefox54 fixed)

Details

MozReview Requests

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(2 attachments)

(Reporter)

Description

a year ago
According to the End to End times dashboard, https://people-mozilla.org/~klahnakoski/MoBuildbotTimings/End-to-End.html, mochitest-media is by far the longest-running suite on TaskCluster linux64 opt/debug, and the one we need to chunk in order to bring down our E2E times.

Let's initially split this into 3 chunks.
(Assignee)

Comment 1

a year ago
I ran 3 chunks of mochitest-media in several platforms
opt https://treeherder.mozilla.org/#/jobs?repo=try&revision=400fcce8889a7bbc4f44e732dbafe16fd86cc329
debug https://treeherder.mozilla.org/#/jobs?repo=try&revision=9cf4be8ddeb152103b7391a61ea196aa6d9d6490

linux64 debug reduced from 1 hour to max 40 minutes, avg. 33 minutes
linux64 opt reduced from 50 minutes to max 35 minutes, avg. 27 minutes
I would run each of the chunks 10 times each as each run is more variable than one would like to admit.

In addition we have linux64-asan (which is different from opt/debug).
(Assignee)

Comment 3

a year ago
Thanks!
Here're summary and each of chunks ran 10 time
opt https://treeherder.mozilla.org/#/jobs?repo=try&revision=400fcce8889a7bbc4f44e732dbafe16fd86cc329
debug https://treeherder.mozilla.org/#/jobs?repo=try&revision=9cf4be8ddeb152103b7391a61ea196aa6d9d6490
linux64 asan https://treeherder.mozilla.org/#/jobs?repo=try&revision=6dc2684aace7a6ba84f4a7c56f17ce2d02eb5218


For statistics I remove those extremely fast, since majority of each data set were close.
linux64 debug reduced from 1 hour to max 42 minutes (chunk1 avg. 32 min., chunk2 avg. 42 min., chunk3 avg. 22 min.)
linux64 opt reduced from 50 minutes to max 34 minutes (chunk1 avg. 25 min., chunk2 avg. 34 min., chunk3 avg. 19 min.)

Result of other platforms was included in the link.
Comment hidden (mozreview-request)
(Assignee)

Comment 7

a year ago
Thanks Kyle for rich dashboard! 
I can see about 15 to 20 minutes reduced by this change.  However if there's a way to split them by running minutes would be nicer.
Here is another result using chunks=5
https://treeherder.mozilla.org/#/jobs?repo=try&author=pyang@mozilla.com&selectedJob=74236498
about 10 more minutes eliminated. Considering longest running task after media suite split is 82 minutes (windows2012 64bit pgo build), we might not gain a lot to have 5 chunks.

jgriffin, do we need more observation or simply move to next target?
Flags: needinfo?(jgriffin)
(Reporter)

Comment 8

a year ago
We're explicitly excluding PGO from our efforts to reduce end-to-end times, because there isn't a lot we can do to improve them.

I think the averages you gave are pretty reasonable, and I think we can green up the tests and finish this.
Flags: needinfo?(jgriffin)
(Assignee)

Updated

a year ago
Attachment #8827377 - Flags: review?(jgriffin)
(Reporter)

Comment 9

a year ago
on the try runs in comment #7, there seem to be perma-fails in chunks 3 and 4 on debug. I don't seem them with just three chunks from try runs in comment #3. Have you investigated these yet?
(Assignee)

Comment 10

a year ago
I retry by 5 chunks with latest m-c.
https://treeherder.mozilla.org/#/jobs?repo=try&revision=8866c2f74dcebd36d9b6ab9ea775c28c86785c3c

We can't see same issue reproduced so possibly a bug fixed.

However in mda3 it showed "INFO - [Child 2098] WARNING: Late NotifyFinished after MediaManager shutdown: file /home/worker/workspace/build/src/dom/media/MediaManager.cpp,"
then test process froze and kept complaining message dropped.

Might be a teardown issue but need more investigating.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=8ff374e9906144befe59d2d262730bdc32648162
While 3 chunks version kept being green.
(Reporter)

Comment 11

a year ago
mozreview-review
Comment on attachment 8827377 [details]
Bug 1329819 - Create 3 chunks for mochitest-media on TC linux64 opt/debug/pgo

https://reviewboard.mozilla.org/r/105074/#review113052
Attachment #8827377 - Flags: review?(jgriffin) → review+
(Reporter)

Comment 12

a year ago
Great, thanks!
(Reporter)

Comment 13

11 months ago
Is this good to land now?
(Assignee)

Comment 14

11 months ago
Comment on attachment 8827377 [details]
Bug 1329819 - Create 3 chunks for mochitest-media on TC linux64 opt/debug/pgo

I think it's good to go.
Attachment #8827377 - Flags: checked-in?
Keywords: checkin-needed

Comment 15

11 months ago
Pushed by cbook@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/e5b189f0363b
Create 3 chunks for mochitest-media on TC linux64 opt/debug/pgo r=jgriffin
Keywords: checkin-needed

Comment 16

11 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/e5b189f0363b
Status: NEW → RESOLVED
Last Resolved: 11 months ago
status-firefox54: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla54
You need to log in before you can comment on or make changes to this bug.