Make playback mochitests faster by making test media files shorter

RESOLVED FIXED in Firefox 52

Status

()

defect
P3
normal
RESOLVED FIXED
4 years ago
2 years ago

People

(Reporter: cpearce, Assigned: kikuo)

Tracking

unspecified
mozilla52
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox52 fixed)

Details

Attachments

(1 attachment, 4 obsolete attachments)

We can make running the media mochitests faster by reducing the length of the media files listed in dom/media/test/manifest.js. We play the files in gSmallTests a lot during a test run, we can make the tests faster if we reduce the duration of the files.

Ideally, all the files in gSmallTests should have duration less than 500ms, and we should have one of every combination of container (MP4,WebM,Ogg,Wav) and codec available in those containers.
Someone in Taipei is welcome to take this too.
Flags: needinfo?(bwu)
Benjamin,
Please help on this. 
Thanks!
Assignee: nobody → bechen
Flags: needinfo?(bwu)
James,
This bug can help you understand media test cases and learn some knowledge about media containers and codecs. 
Enjoy! :-)
Assignee: bechen → jacheng
Mass change P2 -> P3
Priority: P2 → P3
Per offline discussion with James, I'll take this bug and investigate whether I can separate this issue into different aspects first.
Assignee: jacheng → kikuo
(In reply to Kilik Kuo [:kikuo] from comment #5)
> Per offline discussion with James, I'll take this bug and investigate
> whether I can separate this issue into different aspects first.
Cool! This must be too easy for James and he likes to take more challenging ones. :-)
1st try, by replace the following 4 files to a shorter version.

var gSmallTests = { 
...
  // { name:"seek.webm", type:"video/webm", width:320, height:240, duration:3.966 },
  // { name:"vp9.webm", type:"video/webm", width:320, height:240, duration:4 },
  // { name:"detodos.opus", type:"audio/ogg; codecs=opus", duration:2.9135 },
  // { name:"gizmo.mp4", type:"video/mp4", width:560, height:320, duration:5.56 },
  { name:"seek-short.webm", type:"video/webm", width:320, height:240, duration:0.47 },
  { name:"vp9-short.webm", type:"video/webm", width:320, height:240, duration:0.20 },
  { name:"detodos-short.opus", type:"audio/ogg; codecs=opus", duration:0.22 },
  { name:"gizmo-short.mp4", type:"video/mp4", width:560, height:320, duration:0.512 },
...
}

MochitestSuite-Info-Run-Time (./mach mochitest dom/media/test) reduces about 309s from (4379s => 4070s).
Single-file-Info-Run-Time (./mach mochitest dom/media/test/test_play_event.html) reduces about 12s from (18.72s => 6.77s)

I'll check if we can save more by replacing other media lists with these shorter files.
Via command "ffmpeg -i INPUT -t DURATION -c copy INPUT-short",
shorter version of some media files are provided to replace original ones in the lists which are in front of the gPlayTests list.

Besides, some long media files should not be trimmed because they're supposed to be tested that long. e.g. { name:"test-6-5.1.opus", type:"audio/ogg; codecs=opus", duration:10.333 } for multichannel opus. Or { name:"gizmo.mp4", type:"video/mp4", keyframes:[0, 1.0, 2.0, 3.0, 4.0, 5.0 ] } for fast seeking.

So, I'll provide a Part2 to take care of these kind of files.
This test(test_mediatrack_parsing_ogg.html) will be finished soon after HTMLMediaElement.onloadedmetadata is invoked.

We could speed up by shorten these files to reduce io.
Avg. time decreased from 5.7s => 2.75s
Because the shorten files we have, the fewer seekToNextFrame() calls will be invoked in test_seekToNextFrame.html

This reduces the elapsed time from 53.11s to 30.95s.
Actually just reuse those newly-generated *-short.{mp4,webm,etc..}

This modification has'no effect on {test_reset_src,test_load_source}.html because there's no actually playback happening.

It reduced the running time of test_playback.html from 116.68s to 103.4s. It varies, but normally around 8~10%.
Comment on attachment 8790579 [details]
Bug 1257116 - Shorten media files to make mochitest faster.

https://reviewboard.mozilla.org/r/78332/#review76900

Shorten some media files and also keep origin ones because they may be used for certain test purposes.

On my machine (i7 4 cores, SSD).
The elapsed run time for dom/media/test/test*
from
3818s.
to
3116s.

Though we could still make it even better, but I'd like to stop here and spend time on other issus.
Attachment #8790579 - Flags: review?(cpearce)
(In reply to Kilik Kuo [:kikuo] from comment #13)
> Comment on attachment 8790579 [details]
> Bug 1257116 - Shorten media files to make mochitest faster.
> 
> https://reviewboard.mozilla.org/r/78332/#review76900
> 
> Shorten some media files and also keep origin ones because they may be used
> for certain test purposes.
> 
> On my machine (i7 4 cores, SSD).
> The elapsed run time for dom/media/test/test*
> from
> 3818s.
> to
> 3116s.
Wow. You save so much time!! \o/
Comment on attachment 8790579 [details]
Bug 1257116 - Shorten media files to make mochitest faster.

https://reviewboard.mozilla.org/r/78334/#review77966

Great, thanks!
Attachment #8790579 - Flags: review?(cpearce) → review+
Try run, https://treeherder.mozilla.org/#/jobs?repo=try&revision=45a3683b7526&selectedJob=27613676

Some oranges on tier2, but from the log information, I don't think that's relevant to this modification.
Another try run to make sure.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=5289e01aef41
(In reply to Kilik Kuo [:kikuo] from comment #16)
> Try run,
> https://treeherder.mozilla.org/#/
> jobs?repo=try&revision=45a3683b7526&selectedJob=27613676
> 
> Some oranges on tier2, but from the log information, I don't think that's
> relevant to this modification.
> Another try run to make sure.
> 
> https://treeherder.mozilla.org/#/jobs?repo=try&revision=5289e01aef41

A try run WITHOUT any modification, https://treeherder.mozilla.org/#/jobs?repo=try&revision=c5481c8eec0f
I think it's ok to go !
Attachment #8782883 - Attachment is obsolete: true
Attachment #8783439 - Attachment is obsolete: true
Attachment #8785125 - Attachment is obsolete: true
Attachment #8786287 - Attachment is obsolete: true
Keywords: checkin-needed
Pushed by cbook@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/4a5b6754da08
Shorten media files to make mochitest faster. r=cpearce
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/4a5b6754da08
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla52
Depends on: 1364870
See Also: → 1364872
You need to log in before you can comment on or make changes to this bug.