1257116 - Make playback mochitests faster by making test media files shorter

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P3)

Description

[Chris Pearce [:cpearce (Not reading bugmail)]]

We can make running the media mochitests faster by reducing the length of the media files listed in dom/media/test/manifest.js. We play the files in gSmallTests a lot during a test run, we can make the tests faster if we reduce the duration of the files.

Ideally, all the files in gSmallTests should have duration less than 500ms, and we should have one of every combination of container (MP4,WebM,Ogg,Wav) and codec available in those containers.

Comment 1

[Chris Pearce [:cpearce (Not reading bugmail)]]

Someone in Taipei is welcome to take this too.

Comment 2

[Blake Wu [:bwu][:blakewu]]

Benjamin,
Please help on this. 
Thanks!

Comment 3

[Blake Wu [:bwu][:blakewu]]

James,
This bug can help you understand media test cases and learn some knowledge about media containers and codecs. 
Enjoy! :-)

Comment 4

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Mass change P2 -> P3

Comment 5

[Kilik Kuo [:kikuo] (inactive)]

Per offline discussion with James, I'll take this bug and investigate whether I can separate this issue into different aspects first.

Comment 6

[Blake Wu [:bwu][:blakewu]]

(In reply to Kilik Kuo [:kikuo] from comment #5)
> Per offline discussion with James, I'll take this bug and investigate
> whether I can separate this issue into different aspects first.
Cool! This must be too easy for James and he likes to take more challenging ones. :-)

Comment 7

[Kilik Kuo [:kikuo] (inactive)]

1st try, by replace the following 4 files to a shorter version.

var gSmallTests = { 
...
  // { name:"seek.webm", type:"video/webm", width:320, height:240, duration:3.966 },
  // { name:"vp9.webm", type:"video/webm", width:320, height:240, duration:4 },
  // { name:"detodos.opus", type:"audio/ogg; codecs=opus", duration:2.9135 },
  // { name:"gizmo.mp4", type:"video/mp4", width:560, height:320, duration:5.56 },
  { name:"seek-short.webm", type:"video/webm", width:320, height:240, duration:0.47 },
  { name:"vp9-short.webm", type:"video/webm", width:320, height:240, duration:0.20 },
  { name:"detodos-short.opus", type:"audio/ogg; codecs=opus", duration:0.22 },
  { name:"gizmo-short.mp4", type:"video/mp4", width:560, height:320, duration:0.512 },
...
}

MochitestSuite-Info-Run-Time (./mach mochitest dom/media/test) reduces about 309s from (4379s => 4070s).
Single-file-Info-Run-Time (./mach mochitest dom/media/test/test_play_event.html) reduces about 12s from (18.72s => 6.77s)

I'll check if we can save more by replacing other media lists with these shorter files.

Comment 8

[Kilik Kuo [:kikuo] (inactive)]

Attachment: [WIP] Part1 - Provide shorter version of detodos.opus/gizmo.mp4/owl.mp3/seek.ogv/seek.webm/vp9.webm/r11025_s16_c1.wav/big.wav for media lists which are in front of gPlayTests in manifest.js

Via command "ffmpeg -i INPUT -t DURATION -c copy INPUT-short",
shorter version of some media files are provided to replace original ones in the lists which are in front of the gPlayTests list.

Besides, some long media files should not be trimmed because they're supposed to be tested that long. e.g. { name:"test-6-5.1.opus", type:"audio/ogg; codecs=opus", duration:10.333 } for multichannel opus. Or { name:"gizmo.mp4", type:"video/mp4", keyframes:[0, 1.0, 2.0, 3.0, 4.0, 5.0 ] } for fast seeking.

So, I'll provide a Part2 to take care of these kind of files.

Comment 9

[Kilik Kuo [:kikuo] (inactive)]

Attachment: [WIP] Part2 - Shorten media files in gMultitrackInfoOggPlayList

This test(test_mediatrack_parsing_ogg.html) will be finished soon after HTMLMediaElement.onloadedmetadata is invoked.

We could speed up by shorten these files to reduce io.
Avg. time decreased from 5.7s => 2.75s

Comment 10

[Kilik Kuo [:kikuo] (inactive)]

Attachment: [WIP] Part3 - Shorten media files in gSeekToNextFrameTests

Because the shorten files we have, the fewer seekToNextFrame() calls will be invoked in test_seekToNextFrame.html

This reduces the elapsed time from 53.11s to 30.95s.

Comment 11

[Kilik Kuo [:kikuo] (inactive)]

Attachment: [WIP] Part4 - Shorten media files in gPlayTests

Actually just reuse those newly-generated *-short.{mp4,webm,etc..}

This modification has'no effect on {test_reset_src,test_load_source}.html because there's no actually playback happening.

It reduced the running time of test_playback.html from 116.68s to 103.4s. It varies, but normally around 8~10%.

Comment 12

[Kilik Kuo [:kikuo] (inactive)]

Attachment: Bug 1257116 - Shorten media files to make mochitest faster.

Review commit: https://reviewboard.mozilla.org/r/78334/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/78334/

Comment 13

[Kilik Kuo [:kikuo] (inactive)]

Comment on attachment 8790579 [details]
Bug 1257116 - Shorten media files to make mochitest faster.

https://reviewboard.mozilla.org/r/78332/#review76900

Shorten some media files and also keep origin ones because they may be used for certain test purposes.

On my machine (i7 4 cores, SSD).
The elapsed run time for dom/media/test/test*
from
3818s.
to
3116s.

Though we could still make it even better, but I'd like to stop here and spend time on other issus.

Comment 14

[Blake Wu [:bwu][:blakewu]]

(In reply to Kilik Kuo [:kikuo] from comment #13)
> Comment on attachment 8790579 [details]
> Bug 1257116 - Shorten media files to make mochitest faster.
> 
> https://reviewboard.mozilla.org/r/78332/#review76900
> 
> Shorten some media files and also keep origin ones because they may be used
> for certain test purposes.
> 
> On my machine (i7 4 cores, SSD).
> The elapsed run time for dom/media/test/test*
> from
> 3818s.
> to
> 3116s.
Wow. You save so much time!! \o/

Comment 15

[Chris Pearce [:cpearce (Not reading bugmail)]]

Comment on attachment 8790579 [details]
Bug 1257116 - Shorten media files to make mochitest faster.

https://reviewboard.mozilla.org/r/78334/#review77966

Great, thanks!

Comment 16

[Kilik Kuo [:kikuo] (inactive)]

Try run, https://treeherder.mozilla.org/#/jobs?repo=try&revision=45a3683b7526&selectedJob=27613676

Some oranges on tier2, but from the log information, I don't think that's relevant to this modification.
Another try run to make sure.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=5289e01aef41

Comment 17

[Kilik Kuo [:kikuo] (inactive)]

(In reply to Kilik Kuo [:kikuo] from comment #16)
> Try run,
> https://treeherder.mozilla.org/#/
> jobs?repo=try&revision=45a3683b7526&selectedJob=27613676
> 
> Some oranges on tier2, but from the log information, I don't think that's
> relevant to this modification.
> Another try run to make sure.
> 
> https://treeherder.mozilla.org/#/jobs?repo=try&revision=5289e01aef41

A try run WITHOUT any modification, https://treeherder.mozilla.org/#/jobs?repo=try&revision=c5481c8eec0f
I think it's ok to go !

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/4a5b6754da08
Shorten media files to make mochitest faster. r=cpearce

Comment 19

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/4a5b6754da08