Closed Bug 1361655 Opened 3 years ago Closed 3 years ago

Reftest for mediaplayback.

Categories

(Core :: Audio/Video: Playback, enhancement, P3)

enhancement

Tracking

()

RESOLVED FIXED
mozilla55
Tracking Status
firefox55 --- fixed

People

(Reporter: bechen, Assigned: bechen)

References

(Blocks 2 open bugs)

Details

Attachments

(1 file)

Per discussion, I want to leverage the reftest mechanism to verify the "content of mediaplayback".
So the first step is that try to write some code to verify the first/last frame of a video.
Priority: -- → P3
Comment on attachment 8864037 [details]
Bug 1361655 - Add new reftest to verify the first and last frame for short.mp4.

https://reviewboard.mozilla.org/r/135758/#review139884

::: dom/media/test/reftest/generateREF.html:1
(Diff revision 2)
> +<!DOCTYPE HTML>

Please document the usage of this file, when and how to use it.

::: dom/media/test/reftest/short.mp4.firstframe.html:14
(Diff revision 2)
> +  video.autoplay = false;
> +  video.preload = "metadata";
> +  video.addEventListener("loadeddata", function() {
> +    setTimeout(function() {
> +      document.documentElement.removeAttribute('class');
> +    }, 50);

Why waiting for 50m?
Comment on attachment 8864037 [details]
Bug 1361655 - Add new reftest to verify the first and last frame for short.mp4.

https://reviewboard.mozilla.org/r/135758/#review140898

::: dom/media/test/reftest/short.mp4.firstframe.html:7
(Diff revision 3)
> +<html class="reftest-wait">
> +<head>
> +<script type="text/javascript">
> +function doTest() {
> +  var video = document.getElementById("v1");
> +  video.src = "../short.mp4";

Try server shows error when runnign the testcase, it seems that the we don't receive the "loadeddata" event. I guess something wrong about the video.src.
Comment on attachment 8864037 [details]
Bug 1361655 - Add new reftest to verify the first and last frame for short.mp4.

https://reviewboard.mozilla.org/r/135758/#review140898

> Try server shows error when runnign the testcase, it seems that the we don't receive the "loadeddata" event. I guess something wrong about the video.src.

By adding "HTTP(..)" in reftest.list can solve the problem
Now we still have 2 issues:
1.
https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://archive.mozilla.org/pub/firefox/try-builds/bechen@mozilla.com-cab89f8c663c9efdfcbb7ec303a887e92bc76f7a/try-macosx64-debug/try_yosemite_r7-debug_test-reftest-e10s-2-bm133-tests1-macosx-build863.txt.gz&only_show_unexpected=1

It is the result runs at OSX.
The reftest analyzer tells me that we might need fuzzy because the differences are small or golden sample for each platform.

2. The first frame of short.mp4 is "1".
But on win7 platform, the result of the first frame is "2", not "1", something wrong.
Comment on attachment 8864037 [details]
Bug 1361655 - Add new reftest to verify the first and last frame for short.mp4.

https://reviewboard.mozilla.org/r/135758/#review142958

::: dom/media/test/reftest/generateREF.html:28
(Diff revision 4)
> +//window.onload = function() { setTimeout(dumpFirstFrameToConsole, 0); };
> +window.onload = function() { setTimeout(dumpLastFrameToConsole, 0); };
> +function dumpFirstFrameToConsole() {
> +  var video = document.getElementById("v1");
> +  video.src = "short.mp4";
> +  video.autoplay = false;

You don't need to set it to false for 'autoplay' defaults to false.

::: dom/media/test/reftest/generateREF.html:46
(Diff revision 4)
> +function dumpLastFrameToConsole() {
> +  var video = document.getElementById("v1");
> +  video.src = "short.mp4";
> +  video.autoplay = false;
> +  video.preload = "metadata";
> +  video.load();

No need to call load() since setting .src will trigger the load algorithm.

::: dom/media/test/reftest/generateREF.html:49
(Diff revision 4)
> +  video.autoplay = false;
> +  video.preload = "metadata";
> +  video.load();
> +  // Seek to the end
> +  video.addEventListener("loadeddata", function() {
> +    video.currentTime = video.duration;

This might not be a reliable way to get to the last frame for the duration may change later.

::: dom/media/test/reftest/generateREF.html:52
(Diff revision 4)
> +  // Seek to the end
> +  video.addEventListener("loadeddata", function() {
> +    video.currentTime = video.duration;
> +  });
> +  video.addEventListener("ended", function() {
> +    var canvas = document.getElementById("canvas");

There is some common code that can be extracted.

::: dom/media/test/reftest/short.mp4.firstframe.html:8
(Diff revision 4)
> +<head>
> +<script type="text/javascript">
> +function doTest() {
> +  var video = document.getElementById("v1");
> +  video.src = "../short.mp4";
> +  video.autoplay = false;

Ditto.

::: dom/media/test/reftest/short.mp4.firstframe.html:10
(Diff revision 4)
> +function doTest() {
> +  var video = document.getElementById("v1");
> +  video.src = "../short.mp4";
> +  video.autoplay = false;
> +  video.preload = "metadata";
> +  video.load();

Ditto.

::: dom/media/test/reftest/short.mp4.lastframe.html:13
(Diff revision 4)
> +  video.autoplay = false;
> +  video.preload = "metadata";
> +  video.load();
> +  // Seek to the end
> +  video.addEventListener("loadeddata", function() {
> +    video.currentTime = video.duration;

Ditto.
Comment on attachment 8864037 [details]
Bug 1361655 - Add new reftest to verify the first and last frame for short.mp4.

https://reviewboard.mozilla.org/r/135758/#review142958

> This might not be a reliable way to get to the last frame for the duration may change later.

Hmm, since the duration is to reliable, I am considering using video.seekToNextFrame() to get the last frame.
Please review the latest patch.
Flags: needinfo?(jwwang)
Comment on attachment 8864037 [details]
Bug 1361655 - Add new reftest to verify the first and last frame for short.mp4.

https://reviewboard.mozilla.org/r/135758/#review144968

::: dom/media/test/reftest/short.mp4.lastframe.html:20
(Diff revision 5)
> +      callSeekToNextFrame();
> +    };
> +  });
> +
> +  function callSeekToNextFrame() {
> +    video.seekToNextFrame().then(

See bug 1366362.
Since seekToNextFrame is broke, you should disable the test until it is fixed.
Attachment #8864037 - Flags: review?(jwwang) → review+
Flags: needinfo?(jwwang)
Depends on: 1366362
Keywords: checkin-needed
Pushed by ryanvm@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/c4b678956b29
Add new reftest to verify the first and last frame for short.mp4. r=jwwang
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/c4b678956b29
Status: NEW → RESOLVED
Closed: 3 years ago
Flags: in-testsuite+
Resolution: --- → FIXED
Target Milestone: --- → mozilla55
Blocks: 1368902
Blocks: 1368905
See Also: → 1371220
You need to log in before you can comment on or make changes to this bug.