Perma [tier 2] dom/media/test/test_can_play_type_mpeg.html | video/mp4; codecs="av1"='probably' - got "", expected "probably"

RESOLVED FIXED in Firefox 68

Status

()

defect
P5
normal
RESOLVED FIXED
3 months ago
2 months ago

People

(Reporter: intermittent-bug-filer, Assigned: mjf)

Tracking

(Regression, {intermittent-failure, regression})

unspecified
mozilla68
Points:
---

Firefox Tracking Flags

(firefox-esr60 unaffected, firefox67 unaffected, firefox68 fixed)

Details

(Whiteboard: [retriggered][stockwell disabled][cia:tv:2019])

Attachments

(1 attachment, 1 obsolete attachment)

Filed by: ccoroiu [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=244503057&repo=autoland
Full log: https://queue.taskcluster.net/v1/task/dTeOnCtjSOKOxAqmkCrPYg/runs/0/artifacts/public/logs/live_backing.log


15:05:22 INFO - 147 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | VP9 in MP4 should be supported in MSE
15:05:22 INFO - Buffered messages finished
15:05:22 INFO - 148 INFO TEST-UNEXPECTED-FAIL | dom/media/test/test_can_play_type_mpeg.html | video/mp4; codecs="av1"='probably' - got "", expected "probably"
15:05:22 INFO - SimpleTest.is@SimpleTest/SimpleTest.js:320:16
15:05:22 INFO - check@dom/media/test/test_can_play_type_mpeg.html:25:5
15:05:22 INFO - check_mp4@dom/media/test/test_can_play_type_mpeg.html:99:3
15:05:22 INFO - @dom/media/test/test_can_play_type_mpeg.html:155:1

I will take a look.

Flags: needinfo?(mfroman)

Thank you Michael.

Assignee: nobody → mfroman

RDD/AV1 isn't enabled by default yet for Android.

Pushed by mfroman@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/2996e082682c
don't expect av1 decoding for Android. r=bwc
Status: NEW → RESOLVED
Closed: 2 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla68

This is still a perma: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&searchStr=android%2C8.0%2Cpixel2%2Cpgo%2Cmochitests%2Cwithout%2Ce10s%2Ctest-android-hw-p2-8-0-arm7-api-16%2Fpgo-mochitest-media-1proc-1%2Cm-1proc%28mda1%29&selectedJob=245276631

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=245276631&repo=mozilla-central&lineNumber=3156

05:51:20     INFO -  103 INFO TEST-START | dom/media/test/test_can_play_type_mpeg.html
05:51:20     INFO -  Buffered messages logged at 05:51:17
05:51:20     INFO -  104 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | A valid string reason is expected
05:51:20     INFO -  105 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | Reason cannot be empty
05:51:20     INFO -  106 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | video/mp4='maybe'
05:51:20     INFO -  107 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | video/x-m4v='maybe'
05:51:20     INFO -  108 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | audio/mp4='maybe'
05:51:20     INFO -  109 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | audio/x-m4a='maybe'
05:51:20     INFO -  110 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | audio/m4a=''
05:51:20     INFO -  111 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | video/m4v=''
05:51:20     INFO -  112 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | audio/aac='maybe'
05:51:20     INFO -  113 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | audio/aacp='maybe'
05:51:20     INFO -  114 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | video/mp4; codecs="avc1.42E01E, mp4a.40.2"='probably'
05:51:20     INFO -  115 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | video/mp4; codecs="avc1.42E01E, mp3"='probably'
05:51:20     INFO -  116 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | video/mp4; codecs="avc1.42001E, mp4a.40.2"='probably'
05:51:20     INFO -  117 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | video/mp4; codecs="avc1.58A01E, mp4a.40.2"='probably'
05:51:20     INFO -  118 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | video/mp4; codecs="avc1.4D401E, mp4a.40.2"='probably'
05:51:20     INFO -  119 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | video/mp4; codecs="avc1.4D401F, mp4a.40.2"='probably'
05:51:20     INFO -  120 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | video/mp4; codecs="avc1.4D4028, mp4a.40.2"='probably'
05:51:20     INFO -  121 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | video/mp4; codecs="avc1.64001E, mp4a.40.2"='probably'
05:51:20     INFO -  122 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | video/mp4; codecs="avc1.64001F, mp4a.40.2"='probably'
05:51:20     INFO -  123 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | video/mp4; codecs="avc1.42E01E"='probably'
05:51:20     INFO -  124 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | video/mp4; codecs="avc1.42001E"='probably'
05:51:20     INFO -  125 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | video/mp4; codecs="avc1.58A01E"='probably'
05:51:20     INFO -  126 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | video/mp4; codecs="avc1.4D401E"='probably'
05:51:20     INFO -  127 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | video/mp4; codecs="avc1.64001F"='probably'
05:51:20     INFO -  128 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | audio/mp4; codecs="mp4a.40.2"='probably'
05:51:20     INFO -  129 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | audio/mp4; codecs=mp4a.40.2='probably'
05:51:20     INFO -  130 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | audio/x-m4a; codecs="mp4a.40.2"='probably'
05:51:20     INFO -  131 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | audio/x-m4a; codecs=mp4a.40.2='probably'
05:51:20     INFO -  132 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | audio/mp4; codecs="mp4a.40.2,"=''
05:51:20     INFO -  133 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | audio/mp4; codecs="mp4a.40.5"='probably'
05:51:20     INFO -  134 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | audio/mp4; codecs=mp4a.40.5='probably'
05:51:20     INFO -  135 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | audio/x-m4a; codecs="mp4a.40.5"='probably'
05:51:20     INFO -  136 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | audio/x-m4a; codecs=mp4a.40.5='probably'
05:51:20     INFO -  137 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | audio/mp4; codecs="mp4a.40.29"='probably'
05:51:20     INFO -  138 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | audio/mp4; codecs="opus"='probably'
05:51:20     INFO -  139 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | audio/mp4; codecs=opus='probably'
05:51:20     INFO -  140 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | audio/mp4; codecs="flac"='probably'
05:51:20     INFO -  141 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | audio/mp4; codecs=flac='probably'
05:51:20     INFO -  142 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | video/mp4; codecs=vp9='probably'
05:51:20     INFO -  143 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | VP9 in MP4 should be supported in MSE
05:51:20     INFO -  144 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | video/mp4; codecs="vp9"='probably'
05:51:20     INFO -  145 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | VP9 in MP4 should be supported in MSE
05:51:20     INFO -  146 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | video/mp4; codecs="vp9.0"='probably'
05:51:20     INFO -  147 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | VP9 in MP4 should be supported in MSE
05:51:20     INFO -  Buffered messages finished
05:51:20     INFO -  148 INFO TEST-UNEXPECTED-FAIL | dom/media/test/test_can_play_type_mpeg.html | video/mp4; codecs="av1"='' - got "probably", expected ""
05:51:20     INFO -      SimpleTest.is@SimpleTest/SimpleTest.js:320:16
05:51:20     INFO -      check@dom/media/test/test_can_play_type_mpeg.html:25:5
05:51:20     INFO -      check_mp4@dom/media/test/test_can_play_type_mpeg.html:101:5
05:51:20     INFO -      @dom/media/test/test_can_play_type_mpeg.html:164:1
Flags: needinfo?(mfroman)

Backed out changeset 2996e082682c (Bug 1548858) for mda failures at dom/media/test/test_can_play_type_mpeg.html

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=2996e082682ca6418ea92337eb2d12b6b86fd95b

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=245284804&repo=autoland&lineNumber=3181

Backout link: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&revision=37bf1a4ee6bcaa9e0142db6eabd9cd5d6b0f03c4

07:59:03     INFO -  147 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | VP9 in MP4 should be supported in MSE
07:59:03     INFO -  Buffered messages finished
07:59:03     INFO -  148 INFO TEST-UNEXPECTED-FAIL | dom/media/test/test_can_play_type_mpeg.html | video/mp4; codecs="av1"='' - got "probably", expected ""
07:59:03     INFO -      SimpleTest.is@SimpleTest/SimpleTest.js:320:16
07:59:03     INFO -      check@dom/media/test/test_can_play_type_mpeg.html:25:5
07:59:03     INFO -      check_mp4@dom/media/test/test_can_play_type_mpeg.html:101:5
07:59:03     INFO -      @dom/media/test/test_can_play_type_mpeg.html:164:1
07:59:03     INFO -  149 INFO TEST-PASS | dom/media/test/test_can_play_type_mpeg.html | audio/mpeg='maybe'

Started to be a perma again after: https://bugzilla.mozilla.org/show_bug.cgi?id=1536126 got landed

https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=pending%2Crunning%2Csuperseded%2Csuccess%2Ctestfailed%2Cbusted%2Cexception&fromchange=e86a75908fd3f530563496cee8777f0e64e3fd20&searchStr=android%2C8.0%2Cpixel2%2Cdebug%2Cmochitests%2Cwithout%2Ce10s%2Ctest-android-hw-p2-8-0-arm7-api-16%2Fdebug-mochitest-media-1proc-1%2Cm-1proc%28mda1%29

Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Target Milestone: mozilla68 → ---
Regressions: 1548446
No longer regressions: 1548446

:bc, this is a case where test-verify ran, but ran on a config where we don't normally run it (emulator) when it is only run on hardware. - can you put this on a list for test-verify work.

Flags: needinfo?(bob)

Pushed by ncsoregi@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/8deb4484f31c
skip test_can_play_type_mpeg.html on android r=jmaher

Keywords: checkin-needed
Keywords: leave-open
Whiteboard: [retriggered][stockwell disable-recommended] → [retriggered][stockwell disabled]
Flags: needinfo?(bob)
Whiteboard: [retriggered][stockwell disabled] → [retriggered][stockwell disabled]cia:tv:2019]
Whiteboard: [retriggered][stockwell disabled]cia:tv:2019] → [retriggered][stockwell disabled][cia:tv:2019]

This bug started when Bug 1548686 landed causing test_can_play_type_mpeg.html to fail, expecting "probably", got "".
I changed the test to look for Android and expect "".
Then Bug 1536126 landed, and this test started failing, expecting "", got "probably".
This resulted in the original patch for this bug to be backed out. So the test is back to expecting "probably" here[1]

Does the test really need to be disabled? It seems like at this point we're back to the original state. I did a try run off this morning's moz-central to see the current test results here[2].

[1] https://searchfox.org/mozilla-central/source/dom/media/test/test_can_play_type_mpeg.html#99
[2] https://treeherder.mozilla.org/#/jobs?repo=try&revision=ba233b1ecc41f8db52d99e70fcdae47a6dd2806e

Flags: needinfo?(mfroman)

Thanks for the comment :mjf.

this bug had 269 failures in the last week, is it possible that the fix has corrected the issue and we just had to wait for the fix to merge around?
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2019-05-03&endday=2019-05-10&tree=all&bug=1548858

I don't see failures in the last day, maybe we moved too soon on disabling.

:needinfo myself to followup on this later today or tomorrow- would be good to re-enable this once we are more certain it is fixed.

Flags: needinfo?(jmaher)

NI Cristina in case the test needs to be re-enabled.

Flags: needinfo?(ccoroiu)

this test is passing consistently, unfortunately dom/media/test/test_webvtt_seeking.html is failing in pgo mode- we should still re-enable this test.

Cristina can you backout https://hg.mozilla.org/integration/autoland/rev/8deb4484f31c when you get a chance?

Flags: needinfo?(jmaher)
Flags: needinfo?(ccoroiu)
Assignee: mfroman → nobody
Attachment #9063218 - Attachment is obsolete: true

I think this was fixed with a combination of backing out my original attempt to fix, and Bug 1536126 landing. I'm going to close this.

Status: REOPENED → RESOLVED
Closed: 2 months ago2 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla68
Assignee: nobody → mfroman
You need to log in before you can comment on or make changes to this bug.