Closed Bug 1717078 Opened 3 years ago Closed 3 years ago

Permanent Late Beta and Release dom/media/test/reftest/color_quads/720p.png.bt709.bt709.tv.yuv420p.av1.webm == dom/media/test/reftest/color_quads/720p.png | image comparison, max difference: 255, number of differing pixels: 273481

Categories

(Core :: Audio/Video: Playback, defect, P5)

defect

Tracking

()

RESOLVED FIXED
93 Branch
Tracking Status
firefox-esr78 --- unaffected
firefox-esr91 --- fixed
firefox91 --- fixed
firefox92 --- fixed
firefox93 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: jgilbert)

References

(Regression)

Details

(Keywords: intermittent-failure, regression, Whiteboard: [stockwell unknown])

Attachments

(2 files)

Filed by: malexandru [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=343032167&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/D6Hf4_u_TxSPg6-7DYDpCQ/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/D6Hf4_u_TxSPg6-7DYDpCQ/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1


[task 2021-06-17T11:54:25.459Z] 11:54:25     INFO - REFTEST TEST-START | layout/reftests/layers/opacity-keep-intermediate-surface-too-long.html == layout/reftests/layers/opacity-keep-intermediate-surface-too-long-ref.html
[task 2021-06-17T11:54:25.462Z] 11:54:25     INFO - REFTEST TEST-LOAD | file:///C:/tasks/task_1623925205/build/tests/reftest/tests/layout/reftests/layers/opacity-keep-intermediate-surface-too-long.html | 29 / 30 (96%)
[task 2021-06-17T11:54:25.559Z] 11:54:25     INFO - REFTEST TEST-LOAD | file:///C:/tasks/task_1623925205/build/tests/reftest/tests/layout/reftests/layers/opacity-keep-intermediate-surface-too-long-ref.html | 29 / 30 (96%)
[task 2021-06-17T11:54:25.589Z] 11:54:25     INFO - REFTEST TEST-PASS | layout/reftests/layers/opacity-keep-intermediate-surface-too-long.html == layout/reftests/layers/opacity-keep-intermediate-surface-too-long-ref.html | image comparison, max difference: 0, number of differing pixels: 0
[task 2021-06-17T11:54:25.593Z] 11:54:25     INFO - REFTEST TEST-END | layout/reftests/layers/opacity-keep-intermediate-surface-too-long.html == layout/reftests/layers/opacity-keep-intermediate-surface-too-long-ref.html
[task 2021-06-17T11:54:25.605Z] 11:54:25     INFO - REFTEST INFO | Slowest test took 166ms (file:///C:/tasks/task_1623925205/build/tests/reftest/tests/layout/reftests/layers/forced-bg-color-outside-visible-region.html)
[task 2021-06-17T11:54:25.609Z] 11:54:25     INFO - REFTEST INFO | Total canvas count = 3
[task 2021-06-17T11:54:25.693Z] 11:54:25     INFO - JavaScript error: resource:///modules/Interactions.jsm, line 347: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
[task 2021-06-17T11:54:25.954Z] 11:54:25     INFO - 1623930865948	Marionette	TRACE	Received observer notification xpcom-will-shutdown
[task 2021-06-17T11:54:25.958Z] 11:54:25     INFO - 1623930865948	Marionette	INFO	Stopped listening on port 2828
[task 2021-06-17T11:54:25.961Z] 11:54:25     INFO - 1623930865948	Marionette	DEBUG	Marionette stopped listening
[task 2021-06-17T11:54:26.273Z] 11:54:26     INFO - REFTEST INFO | Process mode: e10s
[task 2021-06-17T11:54:26.275Z] 11:54:26  WARNING - leakcheck | refcount logging is off, so leaks can't be detected!
[task 2021-06-17T11:54:26.278Z] 11:54:26     INFO - REFTEST INFO | Running tests in file:///C:/tasks/task_1623925205/build/tests/reftest/tests/dom/media/test/reftest/color_quads/reftest.list
[task 2021-06-17T11:54:26.365Z] 11:54:26     INFO - REFTEST INFO | Running with e10s: True
[task 2021-06-17T11:54:26.367Z] 11:54:26     INFO - REFTEST INFO | Running with fission: False
[task 2021-06-17T11:54:26.371Z] 11:54:26     INFO - REFTEST INFO | Application command: C:\tasks\task_1623925205\build\application\firefox\firefox.exe -marionette --wait-for-browser -profile C:\Users\testdroid.YOGA-035.000\AppData\Local\Temp\tmpe_rjtx2u.mozrunner
[task 2021-06-17T11:54:27.068Z] 11:54:27     INFO - 1623930867065	Marionette	INFO	Marionette enabled
[task 2021-06-17T11:54:27.144Z] 11:54:27     INFO - 1623930867140	Marionette	TRACE	Received observer notification toplevel-window-ready
[task 2021-06-17T11:54:28.169Z] 11:54:28     INFO - console.warn: SearchSettings: "get: No settings file exists, new profile?" (new NotFoundError("Could not open the file at C:\\Users\\testdroid.YOGA-035.000\\AppData\\Local\\Temp\\tmpe_rjtx2u.mozrunner\\search.json.mozlz4", (void 0)))
[task 2021-06-17T11:54:29.059Z] 11:54:29     INFO - 1623930869052	Marionette	TRACE	Received observer notification marionette-startup-requested
[task 2021-06-17T11:54:29.063Z] 11:54:29     INFO - 1623930869052	Marionette	TRACE	Waiting until startup recorder finished recording startup scripts...
[task 2021-06-17T11:54:29.083Z] 11:54:29     INFO - 1623930869075	Marionette	TRACE	All scripts recorded.
[task 2021-06-17T11:54:29.087Z] 11:54:29     INFO - 1623930869078	Marionette	INFO	Listening on port 2828
[task 2021-06-17T11:54:29.091Z] 11:54:29     INFO - 1623930869078	Marionette	DEBUG	Marionette is listening
[task 2021-06-17T11:54:29.408Z] 11:54:29     INFO - 1623930869403	Marionette	DEBUG	Accepted connection 0 from 127.0.0.1:51996
[task 2021-06-17T11:54:29.412Z] 11:54:29     INFO - 1623930869407	Marionette	DEBUG	Closed connection 0
[task 2021-06-17T11:54:29.414Z] 11:54:29     INFO - 1623930869407	Marionette	DEBUG	Accepted connection 1 from 127.0.0.1:51998
[task 2021-06-17T11:54:29.423Z] 11:54:29     INFO - 1623930869418	Marionette	DEBUG	1 -> [0,1,"WebDriver:NewSession",{"strictFileInteractability":true}]
[task 2021-06-17T11:54:29.444Z] 11:54:29     INFO - 1623930869436	Marionette	DEBUG	1 <- [1,1,null,{"sessionId":"202d71e5-dc37-43df-b9f8-2ca3bf8a7800","capabilities":{"browserName":"firefox","browserVersion":"91.0a ... u.mozrunner","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true,"proxy":{}}}]
[task 2021-06-17T11:54:29.457Z] 11:54:29     INFO - 1623930869452	Marionette	DEBUG	1 -> [0,2,"Addon:Install",{"path":"C:\\tasks\\task_1623925205\\build\\tests\\reftest\\reftest","temporary":true}]
[task 2021-06-17T11:54:29.489Z] 11:54:29     INFO - 1623930869485	Marionette	TRACE	Received observer notification toplevel-window-ready
[task 2021-06-17T11:54:29.492Z] 11:54:29     INFO - 1623930869487	Marionette	DEBUG	1 <- [1,2,null,{"value":"reftest@mozilla.org"}]
[task 2021-06-17T11:54:29.503Z] 11:54:29     INFO - 1623930869498	Marionette	DEBUG	1 -> [0,3,"WebDriver:DeleteSession",{}]
[task 2021-06-17T11:54:29.506Z] 11:54:29     INFO - 1623930869501	Marionette	DEBUG	1 <- [1,3,null,{"value":null}]
[task 2021-06-17T11:54:29.580Z] 11:54:29     INFO - 1623930869576	Marionette	DEBUG	Closed connection 1
[task 2021-06-17T11:54:29.699Z] 11:54:29     INFO - REFTEST TEST-START | dom/media/test/reftest/color_quads/720p.png.bt709.bt709.tv.yuv420p.av1.webm == dom/media/test/reftest/color_quads/720p.png
[task 2021-06-17T11:54:29.712Z] 11:54:29     INFO - REFTEST TEST-LOAD | file:///C:/tasks/task_1623925205/build/tests/reftest/tests/dom/media/test/reftest/reftest_video.html?src=color_quads/720p.png.bt709.bt709.tv.yuv420p.av1.webm | 0 / 30 (0%)
[task 2021-06-17T11:54:30.046Z] 11:54:30     INFO - [Child 10388, MediaDecoderStateMachine #1] WARNING: Decoder=1de858e4c00 Decode error: NS_ERROR_DOM_MEDIA_FATAL_ERR (0x806e0005) - Error no decoder found for video/av1: file /builds/worker/checkouts/gecko/dom/media/MediaDecoderStateMachine.cpp:3530
[task 2021-06-17T11:54:30.062Z] 11:54:30     INFO - REFTEST INFO | drawWindow flags = DRAWWINDOW_DRAW_CARET | DRAWWINDOW_DRAW_VIEW | DRAWWINDOW_USE_WIDGET_LAYERS; window size = 816,1039; test browser size = 800,1000
[task 2021-06-17T11:54:32.061Z] 11:54:32     INFO - REFTEST TEST-LOAD | file:///C:/tasks/task_1623925205/build/tests/reftest/tests/dom/media/test/reftest/reftest_img.html?src=color_quads/720p.png | 0 / 30 (0%)
[task 2021-06-17T11:54:32.160Z] 11:54:32     INFO - REFTEST INFO | REFTEST fuzzy test (16, 5234) <= (255, 273481) <= (38, 94640)
[task 2021-06-17T11:54:32.716Z] 11:54:32     INFO - REFTEST TEST-UNEXPECTED-FAIL | dom/media/test/reftest/color_quads/720p.png.bt709.bt709.tv.yuv420p.av1.webm == dom/media/test/reftest/color_quads/720p.png | image comparison, max difference: 255, number of differing pixels: 273481
[task 2021-06-17T11:54:32.716Z] 11:54:32     INFO - REFTEST   IMAGE 1 (TEST):
[task 2021-06-17T11:54:32.717Z] 11:54:32     INFO - REFTEST   IMAGE 2 (REFERENCE):
[task 2021-06-17T11:54:32.719Z] 11:54:32     INFO - REFTEST TEST-END | dom/media/test/reftest/color_quads/720p.png.bt709.bt709.tv.yuv420p.av1.webm == dom/media/test/reftest/color_quads/720p.png
[task 2021-06-17T11:54:32.723Z] 11:54:32     INFO - REFTEST TEST-START | dom/media/test/reftest/color_quads/720p.png.bt709.bt709.tv.yuv420p.vp9.webm == dom/media/test/reftest/color_quads/720p.png.bt709.bt709.tv.yuv420p.av1.webm
[task 2021-06-17T11:54:32.726Z] 11:54:32     INFO - REFTEST TEST-LOAD | file:///C:/tasks/task_1623925205/build/tests/reftest/tests/dom/media/test/reftest/reftest_video.html?src=color_quads/720p.png.bt709.bt709.tv.yuv420p.vp9.webm | 1 / 30 (3%)
[task 2021-06-17T11:54:33.167Z] 11:54:33     INFO - REFTEST TEST-UNEXPECTED-FAIL | dom/media/test/reftest/color_quads/720p.png.bt709.bt709.tv.yuv420p.vp9.webm == dom/media/test/reftest/color_quads/720p.png.bt709.bt709.tv.yuv420p.av1.webm | image comparison, max difference: 255, number of differing pixels: 273482
See Also: → 1718403
Has Regression Range: --- → yes
Flags: needinfo?(jgilbert)
Assignee: nobody → jgilbert
Flags: needinfo?(jgilbert)
Pushed by jgilbert@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/30fefff6d25e
Fix test markings. r=gfx-reviewers,lsalzman
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → 91 Branch
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Target Milestone: 91 Branch → ---
See Also: → 1718621

Hi Jeff, can you please take a look?

Flags: needinfo?(jgilbert)
Whiteboard: [stockwell needswork:owner]

Update:
There have been 47 failures within the last 7 days:
• 1 failures on Android 7.0 x86-64 WebRender opt
• 36 failures on Linux 18.04 x64 WebRender debug
• 7 failures on OS X 10.15 WebRender debug
• 1 failures on Windows 10 x64 WebRender debug
• 2 failures on windows10-aarch64-qr opt
Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=345064305&repo=autoland&lineNumber=24296

'''INFO'''

The bottom of the image is highlighted as different on multiple rows.

'''INFO'''

This is failing permanently for release and late beta (example). Jeff, what's the next step here?

Summary: Intermittent dom/media/test/reftest/color_quads/720p.png.bt709.bt709.tv.yuv420p.av1.webm == dom/media/test/reftest/color_quads/720p.png | image comparison, max difference: 255, number of differing pixels: 273481 → Permanent Late Beta and Release dom/media/test/reftest/color_quads/720p.png.bt709.bt709.tv.yuv420p.av1.webm == dom/media/test/reftest/color_quads/720p.png | image comparison, max difference: 255, number of differing pixels: 273481

It just needs to be marked properly.

Flags: needinfo?(jgilbert)

It looks like it's because AV1 is (maybe?) disabled outside of Nightly.

Otherwise, outside of Nightly, av1 will be disabled on Android, and
these tests will be both orange and not useful.

Pushed by jgilbert@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/c83998e348e7
Enable av1 pref for color_quads tests. r=gfx-reviewers,jrmuizel
Status: REOPENED → RESOLVED
Closed: 3 years ago3 years ago
Resolution: --- → FIXED
Target Milestone: --- → 93 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: