1806995 - Perma dom/media/test/reftest/color_quads/720p.png.bt709.bt709.pc.yuv420p.vp9.webm == dom/media/test/reftest/color_quads/720p.png.bt709.bt709.pc.yuv420p.av1.webm | image comparison, max difference: 36, number of differing pixels: 187407

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: imoraru [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=400472950&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/WCObWSOWQIWRpO8Lc3dnLg/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/WCObWSOWQIWRpO8Lc3dnLg/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

---

''' INFO '''

The colors from test image are brighter than the colors from the reference image. Also there is a big part of the test image highlighted as different.

''' INFO '''

[task 2022-12-22T12:02:10.960Z] 12:02:10     INFO - REFTEST TEST-START | dom/media/test/reftest/color_quads/720p.png.bt709.bt709.pc.yuv420p.vp9.webm == dom/media/test/reftest/color_quads/720p.png.bt709.bt709.pc.yuv420p.av1.webm
[task 2022-12-22T12:02:10.960Z] 12:02:10     INFO - REFTEST TEST-LOAD | file:///opt/worker/tasks/task_1671710295/build/tests/reftest/tests/dom/media/test/reftest/reftest_video.html?src=color_quads/720p.png.bt709.bt709.pc.yuv420p.vp9.webm | 7 / 29 (24%)
[task 2022-12-22T12:02:11.146Z] 12:02:11     INFO - REFTEST TEST-LOAD | file:///opt/worker/tasks/task_1671710295/build/tests/reftest/tests/dom/media/test/reftest/reftest_video.html?src=color_quads/720p.png.bt709.bt709.pc.yuv420p.av1.webm | 7 / 29 (24%)
[task 2022-12-22T12:02:11.301Z] 12:02:11     INFO - REFTEST INFO | REFTEST fuzzy test (0, 0) <= (36, 187407) <= (16, 1874)
[task 2022-12-22T12:02:11.407Z] 12:02:11     INFO - REFTEST TEST-UNEXPECTED-FAIL | dom/media/test/reftest/color_quads/720p.png.bt709.bt709.pc.yuv420p.vp9.webm == dom/media/test/reftest/color_quads/720p.png.bt709.bt709.pc.yuv420p.av1.webm | image comparison, max difference: 36, number of differing pixels: 187407
[task 2022-12-22T12:02:11.407Z] 12:02:11     INFO - REFTEST   IMAGE 1 (TEST): data:image/png;base64,<...>
[task 2022-12-22T12:02:11.407Z] 12:02:11     INFO - REFTEST   IMAGE 2 (REFERENCE): data:image/png;base64,<...>
[task 2022-12-22T12:02:11.407Z] 12:02:11     INFO - REFTEST TEST-END | dom/media/test/reftest/color_quads/720p.png.bt709.bt709.pc.yuv420p.vp9.webm == dom/media/test/reftest/color_quads/720p.png.bt709.bt709.pc.yuv420p.av1.webm
[task 2022-12-22T12:02:11.407Z] 12:02:11     INFO - REFTEST TEST-START | dom/media/test/reftest/color_quads/720p.png.bt709.bt709.pc.yuv420p.h264.mp4 == dom/media/test/reftest/color_quads/720p.png.bt709.bt709.pc.yuv420p.av1.webm

Comment 1

[Iulian Moraru]

Hi Jim! Can you please take a look at this? I'm not really sure what caused this.
Thank you!

Comment 2

[Noemi Erli[:noemi_erli]]

R+B range: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&collapsedPushes=842382&resultStatus=pending%2Crunning%2Csuccess%2Ctestfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&fromchange=320e64207bcac5bb2d8880035e6649ebbff88ead&tochange=27a62fceb6f41ff6e7c6de2f8e4ac0ae8792a39d&searchStr=os%2Cx%2C11%2Cwebrender%2Cshippable%2Copt%2Creftests%2Ctest-macosx1100-64-shippable-qr%2Fopt-reftest%2Cr2&selectedTaskRun=I0bZhbOtSYC7TWvq_8FBzw.0
This shows that the fail started right with the landing of https://bugzilla.mozilla.org/show_bug.cgi?id=1806391

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1806391

Comment 4

[Intermittent Failures Robot]

17 failures in 2999 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 8
• try: 5
• mozilla-beta: 4

Platform and build breakdown:

• macosx1100-64-shippable-qr: 17
  • opt: 17

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1806995&startday=2022-12-19&endday=2022-12-25&tree=all

Comment 5

[Jim Mathies [:jimm]]

I wonder if this has something to do with the color work gfx folks have been doing?

Comment 6

[Intermittent Failures Robot]

32 failures in 1328 pushes (0.024 failures/push) were associated with this bug in the last 7 days.

This is the #17 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 12
• try: 9
• mozilla-beta: 11

Platform and build breakdown:

• macosx1100-64-shippable-qr: 32
  • opt: 32

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1806995&startday=2022-12-26&endday=2023-01-01&tree=all

Comment 7

[Intermittent Failures Robot]

15 failures in 536 pushes (0.028 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 2
• try: 2
• mozilla-beta: 3
• mozilla-esr102: 2
• mozilla-release: 6

Platform and build breakdown:

• macosx1100-64-shippable-qr: 9
  • opt: 9
• windows7-32-shippable-qr: 6
  • opt: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1806995&startday=2023-01-03&endday=2023-01-03&tree=all

Comment 8

[az [:az]]

I think that this can be fixed if I re-add the HW decode support check to the decoder object that was reverted as part of the patch referenced. It seems to work from this run, and I have a longer run going now with some additional tests. We haven't been able to repro the macOS crashes so I'm not certain that re-adding that check won't also cause the crash issue again, but I'll investigate more tomorrow.

Comment 9

[Intermittent Failures Robot]

51 failures in 2607 pushes (0.02 failures/push) were associated with this bug in the last 7 days.

This is the #15 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• try: 10
• mozilla-central: 18
• mozilla-beta: 13
• mozilla-esr102: 2
• mozilla-release: 8

Platform and build breakdown:

• macosx1100-64-shippable-qr: 45
  • opt: 45
• windows7-32-shippable-qr: 6
  • opt: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1806995&startday=2023-01-02&endday=2023-01-08&tree=all

Comment 10

[Kelsey Gilbert [:jgilbert]]

The failure differences look like full-vs-narrow range, or something like that. Maybe something in the decoders/glue are miscommunicating here.
Mac, so bradwerth maybe?

Comment 11

[Kelsey Gilbert [:jgilbert]]

Oh, I missed that :az has a lead on this, so nevermind!

Comment 12

[az [:az]]

Attachment: Bug 1806995 - Directly query macOS decoder HW acceleration status as part of HW support checks. r=jolin

Comment 13

[Intermittent Failures Robot]

32 failures in 3530 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

This is the #34 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 18
• try: 9
• mozilla-release: 3
• mozilla-esr102: 1
• mozilla-beta: 1

Platform and build breakdown:

• macosx1100-64-shippable-qr: 31
  • opt: 31
• windows7-32-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1806995&startday=2023-01-09&endday=2023-01-15&tree=all

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/bfc2d6024269
Directly query macOS decoder HW acceleration status as part of HW support checks. r=jolin

Comment 15

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/bfc2d6024269

Comment 16

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:az, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
• If no, please set status-firefox110 to wontfix.

For more information, please visit auto_nag documentation.

Comment 17

[Intermittent Failures Robot]

36 failures in 3717 pushes (0.01 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 9
• try: 6
• mozilla-beta: 17
• mozilla-release: 3
• mozilla-esr102: 1

Platform and build breakdown:

• macosx1100-64-shippable-qr: 35
  • opt: 35
• windows7-32-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1806995&startday=2023-01-16&endday=2023-01-22&tree=all

Comment 18

[Intermittent Failures Robot]

23 failures in 3914 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 17
• mozilla-release: 3
• mozilla-esr102: 3

Platform and build breakdown:

• macosx1100-64-shippable-qr: 20
  • opt: 20
• windows7-32-shippable-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1806995&startday=2023-01-23&endday=2023-01-29&tree=all

Comment 19

[Intermittent Failures Robot]

23 failures in 3606 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 16
• mozilla-esr102: 4
• mozilla-release: 2
• try: 1

Platform and build breakdown:

• macosx1100-64-shippable-qr: 19
  • opt: 19
• windows7-32-shippable-qr: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1806995&startday=2023-01-30&endday=2023-02-05&tree=all

Comment 20

[Intermittent Failures Robot]

7 failures in 3766 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 7

Platform and build breakdown:

• macosx1100-64-shippable-qr: 7
  • opt: 7

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1806995&startday=2023-02-06&endday=2023-02-12&tree=all

Comment 21

[Intermittent Failures Robot]

5 failures in 3856 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 1
• mozilla-esr102: 4

Platform and build breakdown:

• macosx1100-64-shippable-qr: 1
  • opt: 1
• windows7-32-shippable-qr: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1806995&startday=2023-02-13&endday=2023-02-19&tree=all

Comment 22

[Intermittent Failures Robot]

4 failures in 3643 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 4

Platform and build breakdown:

• macosx1100-64-shippable-qr: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1806995&startday=2023-02-20&endday=2023-02-26&tree=all

Comment 23

[Intermittent Failures Robot]

1 failures in 3752 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 1

Platform and build breakdown:

• macosx1100-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1806995&startday=2023-02-27&endday=2023-03-05&tree=all