1763781 - macOS HDR VP9 video playback should work without a hardware VP9 decoder

Status: VERIFIED FIXED

(Core :: Graphics, defect)

Description

[Alin Ilea, Desktop Test Engineering]

Attachment: Screenshot 2022-04-07 at 15.26.13.png

Affected Versions:

Nightly 101.0a1, Beta 100.0b3 (this feature was implemented in Fx100, in Release 99 it is disabled).

Tested on:

macOS 10.14.4 (iMac Retina 4k, 2019)
macOS 10.15.5 (MacBook Pro, 2017)

Preconditions:

Access: https://drive.google.com/drive/u/0/folders/1S-E4WeKO6y6UySHnSWvfAuCB7N6URdCj and download the files locally on your computer.

Steps to reproduce:

1. Launch Firefox.
2. Open a new tab and drag&drop any video saved in preconditions, from the computer to FF window.
3. Play the video.

Expected Results:

The video should play in HDR.

Actual Results:

The video is not playing in HDR mode.

Notes:

1. The same videos are playing in HDR accordingly on macOS 12, using the same iMac.
2. The same videos are properly displayed in HDR on Chrome, using the same iMac for 10.14 and the same MacBook for 10.15.
3. This could be S3-S4 severity for the browser, but S1-S2 for the feature, since it doesn't work at all on the mentioned OS-es.

Comment 1

[Brad Werth [:bradwerth]]

I've setup a machine with macOS 10.14 and I can reproduce this result. I need to setup that machine with a full development environment so I can debug the problem more thoroughly.

Comment 2

[Brad Werth [:bradwerth]]

Alin, I just found Apple's notes on HDR and it seems like they might have started support in macOS 10.15.4. I am still preparing a test machine, but I just wanted to check if your 10.15 testing was using version 10.15.4 or greater.

Comment 3

[Brad Werth [:bradwerth]]

I am testing now on macOS 10.15.7 and can still reproduce the issue. Once this Bug is fixed, re-testing will show the boundaries of macOS HDR support.

Comment 4

[Brad Werth [:bradwerth]]

Hmm... the AppleDecoderModule only claims to support VP9 when we can ensure it's hardware accelerated, which we can only do in macOS 11+. I'll see if we can relax this and use a software decode path if that's all we can get.

Comment 5

[Brad Werth [:bradwerth]]

Relying on the FFmpegVideoDecoder may be possible, if we avoid the shared memory path and include the fixes from Bug 1763782.

Comment 6

[Brad Werth [:bradwerth]]

Attachment: Bug 1763781 Part 1: Force FFmpegVideoDecoder to use MacIOSurfaceImage for HDR video.

This change ensures that macOS will correctly detect the surface as a video
and use an HDR-capable display layer. Unfortunately, it invokes a slower code
path for this case. Bug 1765388 may restore this fast path for HDR video.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/cdb79df7a017
Part 1: Force FFmpegVideoDecoder to use MacIOSurfaceImage for HDR video. r=media-playback-reviewers,alwu

Comment 8

[Marian-Vasile Laza]

Backed out changeset cdb79df7a017 (Bug 1763781) for causing wpt failures on canvas-display-p3-drawImage-ImageBitmap-video.html.
Backout link
Push with failures - wpt1
Failure Log

Comment 9

[Brad Werth [:bradwerth]]

Attachment: Bug 1763781 Part 2: Update test expectations.

Comment 10

[Brad Werth [:bradwerth]]

I'm having trouble with the landing. I'm going to rebase the patches and try again.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/98fd39c96de8
Part 1: Force FFmpegVideoDecoder to use MacIOSurfaceImage for HDR video. r=media-playback-reviewers,alwu
https://hg.mozilla.org/integration/autoland/rev/3b1422986294
Part 2: Update test expectations. r=jgilbert

Comment 12

[Atila Butkovits]

Backed out for causing wpt failures at canvas-display-p3-drawImage-ImageBitmap-video.html

Backout link: https://hg.mozilla.org/integration/autoland/rev/74b6acb9b236e42207c292fad2ae3f9252ca610a

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&selectedTaskRun=L4Hx1evyRqSGrm4D9GhFKQ.0&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=3b1422986294d2f38d6dee1e44635d38f015f793

Failure log: https://treeherder.mozilla.org/logviewer?job_id=375527057&repo=autoland&lineNumber=5265

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e0c5508d662d
Part 1: Force FFmpegVideoDecoder to use MacIOSurfaceImage for HDR video. r=media-playback-reviewers,alwu
https://hg.mozilla.org/integration/autoland/rev/ed2a721a38c5
Part 2: Update test expectations. r=jgilbert

Comment 14

[Noemi Erli[:noemi_erli]]

Backed out 2 changesets (Bug 1763781) for causing failures in canvas-display-p3-drawImage-ImageBitmap-video.html CLOSED TREE

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&collapsedPushes=842382&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=ed2a721a38c5b3f62d6bec4c93c025af079a383b&selectedTaskRun=QzJNEHPVTHeyA5BMy3zR9g.0

Log: https://treeherder.mozilla.org/logviewer?job_id=375705053&repo=autoland&lineNumber=5265

Backout: https://hg.mozilla.org/integration/autoland/rev/b75ffb54d9e94e302e4e1e7523971c62be4d2b91

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/41028dee28a0
Part 1: Force FFmpegVideoDecoder to use MacIOSurfaceImage for HDR video. r=media-playback-reviewers,alwu
https://hg.mozilla.org/integration/autoland/rev/3a6dbb3555cb
Part 2: Update test expectations. r=jgilbert

Comment 16

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/41028dee28a0
https://hg.mozilla.org/mozilla-central/rev/3a6dbb3555cb

Comment 17

[Brad Werth [:bradwerth]]

Comment on attachment 9272887 [details]
Bug 1763781 Part 1: Force FFmpegVideoDecoder to use MacIOSurfaceImage for HDR video.

Beta/Release Uplift Approval Request

• User impact if declined: macOS users with older hardware (2015) and HDR-capable screens, running macOS 11+ will have bad visuals with HDR video from YouTube
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Medium
• Why is the change risky/not risky? (and alternatives if risky): We are very late in the Release Candidate. The patch introduces an additional copy of frame data for the affected case, which might affect playback performance of high-resolution, high FPS video.
• String changes made/needed:
• Is Android affected?: No

Comment 18

[Dianna Smith [:diannaS]]

Patch https://phabricator.services.mozilla.com/D144228 doesnt graft cleanly on release.
Conflict while merging testing/web-platform/meta/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-video.html.ini

Comment 19

[Brad Werth [:bradwerth]]

(In reply to Dianna Smith [:diannaS] from comment #18)

Patch https://phabricator.services.mozilla.com/D144228 doesnt graft cleanly on release.
Conflict while merging testing/web-platform/meta/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-video.html.ini

Those test expectations were also affected by another recent landing -- which we don't want to uplift. If we will consider just landing D144039 and omitting the test expectations, the merge will work.

Comment 20

[Dianna Smith [:diannaS]]

Comment on attachment 9272887 [details]
Bug 1763781 Part 1: Force FFmpegVideoDecoder to use MacIOSurfaceImage for HDR video.

Approved for 100.0rc2

Comment 21

[Dianna Smith [:diannaS]]

https://hg.mozilla.org/releases/mozilla-release/rev/593471ee11be

Comment 22

[Brad Werth [:bradwerth]]

Attachment: Bug 1763781: Fx100 RC2 Update test expectations.

This is a modified version of D144228 that should merge to the Fx100 RC2.

This gives us better correctness for 10-bit vp9 reftests. In the WPTs, we
turn off some tests on macos because of the difficulty that our
screenshots have in analyzing srgb image data. It seems to consistently
report 191,255,191,73 when we are expecting 255,0,0,255.

Comment 23

[Brad Werth [:bradwerth]]

Comment on attachment 9274109 [details]
Bug 1763781: Fx100 RC2 Update test expectations.

Beta/Release Uplift Approval Request

• User impact if declined: It will be impossible to merge Part 1, D144039, to Fx100 RC2.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: Yes
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Fixes merge problems with test expectations that work in central.
• String changes made/needed:
• Is Android affected?: No

Comment 24

[Dianna Smith [:diannaS]]

Comment on attachment 9274109 [details]
Bug 1763781: Fx100 RC2 Update test expectations.

Approved for 100.0rc2

Comment 25

[Dianna Smith [:diannaS]]

https://hg.mozilla.org/releases/mozilla-release/rev/54538a6f60bb

Comment 26

[Alin Ilea, Desktop Test Engineering]

Hi,
In my attempt to verify this fix, I noticed that I can still reproduce the issue using macOS 10.14.4 (the same machine from the description). The videos are indeed rendered in HDR on macOS 10.15.6, but not on 10.14. Tested in the latest Nightly 101.0a1 (2022-04-28) and latest fx100.0 (build id: 20220425210429)

Comment 27

[Brad Werth [:bradwerth]]

I will investigate. I was not able to get a 10.14 machine setup with a development environment -- our build system essentially requires a 10.15+ machine to work. So I can downgrade my machine to 10.14 for testing, but my ability to debug on that machine is limited. My plan is to land Bug 1766343, which will make our gfx.core-animation.specialize-video.log logging more informative in this situation, and use that as the basis of the debugging effort.

Since this Bug has landed and has been uplifted around quite a bit, solving more problems than just the 10.15 compatibility issue, I'll open a new Bug for this remaining issue.

Comment 28

[Alin Ilea, Desktop Test Engineering]

It makes sense.
I can confirm that the issue is fixed in the latest Nightly 101.0a1 (2022-04-29) and Beta 100.0 (build id: 20220428192727) using macOS 10.15.6.
Will follow up with macOS 10.14 verification once bug 1766879 will be fixed.
Thanks!