Closed Bug 1657521 Opened 6 months ago Closed 5 months ago

Add hardware VP9 support on macos

Categories

(Core :: Audio/Video: Playback, enhancement)

x86_64
macOS
enhancement

Tracking

()

RESOLVED FIXED
81 Branch
Tracking Status
firefox81 --- fixed

People

(Reporter: jya, Assigned: jya)

References

(Blocks 1 open bug)

Details

Attachments

(6 files)

macOS has an API VTRegisterSupplementalVideoDecoderIfAvailable [1]

It takes a CMVideoCodecType which MacOS 11.0 (Big Spur) can now be kCMVideoCodecType_VP9 [3]

This use has been added in the latest webkit [4].

This gives us VP9 HW decoding capabilities on macOS.

[1] https://developer.apple.com/documentation/videotoolbox/3666591-vtregistersupplementalvideodecod?changes=latest_minor
[2] https://developer.apple.com/documentation/coremedia/cmvideocodectype?changes=latest_minor
[3] https://developer.apple.com/documentation/coremedia/kcmvideocodectype_vp9
[4] https://trac.webkit.org/changeset/264659/webkit

The mac VP9 decoder; like the H264 requires some out of band settings before it can be created.

This information is only found in the mp4 container, we can create it from the vp9 bitstream.

For now we ignore the colors information as we can't handle it properly yet in our compositor and this is not available in the bytestream.

Assignee: nobody → jyavenard
Status: NEW → ASSIGNED

To create a VP9 decoder, the VideoToolbox requires a vppC atom similar to how the H264 one requires an avcC one.

That information is typically not available in the webm container and is found in the VP9 bytestream with each keyframe.

In order to minimise the extent of the changes, we move the task of retrieving the vpcC content in the MediaChangeMonitor as it already performs a similar task in order to detect if the format has changed.

The VPXChangeMonitor will now only instantiate a VP9 decoder once a keyframe is seen.

Depends on D86543

The MediaChangeMonitor would always use the selected PDM in order to create a decoder; this only worked if the Decode method returned an error if the format was unsupported and this is how the WMF decoder worked.

However, the AppleVTDecoder fails on creation instead.
Now that the VP9 profile is known at creation time, we should move the WMF decoder to do the same.

Depends on D86544

Pushed by jyavenard@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/132eb437fdda
P1. Add method to generate a vpcC box from the VP9 bitstream. r=jolin
https://hg.mozilla.org/integration/autoland/rev/5c3c1ec039d2
P2. Attempt to use Apple VT decoder before FFmpeg. r=jolin
https://hg.mozilla.org/integration/autoland/rev/8d70f3cb5e3b
P3. Add VP9 HW decoder support on macOS 11 (Big Sur). r=jolin
https://hg.mozilla.org/integration/autoland/rev/e97937bf5e3a
P4. Attempt to create a decoder via the PDMFactory as fallback. r=jolin
https://hg.mozilla.org/integration/autoland/rev/a0f450666c5b
P5. Allow VP9 HW decoder in mac sandbox. r=haik

Backed out 5 changesets (bug 1657521) for assertion failure at Refs.h

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&resultStatus=pending%2Crunning%2Csuccess%2Ctestfailed%2Cbusted%2Cexception&searchStr=crashtest&fromchange=51b465c34f8aca7ed48752788546177ad15da28f&tochange=6a2b9be4ce0097d0d137e2184ec4576328ee292c&selectedTaskRun=SG16_jm2TH2AvYBUY2fKEg.0

Backout link: https://hg.mozilla.org/integration/autoland/rev/6e2551ab206bb2f30f65e12d6c619e48bb277ebd

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=312798340&repo=autoland&lineNumber=24634

...
...
...
[task 2020-08-12T06:26:06.846Z] 06:26:06     INFO -  08-12 07:18:49.808 I/Gecko   ( 2888):
[task 2020-08-12T06:26:06.846Z] 06:26:06     INFO -  08-12 07:18:49.808 I/Gecko   ( 2888): {"action":"log","time":1597213129808,"thread":null,"pid":null,"source":"reftest","level":"DEBUG","message":"[CONTENT] MakeProgress"}
[task 2020-08-12T06:26:06.846Z] 06:26:06     INFO -  08-12 07:18:49.810 D/ServiceAllocator( 2888): org.mozilla.gecko.process.GeckoChildProcessServices$tab0 updateBindings: FOREGROUND priority, 2 importance, 2 successful binds, 0 failed binds, 0 successful unbinds
[task 2020-08-12T06:26:06.846Z] 06:26:06     INFO -  08-12 07:18:49.812 I/Gecko   ( 2888):
[task 2020-08-12T06:26:06.847Z] 06:26:06     INFO -  08-12 07:18:49.812 I/Gecko   ( 2888): {"action":"log","time":1597213129812,"thread":null,"pid":null,"source":"reftest","level":"DEBUG","message":"[CONTENT] MakeProgress: STATE_WAITING_FOR_REFTEST_WAIT_REMOVAL"}
[task 2020-08-12T06:26:06.847Z] 06:26:06     INFO -  08-12 07:18:49.812 I/Gecko   ( 2888):
[task 2020-08-12T06:26:06.847Z] 06:26:06     INFO -  08-12 07:18:49.812 I/Gecko   ( 2888): {"action":"log","time":1597213129812,"thread":null,"pid":null,"source":"reftest","level":"DEBUG","message":"[CONTENT] MakeProgress"}
[task 2020-08-12T06:26:06.847Z] 06:26:06     INFO -  08-12 07:18:49.816 F/MOZ_Assert( 2921): Assertion failure: Ref::mInstance (Null jobject), at /builds/worker/workspace/obj-build/dist/include/mozilla/jni/Refs.h:231
[task 2020-08-12T06:26:06.847Z] 06:26:06     INFO -  08-12 07:18:49.816 F/MOZ_Assert( 2921): #01: ???[/data/app/org.mozilla.geckoview.test-1/lib/x86_64/libxul.so +0xfe341f]
[task 2020-08-12T06:26:06.847Z] 06:26:06     INFO -  08-12 07:18:49.816 F/MOZ_Assert( 2921): #02: ???[/data/app/org.mozilla.geckoview.test-1/lib/x86_64/libxul.so +0xffdb15]
...
...
...
Flags: needinfo?(jyavenard)
Flags: needinfo?(jyavenard)

This is a partial revert of "Bug 1650996 - P3. Have RemoteDecoderManagerChild use a TaskQueue over a media threadpool."

The RemoteDecoderManagerChild dispatch sync synchronously, right now it is doing so on the media controller's thread pool ; however in the following patch it will change the creation to the decoder's thread pool.

If we attempt to instantiate too many decoders at once, we run out of thread and dead-lock in the sync dispatch.

This issue has bitten us in various places already and the solution was always assuming that the decoder will always be created on the controller's thread and used on the decoder's thread.
This assumption won't hold any longer and was difficult to keep anyway.

So we have the RemoteDecoderManagerChild uses a dedicated thread so that we can guarantee there will always be an available thread to create the decoder.

Depends on D86543

Attachment #9169151 - Attachment description: Bug 1657521 - P5. Allow VP9 HW decoder in mac sandbox. r?Alex_Gaynor → Bug 1657521 - P4. Allow VP9 HW decoder in mac sandbox. r?Alex_Gaynor
Attachment #9169149 - Attachment description: Bug 1657521 - P3. Add VP9 HW decoder support on macOS 11 (Big Sur). r?jolin → Bug 1657521 - P5. Add VP9 HW decoder support on macOS 11 (Big Sur). r?jolin
Attachment #9169150 - Attachment description: Bug 1657521 - P4. Attempt to create a decoder via the PDMFactory as fallback. r?jolin → Bug 1657521 - P6. Attempt to create a decoder via the PDMFactory as fallback. r?jolin
Pushed by jyavenard@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/99693abc4499
P1. Add method to generate a vpcC box from the VP9 bitstream. r=jolin
https://hg.mozilla.org/integration/autoland/rev/233bb07b9bb9
P2. Attempt to use Apple VT decoder before FFmpeg. r=jolin
https://hg.mozilla.org/integration/autoland/rev/d266ef2e9ad7
P3. Have the RemoteDecoderManagerChild use its own dedicated thread. r=mattwoodrow
https://hg.mozilla.org/integration/autoland/rev/df297d346db2
P4. Allow VP9 HW decoder in mac sandbox. r=haik
https://hg.mozilla.org/integration/autoland/rev/791287b95414
P5. Add VP9 HW decoder support on macOS 11 (Big Sur). r=jolin
https://hg.mozilla.org/integration/autoland/rev/d6c1978bcabe
P6. Attempt to create a decoder via the PDMFactory as fallback. r=jolin
Duplicate of this bug: 1660036
Blocks: 1648487
Depends on: 1660107
Regressions: 1672901
You need to log in before you can comment on or make changes to this bug.