Add hardware VP9 support on macos
Categories
(Core :: Audio/Video: Playback, enhancement)
Tracking
()
Tracking | Status | |
---|---|---|
firefox81 | --- | fixed |
People
(Reporter: jya, Assigned: jya)
References
(Blocks 1 open bug)
Details
Attachments
(6 files)
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review |
macOS has an API VTRegisterSupplementalVideoDecoderIfAvailable [1]
It takes a CMVideoCodecType which MacOS 11.0 (Big Spur) can now be kCMVideoCodecType_VP9 [3]
This use has been added in the latest webkit [4].
This gives us VP9 HW decoding capabilities on macOS.
[1] https://developer.apple.com/documentation/videotoolbox/3666591-vtregistersupplementalvideodecod?changes=latest_minor
[2] https://developer.apple.com/documentation/coremedia/cmvideocodectype?changes=latest_minor
[3] https://developer.apple.com/documentation/coremedia/kcmvideocodectype_vp9
[4] https://trac.webkit.org/changeset/264659/webkit
Assignee | ||
Comment 1•4 years ago
|
||
The mac VP9 decoder; like the H264 requires some out of band settings before it can be created.
This information is only found in the mp4 container, we can create it from the vp9 bitstream.
For now we ignore the colors information as we can't handle it properly yet in our compositor and this is not available in the bytestream.
Updated•4 years ago
|
Assignee | ||
Comment 2•4 years ago
|
||
Depends on D86542
Assignee | ||
Comment 3•4 years ago
|
||
To create a VP9 decoder, the VideoToolbox requires a vppC atom similar to how the H264 one requires an avcC one.
That information is typically not available in the webm container and is found in the VP9 bytestream with each keyframe.
In order to minimise the extent of the changes, we move the task of retrieving the vpcC content in the MediaChangeMonitor as it already performs a similar task in order to detect if the format has changed.
The VPXChangeMonitor will now only instantiate a VP9 decoder once a keyframe is seen.
Depends on D86543
Assignee | ||
Comment 4•4 years ago
|
||
The MediaChangeMonitor would always use the selected PDM in order to create a decoder; this only worked if the Decode method returned an error if the format was unsupported and this is how the WMF decoder worked.
However, the AppleVTDecoder fails on creation instead.
Now that the VP9 profile is known at creation time, we should move the WMF decoder to do the same.
Depends on D86544
Assignee | ||
Comment 5•4 years ago
|
||
Depends on D86545
Comment 7•4 years ago
|
||
Backed out 5 changesets (bug 1657521) for assertion failure at Refs.h
Backout link: https://hg.mozilla.org/integration/autoland/rev/6e2551ab206bb2f30f65e12d6c619e48bb277ebd
Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=312798340&repo=autoland&lineNumber=24634
...
...
...
[task 2020-08-12T06:26:06.846Z] 06:26:06 INFO - 08-12 07:18:49.808 I/Gecko ( 2888):
[task 2020-08-12T06:26:06.846Z] 06:26:06 INFO - 08-12 07:18:49.808 I/Gecko ( 2888): {"action":"log","time":1597213129808,"thread":null,"pid":null,"source":"reftest","level":"DEBUG","message":"[CONTENT] MakeProgress"}
[task 2020-08-12T06:26:06.846Z] 06:26:06 INFO - 08-12 07:18:49.810 D/ServiceAllocator( 2888): org.mozilla.gecko.process.GeckoChildProcessServices$tab0 updateBindings: FOREGROUND priority, 2 importance, 2 successful binds, 0 failed binds, 0 successful unbinds
[task 2020-08-12T06:26:06.846Z] 06:26:06 INFO - 08-12 07:18:49.812 I/Gecko ( 2888):
[task 2020-08-12T06:26:06.847Z] 06:26:06 INFO - 08-12 07:18:49.812 I/Gecko ( 2888): {"action":"log","time":1597213129812,"thread":null,"pid":null,"source":"reftest","level":"DEBUG","message":"[CONTENT] MakeProgress: STATE_WAITING_FOR_REFTEST_WAIT_REMOVAL"}
[task 2020-08-12T06:26:06.847Z] 06:26:06 INFO - 08-12 07:18:49.812 I/Gecko ( 2888):
[task 2020-08-12T06:26:06.847Z] 06:26:06 INFO - 08-12 07:18:49.812 I/Gecko ( 2888): {"action":"log","time":1597213129812,"thread":null,"pid":null,"source":"reftest","level":"DEBUG","message":"[CONTENT] MakeProgress"}
[task 2020-08-12T06:26:06.847Z] 06:26:06 INFO - 08-12 07:18:49.816 F/MOZ_Assert( 2921): Assertion failure: Ref::mInstance (Null jobject), at /builds/worker/workspace/obj-build/dist/include/mozilla/jni/Refs.h:231
[task 2020-08-12T06:26:06.847Z] 06:26:06 INFO - 08-12 07:18:49.816 F/MOZ_Assert( 2921): #01: ???[/data/app/org.mozilla.geckoview.test-1/lib/x86_64/libxul.so +0xfe341f]
[task 2020-08-12T06:26:06.847Z] 06:26:06 INFO - 08-12 07:18:49.816 F/MOZ_Assert( 2921): #02: ???[/data/app/org.mozilla.geckoview.test-1/lib/x86_64/libxul.so +0xffdb15]
...
...
...
Comment 8•4 years ago
|
||
The following seem to start with the backed out changes:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=312799800&repo=autoland&lineNumber=26672
Assignee | ||
Updated•4 years ago
|
Assignee | ||
Comment 9•4 years ago
|
||
This is a partial revert of "Bug 1650996 - P3. Have RemoteDecoderManagerChild use a TaskQueue over a media threadpool."
The RemoteDecoderManagerChild dispatch sync synchronously, right now it is doing so on the media controller's thread pool ; however in the following patch it will change the creation to the decoder's thread pool.
If we attempt to instantiate too many decoders at once, we run out of thread and dead-lock in the sync dispatch.
This issue has bitten us in various places already and the solution was always assuming that the decoder will always be created on the controller's thread and used on the decoder's thread.
This assumption won't hold any longer and was difficult to keep anyway.
So we have the RemoteDecoderManagerChild uses a dedicated thread so that we can guarantee there will always be an available thread to create the decoder.
Depends on D86543
Updated•4 years ago
|
Updated•4 years ago
|
Updated•4 years ago
|
Comment 10•4 years ago
|
||
Comment 11•4 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/99693abc4499
https://hg.mozilla.org/mozilla-central/rev/233bb07b9bb9
https://hg.mozilla.org/mozilla-central/rev/d266ef2e9ad7
https://hg.mozilla.org/mozilla-central/rev/df297d346db2
https://hg.mozilla.org/mozilla-central/rev/791287b95414
https://hg.mozilla.org/mozilla-central/rev/d6c1978bcabe
Description
•