Using Vulkan Video Decode API to hardware decoding
Categories
(Core :: Audio/Video: Playback, enhancement)
Tracking
()
People
(Reporter: coelacanthushex, Unassigned)
Details
User Agent: Mozilla/5.0 (Android 11; Mobile; rv:98.0) Gecko/98.0 Firefox/98.0
Steps to reproduce:
https://www.khronos.org/registry/vulkan/specs/1.3-extensions/html/chap39.html
https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/chap39.html
https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/chap39.html
Vulkan has an extension to video decoding, it can be used to implement hardware decoding in Firefox as an optional method.
Nvidia driver (Windows 458.17, Linux 455.50.12, since 2021-04-03) has implemented Video Decoding API.
https://developer.nvidia.com/vulkan-driver
It can be a better universal replacement than proprietary Nvidia vdpau API.
This method can be used for intel, AMD and Nvidia graphic Card once they implement this extension, as NVIDIA has done. And it can be used in both Windows, Linux and macOS, because Vulkan is available in all three platforms.
Actual results:
There's no hardware decoding way through Vulkan Video Decoding API extension.
Expected results:
Make hardware decoding through Vulkan Video Decoding API implemented.
ffmpeg is working on it, this is their branch https://github.com/cyanreg/FFmpeg/commits/vulkan_decode . It maybe a example to refer to.
And as MPV developer said ( https://github.com/mpv-player/mpv/issues/9675#issuecomment-1005623063 ), this feature is only added in Nvidia Vulkan Beta driver. It was my mistake to say it has been added to the stable version.
And there is an example to use this feature. https://github.com/nvpro-samples/vk_video_samples
Related(Chromium's feature request): https://bugs.chromium.org/p/chromium/issues/detail?id=1200559
Mesa's Intel Vulkan driver is working on implement this extension, this is their PR. https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13940
Mesa has merged PR that make AMD support Vulkan video decode. It will be a part of mesa 23.1.
https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20388
I'd like to highlight the recent release of FFmpeg version 6.1, which introduces support for Vulkan Video Decoding/Encoding. This could significantly enhance the multimedia capabilities of Firefox.
https://ffmpeg.org/index.html#pr6.1
|
||
Comment 7•8 months ago
|
||
It wasn't mentioned here that only H.264 and H.265 were supported.
Recently AV1 was introduced too:
https://www.khronos.org/blog/khronos-releases-vulkan-video-av1-decode-extension-vulkan-sdk-now-supports-h.264-h.265-encode
In case the blog link breaks eventually, the extension is:
https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VK_KHR_video_decode_av1.html
Mesa support:
mesa mainlined VK_KHR_video_decode_av1 extension support in radv (amd vulkan driver). Would be available for mesa 24.1.
|
||
Comment 9•6 months ago
|
||
MESA v24.0.4- ANV(Intel) supports these extensions as of now:
❯ vulkaninfo | grep video
MESA-INTEL: warning: cannot initialize blitter engine
VK_KHR_video_decode_h264 : extension revision 9
VK_KHR_video_decode_h265 : extension revision 8
VK_KHR_video_decode_queue : extension revision 8
VK_KHR_video_queue : extension revision 8
videoCodecOperations:
videoCodecOperations: count = 2
Description
•