Open Bug 1753129 Opened 2 years ago Updated 16 days ago

Using Vulkan Video Decode API to hardware decoding

Categories

(Core :: Audio/Video: Playback, enhancement)

enhancement

Tracking

()

UNCONFIRMED

People

(Reporter: coelacanthushex, Unassigned)

Details

User Agent: Mozilla/5.0 (Android 11; Mobile; rv:98.0) Gecko/98.0 Firefox/98.0

Steps to reproduce:

https://www.khronos.org/registry/vulkan/specs/1.3-extensions/html/chap39.html
https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/chap39.html
https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/chap39.html
Vulkan has an extension to video decoding, it can be used to implement hardware decoding in Firefox as an optional method.
Nvidia driver (Windows 458.17, Linux 455.50.12, since 2021-04-03) has implemented Video Decoding API.
https://developer.nvidia.com/vulkan-driver
It can be a better universal replacement than proprietary Nvidia vdpau API.
This method can be used for intel, AMD and Nvidia graphic Card once they implement this extension, as NVIDIA has done. And it can be used in both Windows, Linux and macOS, because Vulkan is available in all three platforms.

Actual results:

There's no hardware decoding way through Vulkan Video Decoding API extension.

Expected results:

Make hardware decoding through Vulkan Video Decoding API implemented.

ffmpeg is working on it, this is their branch https://github.com/cyanreg/FFmpeg/commits/vulkan_decode . It maybe a example to refer to.
And as MPV developer said ( https://github.com/mpv-player/mpv/issues/9675#issuecomment-1005623063 ), this feature is only added in Nvidia Vulkan Beta driver. It was my mistake to say it has been added to the stable version.

And there is an example to use this feature. https://github.com/nvpro-samples/vk_video_samples

Mesa's Intel Vulkan driver is working on implement this extension, this is their PR. https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13940

Mesa has merged PR that make AMD support Vulkan video decode. It will be a part of mesa 23.1.
https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20388

I'd like to highlight the recent release of FFmpeg version 6.1, which introduces support for Vulkan Video Decoding/Encoding. This could significantly enhance the multimedia capabilities of Firefox.
https://ffmpeg.org/index.html#pr6.1

mesa mainlined VK_KHR_video_decode_av1 extension support in radv (amd vulkan driver). Would be available for mesa 24.1.

MESA v24.0.4- ANV(Intel) supports these extensions as of now:

❯ vulkaninfo | grep video
MESA-INTEL: warning: cannot initialize blitter engine
	VK_KHR_video_decode_h264                    : extension revision 9
	VK_KHR_video_decode_h265                    : extension revision 8
	VK_KHR_video_decode_queue                   : extension revision 8
	VK_KHR_video_queue                          : extension revision 8
			videoCodecOperations:
			videoCodecOperations: count = 2
You need to log in before you can comment on or make changes to this bug.