[Wayland] Implement YUV to RGB GL shader to convert video frames on GPU
Categories
(Core :: Widget: Gtk, enhancement)
Tracking
()
People
(Reporter: stransky, Unassigned)
References
(Blocks 1 open bug)
Details
Recently video frames produced by libva (ffmepg) are decoded to YUV color space and needs to be transferred to RGB color space. This is done by CPU which is sub-optimal.
The task is to upload video frame as a texture to GPU and then convert
to RGB color space by OpenGL shader by GPU. Then the result can be used as a DMABuf surface / EGLImage.
|
Reporter | |
Comment 1•5 years ago
|
||
See https://bug649417.bmoattachments.org/attachment.cgi?id=531514 for reference.
|
||
Comment 2•5 years ago
|
||
From pure interest: how much more difficult would it be from here to let ffmpeg directly render to a DMABuf surface, using hardware decoding? It sounds so incredibly close already.
|
||
Comment 3•5 years ago
|
||
@stransky, are you working on vaapi decoding now? or is it just plan for the future?
|
|
||
Comment 4•5 years ago
|
||
@robert.mader, with ffmpeg there is not very a big difference between software decoding and vaapi decoding to dmabuf surface. Main work is infrastructure for dmabuf surfaces in firefox, which is in progress (https://bugzilla.mozilla.org/show_bug.cgi?id=1572697). I think we are close to hw video decoding on linux \o/
|
|
Reporter | |
Comment 5•5 years ago
|
||
This bug is about to allow WaylandDMABufSurface to import YUV surface produced by ffmpeg.
libva (ffmpeg) VA-API does not work on Wayland/Intel yet so I can't do any work here until it's supported.
|
||
Comment 6•5 years ago
|
||
Are you sure? I think it wasn't working a while ago, but mpv seems to be able to do this:
[vd] Looking at hwdec h264-vaapi-copy...
[vaapi] libva: VA-API version 1.5.0
[vaapi] libva: va_getDriverName() returns 0
[vaapi] libva: Trying to open /usr/lib/dri/i965_drv_video.so
[vaapi] libva: Found init function __vaDriverInit_1_3
[vaapi] libva: va_openDriver() returns 0
[vaapi] Initialized VAAPI: version 1.5
[vd] Trying hardware decoding via h264-vaapi-copy.
[vd] Pixel formats supported by decoder: vdpau cuda vaapi_vld yuv420p
[vd] Codec profile: High (0x64)
[vd] Requesting pixfmt 'vaapi_vld' from decoder.
[vd] Selected codec: h264 (H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10)
[vo/gpu] reconfig to 1280x546 vaapi[nv12] bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vo/gpu/wayland] Reconfiguring!
[vo/gpu] Resize: 1280x546
[vo/gpu] Window size: 1280x546
[vo/gpu] Video source: 1280x546 (1:1)
[vo/gpu] Video display: (0, 0) 1280x546 -> (0, 0) 1280x546
[vo/gpu] Video scale: 1.000000/1.000000
[vo/gpu] OSD borders: l=0 t=0 r=0 b=0
[vo/gpu] Video borders: l=0 t=0 r=0 b=0
[vo/gpu] Testing FBO format rgba16f
[vo/gpu] Using FBO format rgba16f.
[vo/gpu/wayland] Handling resize on the egl side
[vo/gpu] Resize: 1280x546
[vo/gpu] Window size: 1280x546
[vo/gpu] Video source: 1280x546 (1:1)
[vo/gpu] Video display: (0, 0) 1280x546 -> (0, 0) 1280x546
[vo/gpu] Video scale: 1.000000/1.000000
[vo/gpu] OSD borders: l=0 t=0 r=0 b=0
[vo/gpu] Video borders: l=0 t=0 r=0 b=0
[vo/gpu] Reported display depth: 8
It even works with the Vulkan backend.
|
|
Reporter | |
Comment 7•5 years ago
|
||
(In reply to Laurentiu Nicola from comment #6)
Are you sure? I think it wasn't working a while ago, but mpv seems to be able to do this:
Ok but this bug is really about YUV->RGB GPU conversion, not about va-api playback.
|
|
Reporter | |
Updated•5 years ago
|
Description
•