Closed Bug 1580169 Opened 1 year ago Closed 9 months ago

[Wayland] Implement YUV to RGB GL shader to convert video frames on GPU

Categories

(Core :: Widget: Gtk, enhancement)

enhancement
Not set
normal

Tracking

()

RESOLVED DUPLICATE of bug 1610199

People

(Reporter: stransky, Unassigned)

References

(Blocks 1 open bug)

Details

Recently video frames produced by libva (ffmepg) are decoded to YUV color space and needs to be transferred to RGB color space. This is done by CPU which is sub-optimal.

The task is to upload video frame as a texture to GPU and then convert
to RGB color space by OpenGL shader by GPU. Then the result can be used as a DMABuf surface / EGLImage.

From pure interest: how much more difficult would it be from here to let ffmpeg directly render to a DMABuf surface, using hardware decoding? It sounds so incredibly close already.

@stransky, are you working on vaapi decoding now? or is it just plan for the future?

@robert.mader, with ffmpeg there is not very a big difference between software decoding and vaapi decoding to dmabuf surface. Main work is infrastructure for dmabuf surfaces in firefox, which is in progress (https://bugzilla.mozilla.org/show_bug.cgi?id=1572697). I think we are close to hw video decoding on linux \o/

This bug is about to allow WaylandDMABufSurface to import YUV surface produced by ffmpeg.
libva (ffmpeg) VA-API does not work on Wayland/Intel yet so I can't do any work here until it's supported.

Are you sure? I think it wasn't working a while ago, but mpv seems to be able to do this:

[vd] Looking at hwdec h264-vaapi-copy...
[vaapi] libva: VA-API version 1.5.0
[vaapi] libva: va_getDriverName() returns 0
[vaapi] libva: Trying to open /usr/lib/dri/i965_drv_video.so
[vaapi] libva: Found init function __vaDriverInit_1_3
[vaapi] libva: va_openDriver() returns 0
[vaapi] Initialized VAAPI: version 1.5
[vd] Trying hardware decoding via h264-vaapi-copy.
[vd] Pixel formats supported by decoder: vdpau cuda vaapi_vld yuv420p
[vd] Codec profile: High (0x64)
[vd] Requesting pixfmt 'vaapi_vld' from decoder.
[vd] Selected codec: h264 (H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10)
[vo/gpu] reconfig to 1280x546 vaapi[nv12] bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vo/gpu/wayland] Reconfiguring!
[vo/gpu] Resize: 1280x546
[vo/gpu] Window size: 1280x546
[vo/gpu] Video source: 1280x546 (1:1)
[vo/gpu] Video display: (0, 0) 1280x546 -> (0, 0) 1280x546
[vo/gpu] Video scale: 1.000000/1.000000
[vo/gpu] OSD borders: l=0 t=0 r=0 b=0
[vo/gpu] Video borders: l=0 t=0 r=0 b=0
[vo/gpu] Testing FBO format rgba16f
[vo/gpu] Using FBO format rgba16f.
[vo/gpu/wayland] Handling resize on the egl side
[vo/gpu] Resize: 1280x546
[vo/gpu] Window size: 1280x546
[vo/gpu] Video source: 1280x546 (1:1)
[vo/gpu] Video display: (0, 0) 1280x546 -> (0, 0) 1280x546
[vo/gpu] Video scale: 1.000000/1.000000
[vo/gpu] OSD borders: l=0 t=0 r=0 b=0
[vo/gpu] Video borders: l=0 t=0 r=0 b=0
[vo/gpu] Reported display depth: 8

It even works with the Vulkan backend.

(In reply to Laurentiu Nicola from comment #6)

Are you sure? I think it wasn't working a while ago, but mpv seems to be able to do this:

Ok but this bug is really about YUV->RGB GPU conversion, not about va-api playback.

Status: NEW → RESOLVED
Closed: 9 months ago
Resolution: --- → DUPLICATE
Duplicate of bug: egl-linux-vaapi
You need to log in before you can comment on or make changes to this bug.