Bug 1629788 Comment 10 Edit History

Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.

(In reply to Jean-Yves Avenard [:jya] from comment #8)
> No, I believe this is the opposite, it's sw decoding with HW rendering; and useful across the line not just ffmpeg but also other video decoders (non-ffmpeg such as theora)
> 
> That may be useful in particular once usage becomes more broad and we have to start blacklisting HW decoders (it's almost certain this will happen, lots of HW decoder are buggy, return green frames or incorrectly decode some resolutions); we could instead use SW decoding and directly upload into a GPU surface.

Hi Jean-Yves,

I think there's a bit misunderstanding here. I surely agree that sw decoding + hw rendering is the way to go. But IIUC Jan is want to use dmabuf textures instead of shm textures which may be suboptimal. From my understanding the situation is:

ffmpeg -> cpu memory -> SW YUV -> RGB conversion -> upload to texture -> render

I'm not sure where and how SHM memory is used here, I suspect the RGB conversion result is stored as shm surface and transfered across processes and it's uploaded immediately as a texture just before rendering.

I think the optimal way may looks like:

ffmpeg -> cpu memory -> upload Y and UV textures as NV12 (or different) textures -> share as EGLImages -> do the rendering as with vaapi

OTOH dmabuf surfaces are really GPU pieces of memory. It's expensive/difficult to write into it from CPU. Also when we (firefox) writes directly to dmabuf we can't keep optimal texture format so the texture may be slow to render and may use more GPU memory than necessary. It's definitely better to keep GL implementation (or vaapi) to create and manage the GPU textures and use GL code to upload data to textures.

So this bug may be changed to "upload SW decoded frames to NV12 GL texture and do compositing from it directly", right?
(In reply to Jean-Yves Avenard [:jya] from comment #8)
> No, I believe this is the opposite, it's sw decoding with HW rendering; and useful across the line not just ffmpeg but also other video decoders (non-ffmpeg such as theora)
> 
> That may be useful in particular once usage becomes more broad and we have to start blacklisting HW decoders (it's almost certain this will happen, lots of HW decoder are buggy, return green frames or incorrectly decode some resolutions); we could instead use SW decoding and directly upload into a GPU surface.

Hi Jean-Yves,

I think there's a bit misunderstanding here. I surely agree that sw decoding + hw rendering is the way to go. But IIUC Jan wants to use dmabuf textures instead of shm textures which may be suboptimal. From my understanding the situation is:

ffmpeg -> cpu memory -> SW YUV -> RGB conversion -> upload to texture -> render

I'm not sure where and how SHM memory is used here, I suspect the RGB conversion result is stored as shm surface and transfered across processes and it's uploaded immediately as a texture just before rendering.

I think the optimal way may looks like:

ffmpeg -> cpu memory -> upload Y and UV textures as NV12 (or different) textures -> share as EGLImages -> do the rendering as with vaapi

OTOH dmabuf surfaces are really GPU pieces of memory. It's expensive/difficult to write into it from CPU. Also when we (firefox) writes directly to dmabuf we can't keep optimal texture format so the texture may be slow to render and may use more GPU memory than necessary. It's definitely better to keep GL implementation (or vaapi) to create and manage the GPU textures and use GL code to upload data to textures.

So this bug may be changed to "upload SW decoded frames to NV12 GL texture and do compositing from it directly", right?

Back to Bug 1629788 Comment 10