1961723 - [HDR] Recycle wl_buffers if possible for direct composition

Status: RESOLVED FIXED

(Core :: Graphics: WebRender, enhancement)

Description

[Martin Stránský [:stransky] (ni? me)]

[HDR] Recycle wl_buffers if possible for video playback to save GPU cycles.

Comment 1

[Alan Liu]

Thanks for working on this issue. Here are the stacktraces which may related to firefox requesting compositor to create/delete wl_buffer on every video frame.
https://gist.github.com/alen6516/14fc0d0add2c926e0fe855836492d152

Comment 2

[Martin Stránský [:stransky] (ni? me)]

I think we may extend it to all dmabuf based wl_buffers used by native compositor.

Comment 3

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1961723 [Linux] Add DMABufSurface fields to allow cross process surface recycling r?emilio

Add mPID to hold origin of DMABufSurface creation.

Comment 4

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1961723 [Linux] Add WaylandBufferDMABUFHolder class to cache wl_buffer and allow WaylandBufferDMABUF to set wl_buffer from external source r?emilio

If DMABufSurface are recycled we don't want to create/delete wl_buffer for recycled frames as it eats GPU/power resources.
Instead store wl_buffers for DMABuf surfaces and re-use them if possible.
The behaviour is inspired by MPV which does the same.

Comment 5

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1961723 [Linux] Update WaylandSurface for recycled WaylandBuffers r?emilio

When recycled WaylandBuffer is detached/deleted, don't call wl_buffer_add_listener() as it throws protocol error but only update
WaylandBuffer reference by wl_proxy_set_user_data().

Comment 6

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1961723 [Linux] Rework FFmpegVideoFramePool to keep FFmpeg surface ID and DMABufSurface UID matched r?alwu

We want to recycle dmabuf surfaces and wl_buffers for decoded video frames so we need to make sure we always
use the same smabuf surface for particular ffmpeg frame.

Add mFFMPEGSurfaceID filed do VideoFrameSurface to keep track of assigned FFmpeg frame and also invalidate
all frames when video frames are flushed.

Add mHoldByFFmpeg wich ensured we don't reuse VideoFrameSurface untill is returned by genco rendering process
(with IsUsedByRenderer()).

Implement unified VideoFramePool<LIBAV_VER>::GetTargetVideoFrameSurface() method to get new VideoFrameSurface according to
requested configuration.

Comment 7

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1961723 [Linux] Invalidate FFmpeg frames if video stream is sought r?alwu

During seek FFmpeg may allocate new dmabuf frames so we need to remove ffmpeg ID's and allocate new surfaces.

Comment 8

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1961723 [Linux] Use WaylandBufferDMABUFHolder at NativeLayerWayland to hold wl_buffers for external dmabuf surfaces r?lsalzman

WaylandBufferDMABUFHolder stores wl_buffer for external dmabuf surface and allows us to use existing wl_buffer for dmabuf surface instead of creating
a new one. That saves GPU resources and GPU power (MPV does the same).

In this patch add WaylandBufferDMABUFHolder to NativeLayerRootWayland and cache wl_buffers there for surfaces which can be recycled.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/06ec04c554f6
[Linux] Add DMABufSurface fields to allow cross process surface recycling r=emilio
https://hg.mozilla.org/integration/autoland/rev/cff7e5dc3e0c
[Linux] Add WaylandBufferDMABUFHolder class to cache wl_buffer and allow WaylandBufferDMABUF to set wl_buffer from external source r=emilio
https://hg.mozilla.org/integration/autoland/rev/60eadd491fbd
[Linux] Update WaylandSurface for recycled WaylandBuffers r=emilio
https://hg.mozilla.org/integration/autoland/rev/9ea1e7698e4c
[Linux] Rework FFmpegVideoFramePool to keep FFmpeg surface ID and DMABufSurface UID matched r=alwu
https://hg.mozilla.org/integration/autoland/rev/ad4fbdad7b5e
[Linux] Invalidate FFmpeg frames if video stream is sought r=alwu
https://hg.mozilla.org/integration/autoland/rev/f62a9f8d52f0
[Linux] Use WaylandBufferDMABUFHolder at NativeLayerWayland to hold wl_buffers for external dmabuf surfaces r=lsalzman

Comment 10

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/06ec04c554f6
https://hg.mozilla.org/mozilla-central/rev/cff7e5dc3e0c
https://hg.mozilla.org/mozilla-central/rev/60eadd491fbd
https://hg.mozilla.org/mozilla-central/rev/9ea1e7698e4c
https://hg.mozilla.org/mozilla-central/rev/ad4fbdad7b5e
https://hg.mozilla.org/mozilla-central/rev/f62a9f8d52f0