1729743 - [Pipewire] Better DMABUF import support (upstream backport)

Status: RESOLVED DUPLICATE of bug 1783716

(Core :: WebRTC, enhancement)

Description

[Robert Mader [:rmader]]

Right now we unconditionally memcpy from the Pipewire source even if DMABUF sharing is used. This can be very slow, especially on dedicated cards. Inefficient usage of DMABUF sharing by clients even made Mutter (Gnome) disable DMABUF screensharing on such devices (it's currently only allowlisted for Intel).

Upstream WebRTC just landed[1] an enhancement to import/download the DMABUF data via GL. This is still short of using the DMABUF directly (bug 1729167), but already a big enhancement on affected systems. Lets try to backport the changes if possible.

1: https://webrtc-review.googlesource.com/c/src/+/227022

Comment 1

[Martin Stránský [:stransky] (ni? me)]

Robert, I don't understand what problem this one solves and how it can help us? How we can use the dmabuf/egl image in our pipeline?

Comment 2

[Robert Mader [:rmader]]

(In reply to Martin Stránský [:stransky] (ni? me) from comment #1)

Robert, I don't understand what problem this one solves and how it can help us? How we can use the dmabuf/egl image in our pipeline?

Right now we copy the data from the Pipewire buffer (no matter whether DmaBuf, MemFd or MemPtr) via memcpy[1], and line by line. In case of DmaBuf, this currently works well on integrated Intel cards but has several issues. Quoting from https://docs.pipewire.org/page_dma_buf.html:

1. DMA-BUFs can use hardware-specific tiling and compression as described by modifiers. Thus, a mmap(3) on the DMA-BUF FD will not give a linear view of the buffer contents.
2. DMA-BUFs need to be properly synchronized with the asynchronous reads and writes of the hardware. A mmap(3) call is not enough to guarantee proper synchronization. (Maybe add link to linux syscall doc??)
3. Blindly accessing the DMA-BUFs via mmap(3) can be extremely slow if the buffer has been allocated on discrete hardware. Consumers are better off using a proper graphics API (such as EGL, Vulkan or VA-API) to process the DMA-BUFs.

Especially point 3 shows on discrete AMD cards, and that's what the upstream patch is about.

Right now Mutter disables DmaBuf sharing on non-Intel because of "buggy" clients like WebRTC. However, like other compositors it will soon switch to enabling it whenever the client asks for it (like we do[2]). So in short: our Pipewire DmaBuf support is buggy and only works well on some hardware. We should fix it up by either:

• backport the upstream GL import
• not copying the data to CPU / doing it somewhere else, e.g. after encoding, and also not via memcpy (bug 1729167)
• stop advertising DmaBuf support (basically remove the line in [2]) and strictly use shared memory.

1: https://searchfox.org/mozilla-central/source/third_party/libwebrtc/webrtc/modules/desktop_capture/linux/base_capturer_pipewire.cc#439
2: https://searchfox.org/mozilla-central/source/third_party/libwebrtc/webrtc/modules/desktop_capture/linux/base_capturer_pipewire.cc#132

Comment 3

[Martin Stránský [:stransky] (ni? me)]

Yes, we need to ask for shm unless some decent dmabuf support is implemented (and I don't mean just copy dmabuf to shm but do all the operations we do over the buffer like resize/crop).

Also we may use dmabuf surface for it as it already provides blit support.

Comment 4

[Jan Grulich]

FYI: There is an ongoing review for proper DMA-BUF support to WebRTC: https://webrtc-review.googlesource.com/c/src/+/231180/

Comment 5

[Robert Mader [:rmader]]

Thanks Jan, also just saw your blogpost about that work (https://jgrulich.cz/2021/10/06/dma-buf-support-in-webrtc/). We should definitely backport these improvements! Apart from that I hope FF will catch up faster with upstream after the big update in bug 1654112 is done :/

Comment 6

[Robert Mader [:rmader]]

Bug 1654112 finally landed and the upstream changes apparently went into chromium by now IIUC (thus get tested there). So maybe we can backport those patches now.

Andreas: I don't want to step on your toes regarding future WebRTC update plans - do you see any problems if we backport some Pipewire/Linux specific changes into our tree? Most importantly those mentioned in comment 4?

Comment 7

[Andreas Pehrson [:pehrsons]]

FWIW we are planning to be on a more frequent update cycle now that we've done the large refactorings needed for bug 1654112.

I think backporting this is fine. If the next update we do has all the backported changes that would be easiest as we can just copy these parts of upstream verbatim. Either way, we'll manage.

Comment 8

[Jan Grulich]

Attachment: Bug 1729743 - Backport WebRTC upstream changes to Wayland screensharing support

This includes all the goodies that have been implemented in past
year, like proper DMA-BUF support, mouse cursor monitor implementation
and some other fixes pushed over the time.

You can see all the changes on the link below and check all changes
from beginning of 2021:
https://webrtc-review.googlesource.com/q/status:merged+from:grulja%2540gmail.com

Bug 1729743: [Pipewire] Better DMABUF import support (upstream backport)

Comment 9

[Robert Mader [:rmader]]

Try: https://treeherder.mozilla.org/jobs?repo=try&revision=4992b2f7c2fde10bfb78f7880d92c68bbdda824b

Comment 10

[Martin Stránský [:stransky] (ni? me)]

Jan, we need to check that on try to make sure it actually builds. There's the test build I did for you:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=b111593050bb1b0b74796251736f3cc3a7083875

Comment 11

[Martin Stránský [:stransky] (ni? me)]

Ah, you're quick :)

Comment 12

[Robert Mader [:rmader]]

(In reply to Martin Stránský [:stransky] (ni? me) from comment #11)

Ah, you're quick :)

But looks like I messed up while doing so :P

Comment 13

[Robert Mader [:rmader]]

This was done now by rebasing on top of upstream 103 and bug 1783716