1732951 - (gpu-process-linux-wayland) [Wayland] Implement GPU process on Wayland

Status: NEW

(Core :: Graphics, enhancement)

Description

[Martin Stránský [:stransky] (ni? me)]

We need GPU process on Wayland to use VAAPI outside content process so let's create one.

Comment 1

[Robert Mader [:rmader]]

Short reminder that there's one possible alternative: bug 1731890. Just for the record.

Quote from matrix:

gerard-majax: I'm not sure exactly about VA-API specifically, but the plan is to have such a utilityprocess that can be used by different actors, and each with a specific sandboxing
and from that discussion, I think the first customer might be moving RDD audio decoders to this new

Comment 2

[Martin Stránský [:stransky] (ni? me)]

The GPU process can cooperate with the utility one. I guess we want to have GPU on Wayland anyway.

Comment 3

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

If the goal is to move compositing out of the parent process, then the utility process project isn't really relevant. The original purpose of the GPU process (as I understand it) was to allow recovering from GPU driver bugs that crash the process, so that might be useful in general.

But, as a substitute for having an “RDD but with GPU access” process:

1. We could just give the RDD process GPU access, as long as it's the specifics aren't too unreasonable; it's not running hostile script so it's harder to attack than a content process.
2. The utility process project does matter, because it will eventually let us run codecs that need GPU access in a different process from those that don't.

Comment 4

[Martin Stránský [:stransky] (ni? me)]

eglCreatePlatformWindowSurfaceEXT may be a way how to implement it (EGL rendering to offscreen dmabuf surface).

Comment 5

[Robert Mader [:rmader]]

(In reply to Martin Stránský [:stransky] (ni? me) from comment #4)

eglCreatePlatformWindowSurfaceEXT may be a way how to implement it (EGL rendering to offscreen dmabuf surface).

FWIW, that's AFAIK what Wine-Wayland devs currently explore as well (and AFAIK is used in Chrome-Wayland). Note that, as we'll not use the EGL-Wayland platform any more for buffer handling, we'll have to do extra work if we want to keep e.g. direct-scanout/dmabuf feedback capabilities and stuff like that.

Comment 6

[Martin Stránský [:stransky] (ni? me)]

(In reply to Robert Mader [:rmader] from comment #5)

FWIW, that's AFAIK what Wine-Wayland devs currently explore as well (and AFAIK is used in Chrome-Wayland). Note that, as we'll not use the EGL-Wayland platform any more for buffer handling, we'll have to do extra work if we want to keep e.g. direct-scanout/dmabuf feedback capabilities and stuff like that.

What do you mean with EGL-Wayland platform?

Comment 7

[Robert Mader [:rmader]]

(In reply to Martin Stránský [:stransky] (ni? me) from comment #6)

What do you mean with EGL-Wayland platform?

Essentially the Wayland specific methods like wl_egl_window_create() from <wayland-egl.h> and the buffer handling in https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/egl/drivers/dri2/platform_wayland.c. Or rather: the abstraction that EGL does over the Wayland protocols so we can just use eglMakeCurrent()/eglSwapBuffer() and don't need to call wl_surface_attach()/wl_surface_commit() and handle buffers/the swapchain.

That's what you plan, right? Attaching/committing the dmabuf-buffers to surfaces manually in the main process?

Comment 8

[Martin Stránský [:stransky] (ni? me)]

We need to create offscreen rendering target in GPU process based on dmabuf somehow and then transfer that to main process. How is the dmabuf rendered in main process is another question. The wl_surface_attach / wl_surface_commit may be better than another blit to wl_egl_window.

Comment 9

[BugBot [:suhaib / :marco]]

The bug assignee is inactive on Bugzilla, so the assignee is being reset.

Comment 10

[Suhaib Mujahid [:suhaib]]

Sorry, there was a problem with the detection of inactive users. I'm reverting the change.

Comment 11

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

(In reply to Martin Stránský [:stransky] (ni? me) from comment #0)

We need GPU process on Wayland to use VAAPI outside content process so let's create one.

To update this bug: we were able to support VA-API by using the RDD process, and it wasn't necessary to let it connect to the display server (bug 1769499).

Comment 12

[Martin Stránský [:stransky] (ni? me)]

AFAIK if we want to go this way on Linux/Wayland we can use what we have already implemented for WebGL where shared dmabuf surfaces are used for remote GL rendering.

Comment 13

[Brad Werth [:bradwerth]]

I'll try to implement some more of this, based on the in-progress work for macOS in Bug 1889757.

Comment 14

[Brad Werth [:bradwerth]]

Attachment: WIP: Bug 1732951 Part 1: Prepare GtkCompositorWidget to handle process endpoints.

Comment 15

[Brad Werth [:bradwerth]]

Attachment: WIP: Bug 1732951 Part 2: Hoist CreateLayerForSurfacePresentation into NativeLayerRoot.

This is being done in anticipation of using the surface presentation
approach in NativeLayerRootWayland.

Comment 16

[Brad Werth [:bradwerth]]

Attachment: WIP: Bug 1732951 Part 3: Create NativeLayerWaylandPresentation for surface presentation.

Comment 17

[Brad Werth [:bradwerth]]

Attachment: WIP: Bug 1732951 Part 4: Define the NativeLayerRootRemoteWayland parent and child to send layers from gpu to parent.

Comment 18

[Brad Werth [:bradwerth]]

With parts 1 through 4 of this patch series, we have defined untested but hopefully-useful analogues of the classes used for the macOS gpu process. The notable classes:

NativeLayerRootRemoteWaylandChild a gpu process layer which functions as a root layer. It creates...
NativeLayerRemoteWayland a gpu process layer with a buffer that is rendered into, These buffers, along with other layer commands are sent to...
NativeLayerRootRemoteWaylandParent a parent process layer which receives all these layer commands and buffers, creating as needed...
NativeLayerWaylandPresentation a parent process layer attached the real native layer root, which is the ultimate destination for the buffers.

What remains for a proof-of-concept:

1. Actually create the NativeLayerRootRemoteWaylandChild and NativeLayerRootRemoteWaylandParent peers.
2. Ensure that the child can be rendered into and assemble the right messages.
3. Send the messages and buffers from child to parent.
4. Interpret messages on the parent.

Work continues towards proof-of-concept, which will be a combination of new parts of the patch series, and updates to existing parts.

After the proof-of-concept effort, a later to-be-done piece is support for pixel readback (for reftests and for profiler screenshots). This will be modeled after the macOS SnapshotterCADelegate.

Comment 19

[Brad Werth [:bradwerth]]

Attachment: WIP: Bug 1732951 Part 5: Create the NativeLayerRemote peers.