1577336 - Move async plugin composition to GPU/main process

Status: RESOLVED FIXED

(Core Graveyard :: Plug-ins, task, P1)

Description

[David Parks [:handyman]]

As part of the Win32k-lockdown sandboxing project, we are moving all Win32 operations out of the content process. Plugins use Direct3D in the content process for composition when doing async rendering. This bug is to forward those operations to the GPU process. It looks like it can use some of the GPUVideoTexture/Image stuff to stay simple.

Non-async plugin composition is already win32-kernel-free.

This is only relevant under WebRender. We ignore non-WR, which will not be supported in a win32k-lockdowned process.

The other part of win32k-lockdown wrt plugins is bug 1384260.

Comment 1

[David Parks [:handyman]]

Attachment: Bug 1577336: Part 1 - Remove win32k use from NPDrawingModelAsyncBitmapSurface r=jmathies!

This fallback drawing mode primarily uses in-memory textures in the content process (via Readback) but uses gfxPlatform to establish the GPU texture type. On Windows, this is always Win32 so we can avoid the gfxPlatform call (which uses Win32k heavily). I believe that this removes all Win32 operations involved in this drawing mode in a content process.

Comment 2

[David Parks [:handyman]]

Attachment: Bug 1577336: Part 2 - Add pref to disble Windows DXGI async plugin surfaces r=jmathies!

Use this pref to disable NPDrawingModelAsyncWindowsDXGISurface mode, which will force compatible plugins (Flash) to use NPDrawingModelAsyncBitmapSurface.

Depends on D46083

Comment 3

[David Parks [:handyman]]

Attachment: Bug 1577336: Part 3 - Fetch Windows plugin async graphics capabilities from gpu or main process r=jmathies!,mattwoodrow!

These operations report whether certain async plugin drawing modes are supported on the host architecture. They use kernel graphics operations to decide this so they need to be removed from the content process for sandboxing. We just bounce the requests to the gpu process.

Depends on D46084

Comment 4

[David Parks [:handyman]]

Attachment: Bug 1577336: Part 7 - Move DXGI async plugin operations to compositor process r=jmathies!,mattwoodrow!,mccr8!,sotaro

Previously, we created TextureD3D11 objects in the content process to back surfaces created for the plugin process. Those objects were then composited by the async ImageBridge. In order to remove Win32 kernel operations from content (including DX/GDI operations), this patch bounces the requests from content to the GPU process. The GPU process creates 2 textures to be used for all plugin composition -- one for the plugin process and one for display. The plugin process can freely write to its texture and request composition when it is done, which triggers a blit to the display texture. This mirrors pre-existing behavior.

Depends on D46085

Comment 5

[David Parks [:handyman]]

Matt, I'm making the changes you mentioned in part 4 but I don't know what to use in place of the GPUVideoImage I give to SetCurrentImage in PluginInstanceParent::RecvShowDirectDXGISurface. The old code used a TextureWrapperImage [1], which wraps a TextureClient, which I don't have (which is the stated point of the refactor).

[1] https://searchfox.org/mozilla-central/source/dom/plugins/ipc/PluginInstanceParent.cpp#760

Comment 6

[David Parks [:handyman]]

From Slack conversation: we're going to use the GPUVideo framework but change SurfaceDescriptorGPUVideo's IDs to be a union to separate video and plugins and create a new type for the TextureHost instead of using GPUVideoTextureHost.

Comment 7

[David Parks [:handyman]]

Attachment: Bug 1577336: Part 4 - Refactor some things in RemoteDecoderManager r=mattwoodrow!,jolin!

In anticipation of the rest of this patch series, we make 2 changes to the RemoteDecoderManager:

1. Rename RemoteDecoderManagerChild::DeallocateSurfaceDescriptorGPUVideo to DeallocateSurfaceDescriptor
2. Move call to RemoteDecoderManager::GetSource() from GPUVideoTextureClient to RemoteVideoDecoder.

Depends on D46085

Comment 8

[David Parks [:handyman]]

Attachment: Bug 1577336: Part 5 - Create IGPUVideoSurfaceManager for shared parts of RemoteDecoderManagerChild r=mattwoodrow

IGPUVideoSurfaceManager is an interface that the ImageBridgeChild uses to perform GPUVideoImage operations. RemoteDecoderManagerChild is one. We define another for plugins later in this patch series.

Depends on D52396

Comment 9

[David Parks [:handyman]]

Attachment: Bug 1577336: Part 6 - Make SurfaceDescriptorGPUVideo a union that contains SurfaceDescriptorRemoteDecoder r=mattwoodrow!,jolin!

SurfaceDescriptorGPUVideo, which currently only represents RemoteDecoder video, switches from being a struct to a union that holds a SurfaceDescriptorRemoteDecoder struct. SurfaceDescriptorRemoteDecoder is a new name for the old SurfceDescriptorGPUVideo. This is done so that we can later add SurfaceDescriptorPlugin as another type of SurfaceDescriptorGPUVideo.

Depends on D52397

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/caf9d8851555
Part 1 - Remove win32k use from NPDrawingModelAsyncBitmapSurface r=jmathies
https://hg.mozilla.org/integration/autoland/rev/c9af8e8d6625
Part 2 - Add pref to disble Windows DXGI async plugin surfaces r=jmathies
https://hg.mozilla.org/integration/autoland/rev/84e26e9be4ef
Part 3 - Fetch Windows plugin async graphics capabilities from gpu or main process r=jmathies,mattwoodrow,froydnj
https://hg.mozilla.org/integration/autoland/rev/f6c2f1e4c394
Part 4 - Refactor some things in RemoteDecoderManager r=mattwoodrow,jolin
https://hg.mozilla.org/integration/autoland/rev/f9fe69aa1c6b
Part 5 - Create IGPUVideoSurfaceManager for shared parts of RemoteDecoderManagerChild r=mattwoodrow
https://hg.mozilla.org/integration/autoland/rev/1c8cc6132a46
Part 6 - Make SurfaceDescriptorGPUVideo a union that contains SurfaceDescriptorRemoteDecoder r=mattwoodrow,jolin
https://hg.mozilla.org/integration/autoland/rev/a82a135299d5
Part 7 - Move DXGI async plugin operations to compositor process r=jmathies,mattwoodrow,sotaro,mccr8

Comment 11

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/caf9d8851555
https://hg.mozilla.org/mozilla-central/rev/c9af8e8d6625
https://hg.mozilla.org/mozilla-central/rev/84e26e9be4ef
https://hg.mozilla.org/mozilla-central/rev/f6c2f1e4c394
https://hg.mozilla.org/mozilla-central/rev/f9fe69aa1c6b
https://hg.mozilla.org/mozilla-central/rev/1c8cc6132a46
https://hg.mozilla.org/mozilla-central/rev/a82a135299d5

Comment 12

[David Parks [:handyman]]

We should back this out of beta while I work on approval of fixes. There is a backout patch in bug 1600032 comment 15. NI to ryanvm since he is handling the actual backout work.

Comment 13

[Ryan VanderMeulen [:RyanVM]]

Backed out from Beta per comment 12. It remains landed on m-c for 73+.
https://hg.mozilla.org/releases/mozilla-beta/rev/3af5f7e1bd23