Closed Bug 1577336 Opened 1 year ago Closed 10 months ago

Move async plugin composition to GPU/main process

Categories

(Core :: Plug-ins, task, P1)

Unspecified
Windows
task

Tracking

()

RESOLVED FIXED
mozilla72
Tracking Status
firefox72 --- disabled
firefox73 --- fixed

People

(Reporter: handyman, Assigned: handyman)

References

Details

Attachments

(7 files)

As part of the Win32k-lockdown sandboxing project, we are moving all Win32 operations out of the content process. Plugins use Direct3D in the content process for composition when doing async rendering. This bug is to forward those operations to the GPU process. It looks like it can use some of the GPUVideoTexture/Image stuff to stay simple.

Non-async plugin composition is already win32-kernel-free.

This is only relevant under WebRender. We ignore non-WR, which will not be supported in a win32k-lockdowned process.

The other part of win32k-lockdown wrt plugins is bug 1384260.

This fallback drawing mode primarily uses in-memory textures in the content process (via Readback) but uses gfxPlatform to establish the GPU texture type. On Windows, this is always Win32 so we can avoid the gfxPlatform call (which uses Win32k heavily). I believe that this removes all Win32 operations involved in this drawing mode in a content process.

Use this pref to disable NPDrawingModelAsyncWindowsDXGISurface mode, which will force compatible plugins (Flash) to use NPDrawingModelAsyncBitmapSurface.

Depends on D46083

These operations report whether certain async plugin drawing modes are supported on the host architecture. They use kernel graphics operations to decide this so they need to be removed from the content process for sandboxing. We just bounce the requests to the gpu process.

Depends on D46084

Previously, we created TextureD3D11 objects in the content process to back surfaces created for the plugin process. Those objects were then composited by the async ImageBridge. In order to remove Win32 kernel operations from content (including DX/GDI operations), this patch bounces the requests from content to the GPU process. The GPU process creates 2 textures to be used for all plugin composition -- one for the plugin process and one for display. The plugin process can freely write to its texture and request composition when it is done, which triggers a blit to the display texture. This mirrors pre-existing behavior.

Depends on D46085

Attachment #9093157 - Attachment description: Bug 1577336: Part 3 - Fetch Windows plugin async graphics capabilities from gpu process r=jmathies!,mattwoodrow! → Bug 1577336: Part 3 - Fetch Windows plugin async graphics capabilities from gpu or main process r=jmathies!,mattwoodrow!
Summary: Move async plugin composition to GPU process → Move async plugin composition to GPU/main process

Matt, I'm making the changes you mentioned in part 4 but I don't know what to use in place of the GPUVideoImage I give to SetCurrentImage in PluginInstanceParent::RecvShowDirectDXGISurface. The old code used a TextureWrapperImage [1], which wraps a TextureClient, which I don't have (which is the stated point of the refactor).

[1] https://searchfox.org/mozilla-central/source/dom/plugins/ipc/PluginInstanceParent.cpp#760

Flags: needinfo?(matt.woodrow)

From Slack conversation: we're going to use the GPUVideo framework but change SurfaceDescriptorGPUVideo's IDs to be a union to separate video and plugins and create a new type for the TextureHost instead of using GPUVideoTextureHost.

Flags: needinfo?(matt.woodrow)

In anticipation of the rest of this patch series, we make 2 changes to the RemoteDecoderManager:

  1. Rename RemoteDecoderManagerChild::DeallocateSurfaceDescriptorGPUVideo to DeallocateSurfaceDescriptor
  2. Move call to RemoteDecoderManager::GetSource() from GPUVideoTextureClient to RemoteVideoDecoder.

Depends on D46085

IGPUVideoSurfaceManager is an interface that the ImageBridgeChild uses to perform GPUVideoImage operations. RemoteDecoderManagerChild is one. We define another for plugins later in this patch series.

Depends on D52396

SurfaceDescriptorGPUVideo, which currently only represents RemoteDecoder video, switches from being a struct to a union that holds a SurfaceDescriptorRemoteDecoder struct. SurfaceDescriptorRemoteDecoder is a new name for the old SurfceDescriptorGPUVideo. This is done so that we can later add SurfaceDescriptorPlugin as another type of SurfaceDescriptorGPUVideo.

Depends on D52397

Attachment #9093159 - Attachment description: Bug 1577336: Part 4 - Move DXGI async plugin operations to GPU process r=jmathies!,mattwoodrow! → Bug 1577336: Part 7 - Move DXGI async plugin operations to compositor process r=jmathies!,mattwoodrow!,nical,sotaro
Attachment #9093159 - Attachment description: Bug 1577336: Part 7 - Move DXGI async plugin operations to compositor process r=jmathies!,mattwoodrow!,nical,sotaro → Bug 1577336: Part 7 - Move DXGI async plugin operations to compositor process r=jmathies!,mattwoodrow!,mccr8!,sotaro
Pushed by daparks@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/caf9d8851555
Part 1 - Remove win32k use from NPDrawingModelAsyncBitmapSurface r=jmathies
https://hg.mozilla.org/integration/autoland/rev/c9af8e8d6625
Part 2 - Add pref to disble Windows DXGI async plugin surfaces r=jmathies
https://hg.mozilla.org/integration/autoland/rev/84e26e9be4ef
Part 3 - Fetch Windows plugin async graphics capabilities from gpu or main process r=jmathies,mattwoodrow,froydnj
https://hg.mozilla.org/integration/autoland/rev/f6c2f1e4c394
Part 4 - Refactor some things in RemoteDecoderManager r=mattwoodrow,jolin
https://hg.mozilla.org/integration/autoland/rev/f9fe69aa1c6b
Part 5 - Create IGPUVideoSurfaceManager for shared parts of RemoteDecoderManagerChild r=mattwoodrow
https://hg.mozilla.org/integration/autoland/rev/1c8cc6132a46
Part 6 - Make SurfaceDescriptorGPUVideo a union that contains SurfaceDescriptorRemoteDecoder r=mattwoodrow,jolin
https://hg.mozilla.org/integration/autoland/rev/a82a135299d5
Part 7 - Move DXGI async plugin operations to compositor process r=jmathies,mattwoodrow,sotaro,mccr8
Regressions: 1600032
Regressions: 1600595

We should back this out of beta while I work on approval of fixes. There is a backout patch in bug 1600032 comment 15. NI to ryanvm since he is handling the actual backout work.

Flags: needinfo?(ryanvm)

Backed out from Beta per comment 12. It remains landed on m-c for 73+.
https://hg.mozilla.org/releases/mozilla-beta/rev/3af5f7e1bd23

Flags: needinfo?(ryanvm)
Regressions: 1598680
You need to log in before you can comment on or make changes to this bug.