1406533 - Support remote layer trees in popups on Linux

Status: VERIFIED FIXED

(Core :: Widget: Gtk, defect, P3)

Description

[Andy McKay]

This is a fork off of bug 1356317 comment 23 where we changed that bug to be Windows specific. It looks like the main work to be done isn't quite known at this time.

Comment 1

[:shell escalante]

Moving comment 0 from bug 1356317 to this bug... so it's in one location.  

kmag: Since bug 630346, transparent popups on Linux use a BasicLayerManager to make it easier to compute shape masks. Unfortunately, as a side-effect of this, it's not possible for them to host remote frameloaders.

We need to be able to show remote browsers in popups for OOP WebExtensions, so I think we need to find a better solution for handling transparency here.


What this looks like:
"We don't support transparent popups with remote layer trees. We do support non-transparent popups with remote layer trees, so those popups appear, but with black backgrounds in place of any transparent areas."

The two options to fix it are basically:

1) Switch to composited popups rather than using binary shape masks. That should work for most people, but not for people who don't use composited window managers.

2) Keep using the shape mask, but calculate it in some other way. I think OS-X wound up doing something along these lines to calculate shadow shapes for composited popups.

Comment 2

[:shell escalante]

Hi Karl,

I'm poking around to find who might be interested in resolving how remote layer trees in popups are supported for Linux.  Do you have any ideas?  Markus Stange just fixed a similar OOP issue OSx (if that bug 1370131)

We'd like to be able to enable Out of Process for Webextensions on Linux (to get the speed and stability benefits to Linux) for Firefox 62-63.

It also appears related to a WebRender Linux P2 bug (https://bugzil.la/1444595#c2).  The target timing to land WebRender is Firefox 63.

What worked well for OSx was landing behind a pref, QA the known issues, then turning on for only Nightly and Dev edition audiences to shake out any quirks.

Comment 3

[Karl Tomlinson (:karlt)]

(In reply to :shell escalante from comment #1)
> What this looks like:
> "We don't support transparent popups with remote layer trees. We do support
> non-transparent popups with remote layer trees, so those popups appear, but
> with black backgrounds in place of any transparent areas."

Where is that quoted from?

> The two options to fix it are basically:
> 
> 1) Switch to composited popups rather than using binary shape masks. That
> should work for most people, but not for people who don't use composited
> window managers.

Since https://hg.mozilla.org/mozilla-central/rev/94e12595dad0 popups are
already composited where supported.

AFAIK that should already work, except for the deception going on at 
https://searchfox.org/mozilla-central/rev/f65d7528e34ef1a7665b4a1a7b7cdb1388fcd3aa/layout/xul/nsMenuPopupFrame.cpp#310
If that is still necessary, then I suspect a bug somewhere in Gecko's
compositing (graphics).

You are correct that this wouldn't work as intented for people who don't use
composited window managers.  If popups only have remote content with certain
add-ons, then I guess an option to consider may be to use OMTC when there is
remote content even when compositing is not available.  This will result in
rectangular popups with black in the areas that are intended to be
transparent.

> 2) Keep using the shape mask, but calculate it in some other way. I think
> OS-X wound up doing something along these lines to calculate shadow shapes
> for composited popups.

Currently the shape mask is only applied on the main thread.  This has not
been extended to the OMTC thread.  It would be possible to make all of the
nsWindow::UpdateAlpha() procedure thread-safe.  That procedure overlaps widget
and graphics, but usually the Gecko-compositor-associated code is owned by
graphics.

However, bug 1370131 and bug 1291457 observed that applying the shape mask
only on the main thread is often sufficient because a resize often triggers a
main thread repaint.  I'm not sure that is the case when windows are resized
to a smaller size but it may be worth checking.  If no extensions with
non-resize opacity animations are expected on MacOS, then I guess the GTK port
may temporarily get away with assuming the same.

-moz-window-opacity and -moz-window-transform are not available in the GTK
port, but it looks like arrow panel animations in
browser/base/content/browser.css may already be disabled for the GTK port.
If not, I'd guess the animations could be disabled when compositing is not
available.

ISTR there are also some window managers that don't support the shape
extension.  If a window manager neither composites nor supports the shape
extension, then it is a lost cause.

The first thing to do here would be to revert
https://hg.mozilla.org/mozilla-central/rev/cb4b09616e23 and file a bug
describing the problems and window manager on which the problems are observed.
This was required by review at
https://reviewboard.mozilla.org/r/140250/#comment187910

Comment 4

[Karl Tomlinson (:karlt)]

(In reply to :shell escalante from comment #2)
> The target timing to land WebRender is Firefox 63.

Is that target really for the Linux port?

Comment 5

[Milan Sreckovic [:milan] (needinfo for best results)]

(In reply to Karl Tomlinson (:karlt) from comment #4)
> (In reply to :shell escalante from comment #2)
> > The target timing to land WebRender is Firefox 63.
> 
> Is that target really for the Linux port?

Not currently.  Linux is more likely to be 65 or later, so that's likely not the way to go for 62-63 time frame.

Comment 6

[Karl Tomlinson (:karlt)]

> (In reply to :shell escalante from comment #1)
> > 2) Keep using the shape mask, but calculate it in some other way. I think
> > OS-X wound up doing something along these lines to calculate shadow shapes
> > for composited popups.

(In reply to Karl Tomlinson (:karlt) from comment #3)
> Currently the shape mask is only applied on the main thread.

Sorry, I was wrong here.
When OMTC is used, the shape mask is not applied at all as OnExposeEvent()
returns early and the "if (shaped)" blocks are not run.

https://searchfox.org/mozilla-central/rev/f65d7528e34ef1a7665b4a1a7b7cdb1388fcd3aa/widget/gtk/nsWindow.cpp#2151

If translucent regions need to be supported on non-compositing window managers,
then the shape needs to be updated from the OMTC path also.  I guess that
could be calculated in a similar manner to the basic layers path, at least
when the window is updated from the main thread.  The graphics team would know
better than I about that.

Comment 7

[David Bolter [:davidb] (NeedInfo me for attention)]

Nical, could you triage this further? Is it a lot of work?

Comment 8

[Nicolas Silva [:nical]]

At a glance it looks like a non-trivial amount of work, but I'd need to delve into how this works to tell. I'll  keep the needinfo as a reminder to take a closer look.

Comment 9

[Jim Mathies [:jimm]]

Hey Martin, we're trying to get this opengl rendering bug fixed for Linux so we can get web extensions running out of process. This change improves security and responsiveness for linux users. Do you all have any cycles for something like this?

Comment 10

[Martin Stránský [:stransky] (ni? me)]

(In reply to Jim Mathies [:jimm] from comment #9)
> Hey Martin, we're trying to get this opengl rendering bug fixed for Linux so
> we can get web extensions running out of process. This change improves
> security and responsiveness for linux users. Do you all have any cycles for
> something like this?

Sure, I'm happy to help.

Comment 11

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1406533 - Create and apply XShape mask at WindowSurfaceX11Image,

Derived routines for shape bitmap mask form nsWindow.cpp,
at WindowSurfaceX11Image we compose a final RGBA image and then create
the shape mask from image alpha channel.

Also the implementation is slightly simplified as it's meant as a fallback only.

Review commit: https://reviewboard.mozilla.org/r/259066/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/259066/

Comment 12

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1406533 - Implement a way to transfer shape option from nsWindow to WindowSurfaceX11Image,

Add a new config attribute to GtkCompositorWidgetInitData to allow transfer
the shape option from nsWindow. Also when shape is requested don't use XRender
or Shm widgets - only X11Image is supported now.

Review commit: https://reviewboard.mozilla.org/r/259068/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/259068/

Comment 13

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1406533 - Configure compositor widget to shape X11 window on non-compositing WM,

Also remove mozilla.widget.use-argb-visuals which is useless since we don't allow transparent
toplevel windows.

Review commit: https://reviewboard.mozilla.org/r/259070/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/259070/

Comment 14

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1406533 - Don't force in process rendering for transparent (popup) windows,

Review commit: https://reviewboard.mozilla.org/r/259072/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/259072/

Comment 15

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1406533 - Remove hack for remote popup windows on Linux,

This hack was introduced at Bug 630346 and we should not need it any more.

Review commit: https://reviewboard.mozilla.org/r/259074/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/259074/

Comment 16

[Lee Salzman [:lsalzman]]

Comment on attachment 8994515 [details]
Bug 1406533 - Create and apply XShape mask at WindowSurfaceX11Image,

https://reviewboard.mozilla.org/r/259066/#review266154

Comment 17

[Lee Salzman [:lsalzman]]

Comment on attachment 8994516 [details]
Bug 1406533 - Implement a way to transfer shape option from nsWindow to WindowSurfaceX11Image,

https://reviewboard.mozilla.org/r/259068/#review266158

Comment 18

[Lee Salzman [:lsalzman]]

Comment on attachment 8994517 [details]
Bug 1406533 - Configure compositor widget to shape X11 window on non-compositing WM,

https://reviewboard.mozilla.org/r/259070/#review266160

Comment 19

[Lee Salzman [:lsalzman]]

Comment on attachment 8994518 [details]
Bug 1406533 - Don't force in process rendering for transparent (popup) windows,

https://reviewboard.mozilla.org/r/259072/#review266162

Comment 20

[Karl Tomlinson (:karlt)]

Comment on attachment 8994519 [details]
Bug 1406533 - Remove hack for remote popup windows on Linux,

https://reviewboard.mozilla.org/r/259074/#review266204

Thank you, Martin!

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6af61c29b549
Create and apply XShape mask at WindowSurfaceX11Image, r=lsalzman
https://hg.mozilla.org/integration/autoland/rev/6a9aeec30818
Implement a way to transfer shape option from nsWindow to WindowSurfaceX11Image, r=lsalzman
https://hg.mozilla.org/integration/autoland/rev/0cfb908b2ccf
Configure compositor widget to shape X11 window on non-compositing WM, r=lsalzman
https://hg.mozilla.org/integration/autoland/rev/89b8c49481b2
Don't force in process rendering for transparent (popup) windows, r=lsalzman
https://hg.mozilla.org/integration/autoland/rev/4b73b8c72408
Remove hack for remote popup windows on Linux, r=karlt

Comment 22

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/6af61c29b549
https://hg.mozilla.org/mozilla-central/rev/6a9aeec30818
https://hg.mozilla.org/mozilla-central/rev/0cfb908b2ccf
https://hg.mozilla.org/mozilla-central/rev/89b8c49481b2
https://hg.mozilla.org/mozilla-central/rev/4b73b8c72408

Comment 23

[Vlad Jiman]

Verified using the latest build of Nightly running on Ubuntu 18.04, the issue is not reproducible on this version, the extension panel content is loading successfully. - 08/17/2018

Comment 24

[Vlad Jiman]

Attachment: validate.png