408284 - [Linux/Gnome] Support for translucent windows on Linux

Status: RESOLVED FIXED

(Core :: Widget: Gtk, defect)

Description

[Michael Ventnor]

Attachment: Patch

There is a really easy way to support RGBA per-pixel translucency on windows for Linux, but only while a compositing window manager is being run. This approach gets a suitable RGBA colourmap from the window manager if supported, and applies it to the window. If not, then we just fall back to our old 1-bit transparency.

This was coded under instruction from my boss Roc even though I had next to no knowledge about what I was writing :-)

Roc, pavlov and vlad, please give me your comments.

Comment 1

[Robert O'Callahan (:roc) (email my personal email if necessary)]

+    if (oneBitTransparent || translucent) {
         // Collapse update area to the bounding box. This is so we only have to
         // call UpdateTranslucentWindowAlpha once. After we have dropped

This can be just if (oneBitTransparent), I think.

I wonder if there's a way to get GDK to create an RGBA pixmap for us so we get the performance optimization described by
    // Instead of just doing PushGroup we're going to do a little dance
    // to ensure that GDK creates the pixmap
for translucent windows as well. That would be good. However, since this path is only taken for truly translucent windows, it's not critical right now.

-    ResizeTransparencyBitmap(rect.width, rect.height);
-

I wouldn't move this, it could break things the 1-bit case.

+    GdkColormap* rgbaColormap = gdk_screen_get_rgba_colormap(gdk_screen_get_default());

We should only use an rgbaColormap if gdk_screen_is_composited is true, so we should test gdk_screen_is_composited here. (Fortunately gdk_screen_is_composited seems to be very fast, just looking up a field in a local object.)

+    GdkScreen* defaultScr = gdk_screen_get_default();

We don't have to get the default screen, we have GTK/gdk widgets around we can get the screen for.

+    if (gdk_screen_is_composited(defaultScr) && gdk_screen_get_rgba_colormap(defaultScr)) {

instead of getting the colormap again, maybe just record in nsWindow whether we created it with an rgba colormap or not.

     PRBool       mIsTranslucent;
+    PRBool       mIs1BitTransparent;

These can be PRPackedBools.

Other than that, I think this patch looks good. The big question is whether enabling rgba colormaps when Compiz is enabled imposes a performance penalty on our opaque windows. Since we draw to an opaque backbuffer anyway, actual drawing won't be slowed down, I think, but there may be a penalty when we do the backbuffer flip or when Compiz itself does its screen composition. Or there might not, since that stuff had better be accelerated or you don't want to be using Compiz. Stuart, Vlad, what do you think?

Comment 2

[Michael Ventnor]

Attachment: Patch 2

Comment 3

[Michael Ventnor]

Comment on attachment 293039 [details] [diff] [review]
Patch 2

I'm actually getting quite a few rendering errors and a frequent shutdown crash with this patch. I'm not sure why. I don't think I'm skilled enough for this bug...

Comment 5

[huang li]

I have a question, add
gtk_module_init (gint * argc, gchar *** argv)
{
                        
  GdkVisual *visual = gdk_screen_get_rgba_visual(gdk_screen_get_default());
  colormap = gdk_colormap_new(visual, TRUE);
                     
  gtk_widget_push_colormap(colormap);
                                        
}

for firefox, then watch video with adobe flash pluging, firefox crashed.
The crash reason is fixed firefox use rgba (32 depth) was conflict to adobe flash rgb (24 depth).

The 2th method is that reomve gtk_widget_set_colormap(GTK_WIDGET(container), gdk_rgb_get_colormap()); in mozcontainer and push rgba colormap in nsWindow::NativeCreate.

I hope not to push rgba for docummentviewer::makewindow , or for preventing adobe flash crash, push rgb for nsboxframe::init(windowless plugin drawable) and nsobjectframe::createwidget and nsPluginNativeWindowGtk2::CreateXEmbedWindow(window plugin drawable). because there are more frame need been changed, that's so large work.

I guess because gtk_widget_set_colormap(GTK_WIDGET(container), gdk_rgb_get_colormap()) in mozcontainer_init exist, adobe hardcode rgb colormap.

mozcontainer_init code is so old, gtk and X11 grow so fast in past years, rgba was not what freshness yet.when can mozilla resolve the history left problem?

Comment 6

[Karl Tomlinson (:karlt)]

(In reply to comment #5)
> I hope not to push rgba for docummentviewer::makewindow , or for preventing
> adobe flash crash, push rgb for nsboxframe::init(windowless plugin drawable)
> and nsobjectframe::createwidget and
> nsPluginNativeWindowGtk2::CreateXEmbedWindow(window plugin drawable). because
> there are more frame need been changed, that's so large work.

Is there a problem with only setting the RGB colormap on the GtkSocket in CreateXEmbedWindow?

> mozcontainer_init code is so old, gtk and X11 grow so fast in past years, rgba
> was not what freshness yet.when can mozilla resolve the history left problem?

We can't control what Adobe do, but often we can work around the issues.
Out-of-process plugins (bug 478976) should help.

Comment 7

[huang li]

(In reply to comment #6)
> (In reply to comment #5)
> > I hope not to push rgba for docummentviewer::makewindow , or for preventing
> > adobe flash crash, push rgb for nsboxframe::init(windowless plugin drawable)
> > and nsobjectframe::createwidget and
> > nsPluginNativeWindowGtk2::CreateXEmbedWindow(window plugin drawable). because
> > there are more frame need been changed, that's so large work.
> 
> Is there a problem with only setting the RGB colormap on the GtkSocket in
> CreateXEmbedWindow?

It's failure, because the code is for window plugin, but when a flash was played first in browser, it was windowless plugin, windowless plugin use some frame's inner gdk window as drawable.
> 
> > mozcontainer_init code is so old, gtk and X11 grow so fast in past years, rgba
> > was not what freshness yet.when can mozilla resolve the history left problem?
> 
> We can't control what Adobe do, but often we can work around the issues.
> Out-of-process plugins (bug 478976) should help.

Comment 8

[Karl Tomlinson (:karlt)]

When quoting other comments, only quote the relevant section so as to keep the bug reports as readable as possible.

(In reply to comment #7)
> windowless plugin use some frame's inner gdk window as drawable.

If the DefaultVisualOfScreen has no alpha, then you probably just need a hack
to remove DRAW_SUPPORTS_NONDEFAULT_VISUAL for Adobe Flash Player.

http://hg.mozilla.org/mozilla-central/annotate/231a65a2d273/layout/generic/nsObjectFrame.cpp#l4852

If DefaultVisualOfScreen is ARGB, then some similar flag could be designed.
(I don't know precisely which formats Flash Player can handle.)

Comment 9

[huang li]

(In reply to comment #8)
> When quoting other comments, only quote the relevant section so as to keep the
> bug reports as readable as possible.
> 
> (In reply to comment #7)
> > windowless plugin use some frame's inner gdk window as drawable.
> 
> If the DefaultVisualOfScreen has no alpha, then you probably just need a hack
> to remove DRAW_SUPPORTS_NONDEFAULT_VISUAL for Adobe Flash Player.
> 
> http://hg.mozilla.org/mozilla-central/annotate/231a65a2d273/layout/generic/nsObjectFrame.cpp#l4852
> 
> If DefaultVisualOfScreen is ARGB, then some similar flag could be designed.
> (I don't know precisely which formats Flash Player can handle.)
That's not problem core whether draw with cario in DRAW_SUPPORTS_NONDEFAULT_VISUAL. wrong is surface(drawable) not flags.

I have resolved the problem. push rgba in justcreatetoplevelwindow and nsmenupopupframe::createwidgetforview for transparent window. force depth = 24 in onexpose and when new pixmap, let drawable null.Don't forget to push rgb in gtk_module_init.

transparent window that use above method , it own (0,0,0,0) not (0,0,0)black, so put a transluncent image on it for get a translucent window.

Comment 10

[huang li]

Attachment: use x11 composited and rgba.

when I use the patch, I didn't find crash or workless. please teach me write some mochitest or talos test, thx.

Comment 11

[huang li]

Comment on attachment 419659 [details] [diff] [review]
use x11 composited and rgba.

can the patch enter mozilla code?

Comment 12

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Sorry about the delay. I don't understand how the patch works. It looks like you are making every XUL window use the RGBA visual, even windows which are not transparent. And I presume that this patch would not make any popups use the RGBA visual.

Comment 13

[Michael Ventnor]

(In reply to comment #12)
> Sorry about the delay. I don't understand how the patch works. It looks like
> you are making every XUL window use the RGBA visual, even windows which are not
> transparent. And I presume that this patch would not make any popups use the
> RGBA visual.

You can only set the visual on a window before it is mapped. You can't do it on the fly like the Mozilla code wants to do.
What popups won't use the RGBA visual? Because all windows created in Mozilla go through nsWindow, right?

Comment 14

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Yes, but this patch isn't enabling RGBA visuals for all nsWindows, only for those initialized in nsAppShellService::JustCreateTopWindow.

We might be able to get away with making all windows use RGBA, but we'd need to do some careful performance measurement to prove that it doesn't slow down rendering to opaque windows.

Comment 15

[huang li]

(In reply to comment #14)
> Yes, but this patch isn't enabling RGBA visuals for all nsWindows, only for
> those initialized in nsAppShellService::JustCreateTopWindow.
> 
> We might be able to get away with making all windows use RGBA, but we'd need to
> do some careful performance measurement to prove that it doesn't slow down
> rendering to opaque windows.

wait expectantly.

Comment 16

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Comment on attachment 419659 [details] [diff] [review]
use x11 composited and rgba.

I'm not sure what you're waiting for. There are two problems with this patch, I think:
1) popups not created with RGBA visuals
2) opaque windows created with an RGBA visual may have a performance hit

Comment 17

[huang li]

(In reply to comment #16)
> (From update of attachment 419659 [details] [diff] [review])
> I'm not sure what you're waiting for. There are two problems with this patch, I
> think:
> 1) popups not created with RGBA visuals
> 2) opaque windows created with an RGBA visual may have a performance hit

I still thought you could fix the 2 issues.

Comment 19

[Juliano F. Ravasi]

I was trying to figure out why Mozilla windows don't respect QtCurve style blur/transparencies on Linux, and found this bug. It seems to be another instance of broken Mozilla emulation/imitation of GTK widgets.

I would like to note that the GTK theme engine may have an ultimate saying on which windows have X11 ARGB (not RGBA) visuals. For the QtCurve engine, for example, the style itself defines the transparency of widgets, and in this case, **ALL** windows have transparency enabled by default (there is no such thing as "opaque windows", it is up to the theme engine to determine the transparency of each pixel based on which widgets are stacked there).

Of course, this is broken now in Mozilla.

If there is a concern about performance, I think that if the theme engine is making all windows transparent (because it knows that the user is running Composite + Compiz or KWin), just respect it and let it use an ARGB visual (all windows). It affects the entire user desktop anyway; performance is a different problem.

Otherwise there will be more and more people filing bugs about Firefox and Thunderbird not working with styles and engines like QtCurve and Oxygen-transparent.

Comment 20

[Mounir Lamouri (:mounir)]

This bug is blocking bug 604257 which is blocking final. Shouldn't we make this blocking final or betaN to make sure it's visible (and done)?

Comment 21

[Karl Tomlinson (:karlt)]

This would provide only a partial solution for bug 604257, and it is not the only possible solution.

Comment 22

[Joe Drew (not getting mail)]

I'd be glad if this was fixed by someone, but we can release 2.0 without it.

Comment 24

[Chris Coulson]

I started looking at this today, and I've got a WIP patch which kind of works and doesn't have the problems mentioned in comment 16. It's a bit tricky because the visual of the widget can't be changed once it has been realized, which means I have to unrealize/realize the native widgets in nsWindow::SetTransparencyMode().

What doesn't work at the moment is translucency for native-styled widgets.

Comment 25

[Chris Coulson]

Attachment: Screenshot of translucent tooltip

Here's a screenshot I took with non-native styled tooltip

Comment 26

[Karl Tomlinson (:karlt)]

I had a quick look to see whether it was feasible to determine transparency
before creating the window, but that doesn't look like it is going to be
practical given the way windows are currently created first and then documents
are loaded into them.

Giving all popup windows ARGB visuals shouldn't have too much effect on performance.

NoAutoHide panels are the exception as they can be long lived, and may have window decorations.

Perhaps give all override-redirect windows (i.e. GTK_WINDOW_POPUP) ARGB visuals.
Menus probably don't need to be translucent, but I doubt it is worth special-case optimizing those.

Comment 27

[Chris Coulson]

Hi Karl,

I've used your approach and updated my patch to set RGBA visuals on all short-lived (GTK_WINDOW_POPUP) windows.

One thing I'm wondering is why this exists:

http://hg.mozilla.org/mozilla-central/file/5faccd0b7618/widget/gtk2/mozcontainer.c#l179

I found that when trying to figure out why I have to explicitly set the visual on the child MozContainer, rather than having it be inherited from the top-level window.

Comment 28

[Chris Coulson]

Attachment: Support RGBA windows on Linux

Here's the current WIP patch to actually enable RGBA windows. I ended up just removing the bit of code in moz_container_init() which hard-coded the colormap, so that it's always inherited from the top-level window now.

(Note, the tooltip style change in this patch is just for demonstration until I get the rendering of native styled elements working properly. I've been looking at that today, but I probably won't have that working until after the weekend)

Comment 29

[Karl Tomlinson (:karlt)]

(In reply to Chris Coulson from comment #28)
> I ended up
> just removing the bit of code in moz_container_init() which hard-coded the
> colormap, so that it's always inherited from the top-level window now.

That sounds the right thing to do.

Comment 30

[Jared Wein [:jaws] (please needinfo? me)]

Chris, do you think this will be landing for Fx18? Bug 786125 could use the translucent windows in the rewrite :)

Comment 31

[Karl Tomlinson (:karlt)]

(In reply to Jared Wein [:jaws] from comment #30)
> Chris, do you think this will be landing for Fx18? Bug 786125 could use the
> translucent windows in the rewrite :)

I doubt this will be immediately useful for bug 786125, because that looks like it will need some way of knowing whether translucency is available yet so that the opacity transition is not used when the window manager is not compositing, and I don't know that we have that.  There is also the question of whether a compositing window manager will be doing its own animation.  However, on most Linux desktops, we should be using libnotify for native notifications instead of the XUL nsIAlertsService fallback.

I would like to get this in for bug 635897, however, because that is now more frequently bad after https://hg.mozilla.org/integration/mozilla-inbound/rev/f92968ac30fa.

Comment 32

[Karl Tomlinson (:karlt)]

Attachment: no need to apply transparency bitmap on each show

Removing some code left over from when GTK was used to set shape masks, or before then.  We now use X directly because setting shape masks through GTK interfered with invalidation of hidden regions.  mTransparencyBitmap won't be set on the first show anyway, and subsequent shows would just be setting the same mask again.

Comment 33

[Karl Tomlinson (:karlt)]

Attachment: clear transparency bitmap when window is hidden

Popup windows tend to hang around for the life of the parent window.
This avoids keeping the bitmaps around while the windows are not in use.
The popup will need to be repainted when it is shown again, so it is not too much trouble to apply the shape mask again at that time.

The shape mask is actually removed from the X window as well because mTransparencyBitmap is used in a subsequent patch as an indicator that there is a shape to clear.

The old code for SetTransparencyMode(opaque) was not right after the change from setting masks through GTK to X11 directly, but that path is apparently not used.

Comment 34

[Karl Tomlinson (:karlt)]

Attachment: let MozContainer widgets inherit parent colormap instead of explicitly setting rgb colormap

From Chris's patch.

This is old code from when colors were in short supply and only one client's colormap was active at a time, but this is unnecessary because nsAppRunner sets the default colormap.  It would probably be better to use the screen visual as an indicator of the display's capabilities than trying to use the best colormap anyway.

Comment 35

[Karl Tomlinson (:karlt)]

Attachment: use ARGB visuals for popup windows when window manager is compositing

Also from Chris's patch.

The only things I changed was to replace the MOZ_WIDGET_GTK3 elif with an else because we don't have a MOZ_WIDGET_GTK3 define, and removing the mHasRGBAVisual variable, which I'll do differently because that variable won't be propagated to child windows (even though we rarely, if ever, use child windows with translucent popups).

Comment 36

[Karl Tomlinson (:karlt)]

Attachment: don't shape for transparency on ARGB windows

I used slightly different logic to attachment 661250 [details] [diff] [review], so that shaping is used even on ARGB windows if the window manager is no longer compositing.  It handles switches in compositing state during paint as well as on window hide/show, even though changes there are rare because popups are hidden when focus changes.  I'm expecting a repaint when switching compositing state.

Attachment 666836 [details] [diff] is such that popups will only have ARGB visuals if compositing was active when the window was created.  That seems a sensible compromise because usually no compositing manager at creation means there will be no compositing for the life of application.  When we add a method for the style system to detect whether translucency is available, we may have to revisit that if that method can't easily check the relevant window.

One thing this doesn't handle is setting the input shape mask on the translucent ARGB windows.  This means that clicking on the transparent region will register on the popup instead of the window behind.  Usually we don't make large portions of the window transparent, and this is used only for popup windows where the only action of clicking outside the window is to close the popup, so I'm hoping this is not a big issue.  Setting the input mask would require us to continue to use the inefficient readback code.

Comment 37

[Karl Tomlinson (:karlt)]

(In reply to Karl Tomlinson (:karlt) from comment #36)
> One thing this doesn't handle is setting the input shape mask on the
> translucent ARGB windows.  ... this is used only
> for popup windows

Yes, there seems little point in doing this.  Because the popup windows grab the pointer, all mouse input events are directed at the popup window anyway.
Even when the bounding shape (which affects input) is applied, clicking in the transparent part of the window does not cause the popup to be closed.  It is client-side code (check_for_rollup I expect) doing the hit testing.

Comment 38

[Chris Coulson]

Hi Karl, thanks for that (and sorry, I keep getting dragged away on to other things at the moment :( )

Comment 39

[Karl Tomlinson (:karlt)]

Thanks for getting this moving again, Chris.
I don't know what's involved for transparent-themed widgets.
That might be quite a bit more work, so let's track that in a different bug.

https://tbpl.mozilla.org/?tree=Try&rev=9e00d1a3cdae

https://hg.mozilla.org/integration/mozilla-inbound/rev/6552072432a2
https://hg.mozilla.org/integration/mozilla-inbound/rev/f42211cc87cd
https://hg.mozilla.org/integration/mozilla-inbound/rev/6c2cfb9e0b45
https://hg.mozilla.org/integration/mozilla-inbound/rev/4aac63aa19dc
https://hg.mozilla.org/integration/mozilla-inbound/rev/b2fd5b6ca0d9

Comment 40

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/6552072432a2
https://hg.mozilla.org/mozilla-central/rev/f42211cc87cd
https://hg.mozilla.org/mozilla-central/rev/6c2cfb9e0b45
https://hg.mozilla.org/mozilla-central/rev/4aac63aa19dc
https://hg.mozilla.org/mozilla-central/rev/b2fd5b6ca0d9