943204 - OMTC rendering to popup windows (e.g., Australis menu)

Status: VERIFIED FIXED

(Core :: Graphics: Layers, defect)

Description

[Nick Cameron [:nrc]]

Using OMTC to render to popup windows on Windows is broken. Nothing renders at all. There are absolutely no error messages, even from d3d in debug mode. This only happens on Windows (Linux is fine) and is independent of backend (basic, d3d9, and d3d11 are all broken in the same way). Rendering the main window OMTC and the popup MT is OK (note that the Australis menu is a panel, so IsSmallPopup is false, so OMTC is the default way to render it). Works fine for MT rendering with any backend.

With d3d9 I have verified that everything is happening OK (data is drawn correctly on the main thread, uploading to d3d texture is OK (and I dumped just before upload, so communication to the compositor is OK), all the correct setup is happening and in the right order (i.e., we are not initialising one compositor then drawing with another), nothing is returning failure values). Compositing is happening and we get the expected number of DrawQuad calls, again no early return, no errors. Just nothing appearing on screen. As expected, we have a single d3d9 device and two d3d9 swap chains.

With basic, we do not initialise properly - we fail here (http://dxr.mozilla.org/mozilla-central/source/widget/windows/nsWindow.cpp#3553) because the gfxWindowsSurface has 0 size. I could not work out why. There is nothing else wrong with it. I replaced this code with code using the size from GetClientBounds, but still nothing. I experimented with using various different HDCs but that didn't seem to help.

The widget looks fine, afaict - correct size, no worrying field values. mTransparencyMode == eTransparencyOpaque, which triggers some special cases, but we seem to do the same thing for MT and OMTC rendering.

Anyone got any ideas?

Comment 1

[Andreas Gal :gal]

I have a similar bug on mac. There ContentClientIncremental is broken I believe (verifying as we speak).

Comment 2

[Nick Cameron [:nrc]]

In case anyone wants to try this out, just run a nightly recent enough to have Australis, flick the OMTC pref, restart, click the hamburger menu icon. Expected: you see the menu; found: you don't.

Comment 3

[Matt Woodrow (:mattwoodrow)]

http://mxr.mozilla.org/mozilla-central/source/gfx/layers/d3d9/DeviceManagerD3D9.cpp#58

Is that rect normal?

Comment 4

[Nick Cameron [:nrc]]

(In reply to Matt Woodrow (:mattwoodrow) from comment #3)
> http://mxr.mozilla.org/mozilla-central/source/gfx/layers/d3d9/
> DeviceManagerD3D9.cpp#58
> 
> Is that rect normal?

Yes, it is fine.

Comment 5

[Nick Cameron [:nrc]]

I got a lead! As I mentioned above, mTransparencyMode == eTransparencyTransparent for the popup (also for the main window, I think). There are a few cases where that forces a separate path. I tried undoing a few of them before, but to no avail. However, if I always force mTransparencyMode to be eTransparencyOpaque, then I see the menu (rendered poorly for obvious reasons, but it is there).

Tomorrow I will figure this out properly.

Comment 6

[Nick Cameron [:nrc]]

Reversing the check added in bug 802316 so that we get a LAYERED window, but not a COMPOSITED one makes the menu visible, but not transparent. I don't know why this would only affect OMTC rendering yet, but we do some funky stuff for rendering to transparent windows that we might not be doing correctly with OMTC.

James: do you have any insight as to what might be happening here?

Comment 7

[Nick Cameron [:nrc]]

Just removing the check from bug 802316 entirely has the same effect - the menu renders but what should be transparent is rendered as black.

Comment 8

[James]

You might try removing the WS_EX_COMPOSITED flag on popup creation to see if you get the same effect as reversing bug 802316.
http://mxr.mozilla.org/mozilla-central/source/widget/windows/nsWindow.cpp#485

I remember running into similar effects (black popups) when trying to change flags later

Comment 9

[Nick Cameron [:nrc]]

(In reply to James from comment #8)
> You might try removing the WS_EX_COMPOSITED flag on popup creation to see if
> you get the same effect as reversing bug 802316.
> http://mxr.mozilla.org/mozilla-central/source/widget/windows/nsWindow.cpp#485
> 
> I remember running into similar effects (black popups) when trying to change
> flags later

The menu/window is still invisible. So, this looks like I need WS_EX_COMPOSITED to see the window. I am not sure why that is. Perhaps the behaviour around WM_PAINT messages is changed. Also, I don't understand why I am not getting transparency. I suspect that solving things this way is not correct, and somehow our handling of the memory DC for translucent images is not playing well with OMTC.

Comment 10

[Nick Cameron [:nrc]]

Attachment: Fix basic compositor

Comment 11

[Nick Cameron [:nrc]]

Attachment: Don't use HWA for transparent windows

Comment 12

[Nick Cameron [:nrc]]

So, turns out the Window style flags had nothing to do with this. For transparent windows we do a whole bunch of fun stuff using an in-memory bitmap. That requires using the correct DC for drawing and calling UpdateTranslucentWindow() after drawing (frustratingly, I tried both of these things earlier, but didn't realise they were closely related, so I didn't try doing both).

For the HWA compositors, we just need to not use them for transparent windows, just like we don't for main thread rendering (http://dxr.mozilla.org/mozilla-central/source/widget/windows/nsWindow.cpp#3282).

Comment 13

[Matt Woodrow (:mattwoodrow)]

Comment on attachment 8339666 [details] [diff] [review]
Fix basic compositor

Review of attachment 8339666 [details] [diff] [review]:
-----------------------------------------------------------------

::: widget/windows/nsWindow.cpp
@@ +3557,1 @@
>    ReleaseDC(mWnd, mCompositeDC);

Change this to FreeNativeData()

Comment 14

[Nick Cameron [:nrc]]

(In reply to Matt Woodrow (:mattwoodrow) from comment #13)
> Comment on attachment 8339666 [details] [diff] [review]
> Fix basic compositor
> 
> Review of attachment 8339666 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: widget/windows/nsWindow.cpp
> @@ +3557,1 @@
> >    ReleaseDC(mWnd, mCompositeDC);
> 
> Change this to FreeNativeData()

Good catch, thanks

Comment 15

[Nick Cameron [:nrc]]

Attachment: Fix basic compositor

With Matt's change

Comment 16

[Nick Cameron [:nrc]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/27a9c236606b
https://hg.mozilla.org/integration/mozilla-inbound/rev/e52c27213bb7

Comment 17

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/27a9c236606b
https://hg.mozilla.org/mozilla-central/rev/e52c27213bb7

Comment 19

[Andrei Vaida [:avaida]]

I managed to reproduce the issue using Nightly 28.0a1 (2013-11-30) [1] and Fx 27.0.1 (Build ID: 20140212131424) [2]  with the layers.offmainthreadcomposition.enabled pref set to true.

I was able to confirm the fix for this issue on Windows 7 64-bit, using:
- the latest Beta (Build ID: 20140213172947) [1]
- the latest Aurora (Build ID: 20140217004001) [3]
- the latest Nightly (Build ID: 20140217030202) [4]


[1] Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0
[2] Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0
[3] Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0
[4] Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:30.0) Gecko/20100101 Firefox/30.0