Why are mask surfaces not playing nicely with each other?

RESOLVED FIXED in mozilla20

Status

()

RESOLVED FIXED
6 years ago
5 years ago

People

(Reporter: nrc, Unassigned)

Tracking

15 Branch
mozilla20
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

(Reporter)

Description

6 years ago
Find out why, fix it, undo bug 780868. See that bug for more details.
(Reporter)

Comment 1

6 years ago
Investigations so far: I've managed to reliably recreate this bug using a small but ugly hack, using a D3D10 layer manager just to create the mask surface. Of course this could cause different bugs, but hopefully not. Without this hack I could not recreate the intermittent fails that we see on Try, even with an old version of the tree. There is nothing wrong with the actual surface as far as I can see. Every where in the code I look at it in the debugger it is fine, and Cairo seems to be using it fine. This is the same as Matt Woodrow observed. However, I am noticing problems that seem to be transform related. This doesn't make much sense because nothing about a transform is cached, and the transform is associated with the mask layer, not the mask image, but we do transform the context before drawing into it, so maybe that is an issue.
(Reporter)

Comment 2

6 years ago
Further investigation reveals that it is probably nothing to do with transforms after all.
(Reporter)

Comment 3

6 years ago
Well, it seems to be kinda related to transforms. My best guess is that Cairo D2D surfaces do something differently with transforms which does not play nice when Cairo uses that D2D surface as a mask for a non-D2D-surface-backed context. Specifically, I'm guess it is something to do with what happens when we draw outside the surface - if I use surface which is larger than required, the bug goes away. Obviously, it is not as simple as that to fix, because there is no bug when the D2D surface is used as a mask using the D3D10 backend, which is why I think there is something funny going on in Cairo.
(Reporter)

Comment 4

6 years ago
Bas: any idea what could be going on here?
(Reporter)

Comment 5

6 years ago
This is puzzling me. There is something 'wrong' with the surface - painting stuff into it to try to help debugging doesn't work properly. I tried to set rgba = 0.5 everywhere, but this has the same effect as rgba = 1, any value > 0.0 has the same effect. I've played around scaling and translating various surfaces and don't get anything like the results I expect. I've copied the surface to an intermediate image surface, and that makes no difference. That does show that the pixel data looks plausible (it is not all ffffffff, as you would expect from rendering), I can't judge if it is exactly right but it seems on the right track. However, it looks like the rgb channels are varying independently, which I would not expect, but possibly some kind of AA artifact? I see, for e.g., 0080ffff.

I think there is some problem in the Cairo D2D code, I can't think of anything else that could be causing this, but can't get any more specific.
(Reporter)

Comment 6

6 years ago
I've just come back to this after it slipped under my radar for a while, and it looks like the problem has magically resolved it self. I don't think this is work trying to bisect though.

Try run with 780868 backed out: https://tbpl.mozilla.org/?tree=Try&rev=979facc49d8d

With the hack described above I couldn't recreate the problems (and it could previously reproduce the problem), nor could I see the problems with repeated runs of local or Try ref tests.

So, I vote we backout the patch from 780868 and see what happens.
(Reporter)

Comment 7

6 years ago
Created attachment 690994 [details] [diff] [review]
undo 780868
Attachment #690994 - Flags: review?(matt.woodrow)
(Reporter)

Comment 8

6 years ago
Created attachment 690996 [details]
hack patch

Keeping the mixed layer manager hack around for posterity and in case we need it again. Don't even think about landing this thing.
Attachment #690994 - Flags: review?(matt.woodrow) → review+
https://hg.mozilla.org/mozilla-central/rev/390fb4d21787
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla20
(Reporter)

Updated

5 years ago
Assignee: ncameron → nobody
You need to log in before you can comment on or make changes to this bug.