Find out why, fix it, undo bug 780868. See that bug for more details.
Investigations so far: I've managed to reliably recreate this bug using a small but ugly hack, using a D3D10 layer manager just to create the mask surface. Of course this could cause different bugs, but hopefully not. Without this hack I could not recreate the intermittent fails that we see on Try, even with an old version of the tree. There is nothing wrong with the actual surface as far as I can see. Every where in the code I look at it in the debugger it is fine, and Cairo seems to be using it fine. This is the same as Matt Woodrow observed. However, I am noticing problems that seem to be transform related. This doesn't make much sense because nothing about a transform is cached, and the transform is associated with the mask layer, not the mask image, but we do transform the context before drawing into it, so maybe that is an issue.
Further investigation reveals that it is probably nothing to do with transforms after all.
Well, it seems to be kinda related to transforms. My best guess is that Cairo D2D surfaces do something differently with transforms which does not play nice when Cairo uses that D2D surface as a mask for a non-D2D-surface-backed context. Specifically, I'm guess it is something to do with what happens when we draw outside the surface - if I use surface which is larger than required, the bug goes away. Obviously, it is not as simple as that to fix, because there is no bug when the D2D surface is used as a mask using the D3D10 backend, which is why I think there is something funny going on in Cairo.
Bas: any idea what could be going on here?
This is puzzling me. There is something 'wrong' with the surface - painting stuff into it to try to help debugging doesn't work properly. I tried to set rgba = 0.5 everywhere, but this has the same effect as rgba = 1, any value > 0.0 has the same effect. I've played around scaling and translating various surfaces and don't get anything like the results I expect. I've copied the surface to an intermediate image surface, and that makes no difference. That does show that the pixel data looks plausible (it is not all ffffffff, as you would expect from rendering), I can't judge if it is exactly right but it seems on the right track. However, it looks like the rgb channels are varying independently, which I would not expect, but possibly some kind of AA artifact? I see, for e.g., 0080ffff. I think there is some problem in the Cairo D2D code, I can't think of anything else that could be causing this, but can't get any more specific.
I've just come back to this after it slipped under my radar for a while, and it looks like the problem has magically resolved it self. I don't think this is work trying to bisect though. Try run with 780868 backed out: https://tbpl.mozilla.org/?tree=Try&rev=979facc49d8d With the hack described above I couldn't recreate the problems (and it could previously reproduce the problem), nor could I see the problems with repeated runs of local or Try ref tests. So, I vote we backout the patch from 780868 and see what happens.
Created attachment 690994 [details] [diff] [review] undo 780868
Attachment #690994 - Flags: review?(matt.woodrow)
Created attachment 690996 [details] hack patch Keeping the mixed layer manager hack around for posterity and in case we need it again. Don't even think about landing this thing.
Attachment #690994 - Flags: review?(matt.woodrow) → review+
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla20
You need to log in before you can comment on or make changes to this bug.