1344350 - Get reftests running on Windows QR builds in automation

Status: RESOLVED FIXED

(Core :: Graphics: WebRender, defect, P1)

Description

[Kartikaya Gupta (email:[e-mail])]

Bug to track the work needed to get windows automation for QuantumRender.

Some relevant info is on the wiki page at https://wiki.mozilla.org/QA/Platform/Graphics/Quantum/Renderer

Comment 1

[Kartikaya Gupta (email:[e-mail])]

For the record I did a try push and webrender is crashing left and right, mostly in nvoglv32.dll [2]. I'm hoping the switch to angle will help here, and if not, I'll look into it after that so that I'm not going down a blind alley.

[1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=8f4a28f95d94a93a2dde5d7fa761a18be42c8337
[2] https://treeherder.mozilla.org/logviewer.html#?job_id=82560817&repo=try&lineNumber=2604

Comment 2

[Kartikaya Gupta (email:[e-mail])]

New try push: https://treeherder.mozilla.org/#/jobs?repo=try&revision=4aadca662b7f6b4e0a013c89adaa156ffc24ab22

Comment 3

[Kartikaya Gupta (email:[e-mail])]

I'm going to slowly dig through the crashes and file issues/bugs for them.

Comment 4

[Kartikaya Gupta (email:[e-mail])]

I did another try push recently: https://treeherder.mozilla.org/#/jobs?repo=try&revision=db7e9f22bb8ab7da5820bdd071cedde86d894219

Note that win7 and non-e10s win8 get WR disabled at runtime because of no GPU process, so right now we only care about the win8 e10s jobs.

The results are actually much better than they used to be. For one thing webrender doesn't crash on startup, so that's good. The mochitest failures can mostly be ignored since they would probably fail on Linux too if we ran them there. The reftests are failing, but pretty much every failure is due to a fuzz with max difference=1.

Comment 5

[Kartikaya Gupta (email:[e-mail])]

When we are finally ready to get this running in automation we'll need some changes to the buildbot-configs and stuff. Adding a related bug where similar changes were made for running QR reftests on the graphics branch. (Mostly adding this so I can find it again later).

Comment 6

[Aria Beingessner [:Gankra]]

Attachment: misc-failures.txt

Here's a bunch of failures that I categorized as "very wrong" but didn't look at close enough to triage fully. (attaching here so they aren't lost)

Comment 7

[Aria Beingessner [:Gankra]]

Attachment: loosely-categorized-failures.txt

Here's some issues that I was able to better categorize, but didn't (yet) make an issue for. (attaching here so it's not lost)

Comment 8

[Kartikaya Gupta (email:[e-mail])]

The reftests are still way too nondeterministic to be able to annotate them and get them running. See for example try push at https://treeherder.mozilla.org/#/jobs?repo=try&revision=5ad02d0418c2594cde92c62b42b8887d26e0ac4d which I have a bunch of annotations in based on previous pushes/Gankro's notes, and in this one I'm getting new reftest-sanity failures. If the sanity reftests are failing something is fundamentally broken in the reftest machinery for webrender, so none of the other results are meaningful.

Comment 9

[Kartikaya Gupta (email:[e-mail])]

Windows10-64 reftests (non-webrender) just got migrated to taskcluster in bug 1435844. I did a try push for windows10-64-qr using the taskcluster setup but webrender fails on startup because of this:

19:38:02     INFO -  WebRender - OpenGL version new OpenGL ES 3.0 (ANGLE 2.1.0.db3422764a9b)
19:38:02     INFO -  WebRender - OpenGL version new OpenGL ES 3.0 (ANGLE 2.1.0.db3422764a9b)
19:38:02     INFO -  [GFX1]: Failed to create Pbuffer of back buffer error: 0x300c Size : Size(120,0)
19:38:02     INFO -  [GFX1-]: Compositors might be mixed (5,3)

from is coming from [1]. Error code 0x300c is EGL_BAD_PARAMETER. Sotaro, do you know what we should do to resolve this? Presumably the 0 height is being rejected.

[1] https://searchfox.org/mozilla-central/rev/bd05e3853c6e982e2a35c1cc404b987b2bc914d6/gfx/webrender_bindings/RenderCompositorANGLE.cpp#275

Comment 10

[Kartikaya Gupta (email:[e-mail])]

The try push is here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=2ac8409339f0e29ba8f92abbff040e235690c822

The debug jobs look like they ran successfully with WR. So maybe this failure is just intermittent. I'll retrigger a few times.

Comment 11

[Sotaro Ikeda [:sotaro]]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #9)
> 
> from is coming from [1]. Error code 0x300c is EGL_BAD_PARAMETER. Sotaro, do
> you know what we should do to resolve this? Presumably the 0 height is being
> rejected.

0 height should not be related. EGL accept 0 size and I confirmed that 0 height worked. I thought that Bug 1441056 might be related.

Comment 12

[Kartikaya Gupta (email:[e-mail])]

Thanks. I did retrigger it and it seems intermittent, not very frequent. So good to fix, but it's not blocking me too much right now.

Comment 13

[Kartikaya Gupta (email:[e-mail])]

Bug 1435844 was backed out due to bug 1441208. Until that relands I'm putting this on hold, because the nondeterminism disappeared on the new hardware (on the runs I managed to do before the machines got totally backlogged) so there's no point spending/wasting time tracking that down on the old setup.

Comment 14

[Kartikaya Gupta (email:[e-mail])]

The new hardware is back and appears to have stuck. I did a new try push with what I hope is a complete annotation for windows qr reftests: https://treeherder.mozilla.org/#/jobs?repo=try&revision=ed98bc930d7b04d2eb606f7cb554262175b7ce3b

Comment 15

[Kartikaya Gupta (email:[e-mail])]

Silly typo in one of the annotations. New try push with that fixed: https://treeherder.mozilla.org/#/jobs?repo=try&revision=a5acd397f837eeff5e03fac48f429bdb4a48017f

Comment 16

[Kartikaya Gupta (email:[e-mail])]

That one was green, but infra melted before my retriggers ran so there was a bunch of orange from that. And I guess there was one intermittent in the R2 debug job. But it's close enough that I want to get the patches reviewed and ready to land.

Comment 17

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1344350 - Mark reftests failing with webrender on windows.

Many of these could probably be fuzzed but in the interests of getting
the reftest suite turned on sooner I'm doing a blanket fails-if. This
covers all the reftests where there is more fuzz with webrender on
windows than any of existing annotations account for. In some cases the
fuzz is only a few pixels more than the equivalent Linux fuzz already
annotated, but I'll clean that up in a future bug.

Review commit: https://reviewboard.mozilla.org/r/234630/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/234630/

Comment 18

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1344350 - Mark reftests passing with webrender on Windows.

This covers all the reftests that have lower fuzz (or zero fuzz) and
were producing an UNEXPECTED-PASS result with webrender on windows. In
many cases I just adjusted the lower bound of the existing webrender
fuzz. In other cases existing fails-if conditions had to be tweaked to
exclude webrender.

Review commit: https://reviewboard.mozilla.org/r/234632/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/234632/

Comment 19

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1344350 - Enable reftests on windows-qr in automation.

Review commit: https://reviewboard.mozilla.org/r/234634/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/234634/

Comment 20

[Jeff Muizelaar [:jrmuizel]]

Comment on attachment 8965796 [details]
Bug 1344350 - Mark reftests failing with webrender on windows.

https://reviewboard.mozilla.org/r/234630/#review240298

Comment 21

[Jeff Muizelaar [:jrmuizel]]

Comment on attachment 8965797 [details]
Bug 1344350 - Mark reftests passing with webrender on Windows.

https://reviewboard.mozilla.org/r/234632/#review240300

Comment 22

[Jeff Muizelaar [:jrmuizel]]

Comment on attachment 8965798 [details]
Bug 1344350 - Enable reftests on windows-qr in automation.

https://reviewboard.mozilla.org/r/234634/#review240302

Comment 23

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/19a80805c67f
Mark reftests failing with webrender on windows. r=jrmuizel
https://hg.mozilla.org/integration/autoland/rev/be1b225e9bd9
Mark reftests passing with webrender on Windows. r=jrmuizel
https://hg.mozilla.org/integration/autoland/rev/aa377b9ced17
Enable reftests on windows-qr in automation. r=jrmuizel

Comment 24

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/19a80805c67f
https://hg.mozilla.org/mozilla-central/rev/be1b225e9bd9
https://hg.mozilla.org/mozilla-central/rev/aa377b9ced17