Get reftests running on Windows QR builds in automation

RESOLVED FIXED in Firefox 61

Status

()

Core
Graphics: WebRender
P1
normal
RESOLVED FIXED
a year ago
3 months ago

People

(Reporter: kats, Assigned: kats)

Tracking

(Depends on: 1 bug, Blocks: 2 bugs)

Other Branch
mozilla61
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox56 unaffected, firefox57 unaffected, firefox61 fixed)

Details

(Whiteboard: [wr-reserve] [gfx-noted])

MozReview Requests

()

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(5 attachments)

Bug to track the work needed to get windows automation for QuantumRender.

Some relevant info is on the wiki page at https://wiki.mozilla.org/QA/Platform/Graphics/Quantum/Renderer
For the record I did a try push and webrender is crashing left and right, mostly in nvoglv32.dll [2]. I'm hoping the switch to angle will help here, and if not, I'll look into it after that so that I'm not going down a blind alley.

[1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=8f4a28f95d94a93a2dde5d7fa761a18be42c8337
[2] https://treeherder.mozilla.org/logviewer.html#?job_id=82560817&repo=try&lineNumber=2604
Depends on: 1323316
Depends on: 1355679
I'm going to slowly dig through the crashes and file issues/bugs for them.
Depends on: 1361434
Depends on: 1360613
I did another try push recently: https://treeherder.mozilla.org/#/jobs?repo=try&revision=db7e9f22bb8ab7da5820bdd071cedde86d894219

Note that win7 and non-e10s win8 get WR disabled at runtime because of no GPU process, so right now we only care about the win8 e10s jobs.

The results are actually much better than they used to be. For one thing webrender doesn't crash on startup, so that's good. The mochitest failures can mostly be ignored since they would probably fail on Linux too if we ran them there. The reftests are failing, but pretty much every failure is due to a fuzz with max difference=1.
Depends on: 1362397
Depends on: 1365250
Depends on: 1367452

Updated

11 months ago
Status: NEW → ASSIGNED
Priority: P3 → P1
Whiteboard: [gfx-noted] → [wr-mvp] [gfx-noted]
Target Milestone: --- → mozilla57
status-firefox56: --- → unaffected
status-firefox57: --- → unaffected
(Assignee)

Updated

10 months ago
Assignee: bugmail → nobody
Status: ASSIGNED → NEW
Priority: P1 → P2
Target Milestone: mozilla57 → ---

Updated

8 months ago
Priority: P2 → P3
Whiteboard: [wr-mvp] [gfx-noted] → [wr-reserve] [gfx-noted]
(Assignee)

Updated

7 months ago
Depends on: 1412186
When we are finally ready to get this running in automation we'll need some changes to the buildbot-configs and stuff. Adding a related bug where similar changes were made for running QR reftests on the graphics branch. (Mostly adding this so I can find it again later).
See Also: → bug 1362397
Priority: P3 → P1
Assignee: nobody → bugmail
Depends on: 1429515
Depends on: 1429471
Depends on: 1429524
Depends on: 1429568
Depends on: 1429577
Depends on: 1429581
Depends on: 1429582
Created attachment 8941592 [details]
misc-failures.txt

Here's a bunch of failures that I categorized as "very wrong" but didn't look at close enough to triage fully. (attaching here so they aren't lost)
Created attachment 8941597 [details]
loosely-categorized-failures.txt

Here's some issues that I was able to better categorize, but didn't (yet) make an issue for. (attaching here so it's not lost)
The reftests are still way too nondeterministic to be able to annotate them and get them running. See for example try push at https://treeherder.mozilla.org/#/jobs?repo=try&revision=5ad02d0418c2594cde92c62b42b8887d26e0ac4d which I have a bunch of annotations in based on previous pushes/Gankro's notes, and in this one I'm getting new reftest-sanity failures. If the sanity reftests are failing something is fundamentally broken in the reftest machinery for webrender, so none of the other results are meaningful.
Windows10-64 reftests (non-webrender) just got migrated to taskcluster in bug 1435844. I did a try push for windows10-64-qr using the taskcluster setup but webrender fails on startup because of this:

19:38:02     INFO -  WebRender - OpenGL version new OpenGL ES 3.0 (ANGLE 2.1.0.db3422764a9b)
19:38:02     INFO -  WebRender - OpenGL version new OpenGL ES 3.0 (ANGLE 2.1.0.db3422764a9b)
19:38:02     INFO -  [GFX1]: Failed to create Pbuffer of back buffer error: 0x300c Size : Size(120,0)
19:38:02     INFO -  [GFX1-]: Compositors might be mixed (5,3)

from is coming from [1]. Error code 0x300c is EGL_BAD_PARAMETER. Sotaro, do you know what we should do to resolve this? Presumably the 0 height is being rejected.

[1] https://searchfox.org/mozilla-central/rev/bd05e3853c6e982e2a35c1cc404b987b2bc914d6/gfx/webrender_bindings/RenderCompositorANGLE.cpp#275
Flags: needinfo?(sotaro.ikeda.g)
The try push is here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=2ac8409339f0e29ba8f92abbff040e235690c822

The debug jobs look like they ran successfully with WR. So maybe this failure is just intermittent. I'll retrigger a few times.

Updated

5 months ago
Depends on: 1441056
(In reply to Kartikaya Gupta (email:kats@mozilla.com) from comment #9)
> 
> from is coming from [1]. Error code 0x300c is EGL_BAD_PARAMETER. Sotaro, do
> you know what we should do to resolve this? Presumably the 0 height is being
> rejected.

0 height should not be related. EGL accept 0 size and I confirmed that 0 height worked. I thought that Bug 1441056 might be related.
Flags: needinfo?(sotaro.ikeda.g)
Thanks. I did retrigger it and it seems intermittent, not very frequent. So good to fix, but it's not blocking me too much right now.
(Assignee)

Updated

5 months ago
Depends on: 1435844
Bug 1435844 was backed out due to bug 1441208. Until that relands I'm putting this on hold, because the nondeterminism disappeared on the new hardware (on the runs I managed to do before the machines got totally backlogged) so there's no point spending/wasting time tracking that down on the old setup.
(Assignee)

Updated

5 months ago
Depends on: 1439952
The new hardware is back and appears to have stuck. I did a new try push with what I hope is a complete annotation for windows qr reftests: https://treeherder.mozilla.org/#/jobs?repo=try&revision=ed98bc930d7b04d2eb606f7cb554262175b7ce3b
Silly typo in one of the annotations. New try push with that fixed: https://treeherder.mozilla.org/#/jobs?repo=try&revision=a5acd397f837eeff5e03fac48f429bdb4a48017f
That one was green, but infra melted before my retriggers ran so there was a bunch of orange from that. And I guess there was one intermittent in the R2 debug job. But it's close enough that I want to get the patches reviewed and ready to land.
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)

Comment 20

4 months ago
mozreview-review
Comment on attachment 8965796 [details]
Bug 1344350 - Mark reftests failing with webrender on windows.

https://reviewboard.mozilla.org/r/234630/#review240298
Attachment #8965796 - Flags: review?(jmuizelaar) → review+

Comment 21

4 months ago
mozreview-review
Comment on attachment 8965797 [details]
Bug 1344350 - Mark reftests passing with webrender on Windows.

https://reviewboard.mozilla.org/r/234632/#review240300
Attachment #8965797 - Flags: review?(jmuizelaar) → review+

Comment 22

4 months ago
mozreview-review
Comment on attachment 8965798 [details]
Bug 1344350 - Enable reftests on windows-qr in automation.

https://reviewboard.mozilla.org/r/234634/#review240302
Attachment #8965798 - Flags: review?(jmuizelaar) → review+

Comment 23

4 months ago
Pushed by kgupta@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/19a80805c67f
Mark reftests failing with webrender on windows. r=jrmuizel
https://hg.mozilla.org/integration/autoland/rev/be1b225e9bd9
Mark reftests passing with webrender on Windows. r=jrmuizel
https://hg.mozilla.org/integration/autoland/rev/aa377b9ced17
Enable reftests on windows-qr in automation. r=jrmuizel

Updated

4 months ago
Depends on: 1452236

Updated

4 months ago
Depends on: 1452240

Comment 24

4 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/19a80805c67f
https://hg.mozilla.org/mozilla-central/rev/be1b225e9bd9
https://hg.mozilla.org/mozilla-central/rev/aa377b9ced17
Status: NEW → RESOLVED
Last Resolved: 4 months ago
status-firefox61: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla61
(Assignee)

Updated

3 months ago
Depends on: 1452658
You need to log in before you can comment on or make changes to this bug.