Closed
Bug 1322817
Opened 7 years ago
Closed 7 years ago
Make webrender reftests deterministic enough to get consistent pass/fail results
Categories
(Core :: Graphics: WebRender, defect, P3)
Tracking
()
RESOLVED
FIXED
mozilla54
Tracking | Status | |
---|---|---|
firefox53 | --- | affected |
People
(Reporter: kats, Assigned: kats)
References
Details
(Whiteboard: [gfx-noted])
In bug 1319188 I'm making the reftest harness assume that all tests that are EXPECTED_PASS or EXPECTED_FUZZY can randomly fail when webrender is enabled. That's because we don't seem to be producing deterministic results from webrender. We need to fix that, so that we can start figuring out which tests are actually passing and which are actually failing.
Comment 1•7 years ago
|
||
Can you point me to some randomless logs? I wonder if it's just bug 1322796. Thanks!
Flags: needinfo?(bugmail)
Assignee | ||
Comment 2•7 years ago
|
||
It might be. I'll re-do some try pushes and see if I get random results or not. Leaving ni on me for now.
Assignee | ||
Comment 4•7 years ago
|
||
It looks all the nondeterminism is in the layout/reftests/position-dynamic-changes folder. For example, in the try push at [1] there were a bunch of failures. I took all the opt e10s failures, marked them fails-if(webrender) and ran it again [2]. If the failures were the same every time, this try push should have all green opt-e10s reftests. However, there are some additional UNEXPECTED-FAIL and some UNEXPECTED-PASS results in the position-dynamic-changes reftests. Also note that in [1] the opt non-e10s, debug e10s, and debug non-e10s have different sets of failing position-dynamic-changes tests. I'll do another try push with all the position-dynamic-changes ones marked as random-if(webrender) and see if that fares better. [1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=f1797c4f3dd4bc4005cb4d037d19e30ace93e5c2 [2] https://treeherder.mozilla.org/#/jobs?repo=try&revision=5796575fee2b7e87b2bec2f841f4899dbc4d3844
Assignee | ||
Comment 5•7 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=fb46a049e932fc266a9c98ca1a4bee5f6e592023 seems to be better. Almost too good to be true, actually...
Flags: needinfo?(bugmail)
Assignee | ||
Updated•7 years ago
|
Assignee: nobody → bugmail
Pushed by kgupta@mozilla.com: https://hg.mozilla.org/projects/graphics/rev/6c6f13287120 Mark many position-dynamic-changes reftests as random-if(webrender). r=gfx? https://hg.mozilla.org/projects/graphics/rev/a2040adb5434 Mark reftests failing on WR builds as fails-if(webrender). r=gfx? https://hg.mozilla.org/projects/graphics/rev/b6a73b6f177e Mark zoomed video tests as failing with non-e10s WR. r=gfx? https://hg.mozilla.org/projects/graphics/rev/92f621d51f1f Stop assuming all reftests behave randomly on WR builds. r=gfx?
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Pushed by kgupta@mozilla.com: https://hg.mozilla.org/projects/graphics/rev/186e023dadf6 Mark a couple more tests as random-if(webrender) since they fail intermittently. r=gfx?
Assignee | ||
Updated•7 years ago
|
Target Milestone: --- → mozilla54
You need to log in
before you can comment on or make changes to this bug.
Description
•