1436084 - Audit failing WR reftests

Status: RESOLVED FIXED

(Core :: Graphics: WebRender, defect, P1)

Description

[Kartikaya Gupta (email:[e-mail])]

+++ This bug was initially created as a clone of Bug #1429469 +++

Many of these are passing now, but are fuzzy. Because a fails-if annotation will check for zero fuzz, it will treat any fuzzy tests as failing, and therefore the annotation will "pass". However removing the fails-if and replacing it with an appropriate fuzzy-if will also make the test pass, and is more correct.

Comment 1

[Kartikaya Gupta (email:[e-mail])]

Here is the try push where I marked everything passing to identify which ones are still actually failing:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=4d493998982827d31e4a0a8310c5c9b17f5e3c1b

Comment 2

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1436084 - Mark some reftests as passing with webrender.

Review commit: https://reviewboard.mozilla.org/r/218110/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/218110/

Comment 3

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1436084 - Update webrender reftest expectations in w3c-css received reftests.

In some cases there are imported tests where the same test is compared against
two reference pages, once as == and once as !=. Webrender is currently failing
the == tests but not the != tests. However there is no way to annotate this
precisely in the failures.list file, so we had previously marked the tests
as random-if. This patch modifies the import-tests.py to recognize a ref:<file>
argument in failures.list to match the reference file as well, and uses it to
clarify the webrender reftest expectations.

Review commit: https://reviewboard.mozilla.org/r/218112/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/218112/

Comment 4

[Jeff Muizelaar [:jrmuizel]]

Comment on attachment 8948718 [details]
Bug 1436084 - Mark some reftests as passing with webrender.

https://reviewboard.mozilla.org/r/218110/#review223944

Comment 5

[Kartikaya Gupta (email:[e-mail])]

Here's a try push based on autoland (includes latest WR update) and with these patches:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=4d83cc4fe3fa7bbecd1a90ce29597253684b6344

Comment 6

[Xidorn Quan [:xidorn] UTC+10]

Comment on attachment 8948719 [details]
Bug 1436084 - Update webrender reftest expectations in w3c-css received reftests.

https://reviewboard.mozilla.org/r/218112/#review224022

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b97c18388c86
Mark some reftests as passing with webrender. r=jrmuizel
https://hg.mozilla.org/integration/autoland/rev/b12895bb88fe
Update webrender reftest expectations in w3c-css received reftests. r=xidorn

Comment 8

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/b97c18388c86
https://hg.mozilla.org/mozilla-central/rev/b12895bb88fe