Closed Bug 1401997 Opened 3 years ago Closed 3 years ago
Report the reftest difference for fails-if tests that do fail
59 bytes, text/x-review-board-request
Right now, when reftests are run, we report the pixel differences for tests marked fuzzy, and for tests that result in UNEXPECTED-FAIL. I think we should just always report it. In particular, it's useful to see if tests marked as fails[-if] are actually very close to passing (e.g. they have a 1-pixel difference). In such cases the harness will treat the test as TEST-KNOWN-FAIL which is fine but if the difference is small the test could just as well be annotated as fuzzy-if and be made passing.
Comment on attachment 8910786 [details] Bug 1401997 - Always report pixel difference counts for reftests. https://reviewboard.mozilla.org/r/182246/#review187586 this seems very useful :)
Attachment #8910786 - Flags: review?(jmaher) → review+
Pushed by firstname.lastname@example.org: https://hg.mozilla.org/integration/autoland/rev/2b87ee06ac04 Always report pixel difference counts for reftests. r=jmaher
You need to log in before you can comment on or make changes to this bug.