Closed Bug 1584760 Opened 6 months ago Closed 6 months ago

Intermittent TEST-UNEXPECTED-PASS | border-image/svg-as-border-image-4b.html == border-image/svg-as-border-image-4-ref.html | image comparison, max difference: 16, number of differing pixels: 69

Categories

(Core :: Layout, defect, P5)

defect

Tracking

()

RESOLVED FIXED
mozilla71
Tracking Status
firefox-esr60 --- unaffected
firefox-esr68 --- unaffected
firefox67 --- unaffected
firefox68 --- unaffected
firefox69 --- unaffected
firefox70 --- unaffected
firefox71 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: lsalzman)

References

Details

(Keywords: intermittent-failure, regression)

Attachments

(2 files)

Pushed by lsalzman@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/24a4aba3f466
adjust fuzz more for svg-as-border-image-4b.html. r=jrmuizel

:lsalzman - Thanks for this. You probably didn't notice that some of the most recent unexpected passes have 0 differences:

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=269172218&repo=autoland&lineNumber=4320

Expand the range further?

Flags: needinfo?(lsalzman)
Status: NEW → RESOLVED
Closed: 6 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla71
Assignee: nobody → lsalzman

Side note that the logs there do show a bit of difference if you view them in reftest-analyzer, even though we're reporting max difference: 0, number of differing pixels: 0. This is because we apparently "round down" fuzz in some cases via this autofuzz clause:
https://searchfox.org/mozilla-central/rev/35cc00a25c4471993fdaa5761952bd3afd4f1731/layout/tools/reftest/reftest.jsm#1128-1137

As shown in one log:

INFO - REFTEST TEST-LOAD | http://10.7.205.221:8854/tests/layout/reftests/border-image/svg-as-border-image-4b.html | 306 / 1928 (15%)
INFO - REFTEST INFO | REFTEST wr-on-android dropping fuzz of (1, 36) to (0, 0)
INFO - REFTEST INFO | REFTEST fuzzy test (16, 69) <= (0, 0) <= (16, 150)
INFO - REFTEST TEST-UNEXPECTED-PASS | http://10.7.205.221:8854/tests/layout/reftests/border-image/svg-as-border-image-4b.html == http://10.7.205.221:8854/tests/layout/reftests/border-image/svg-as-border-image-4-ref.html | image comparison, max difference: 0, number of differing pixels: 0

Anyway, even the "real" difference here (1,36) is below the acceptable-fuzz-range that we added in this bug. And since autofuzz bumps it down 0,0, we need to allow 0,0 in our ranges.

Status: RESOLVED → REOPENED
Flags: needinfo?(lsalzman)
Resolution: FIXED → ---
Pushed by lsalzman@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/db0a3abfc428
expand fuzz range more for svg-as-border-image-4b.html. r=jrmuizel
Status: REOPENED → RESOLVED
Closed: 6 months ago6 months ago
Resolution: --- → FIXED
Duplicate of this bug: 1585783
You need to log in before you can comment on or make changes to this bug.