Closed Bug 1581798 Opened 4 years ago Closed 4 years ago

Perma Fail Tier 2 Android border-image/svg-as-border-image-4b.html == border-image/svg-as-border-image-4-ref.html | image comparison, max difference: X, number of differing pixels: Y

Categories

(Core :: Layout, defect, P5)

Product:
Core
Component:
Layout
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla71
Tracking Flags:
Tracking Status
firefox-esr60 --- unaffected
firefox-esr68 --- unaffected
firefox67 --- unaffected
firefox68 --- unaffected
firefox69 --- unaffected
firefox70 --- unaffected
firefox71 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: nical)

References

(Regression)

Details

(Keywords: regression)

Attachments

(1 file, 1 obsolete file)

Bug 1581798 - Remove fuzziness from reftest svg-as-border-image-4b.html on android. r=jnicol
47 bytes, text/x-phabricator-request
Details | Review

Filed by: btara [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=267024843&repo=autoland
Full log: https://queue.taskcluster.net/v1/task/WCy7IPEPSi6N04hqeOIxVQ/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/WCy7IPEPSi6N04hqeOIxVQ/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

[task 2019-09-17T11:36:26.850Z] 11:36:26 INFO - REFTEST TEST-START | http://10.7.205.221:8854/tests/layout/reftests/border-image/svg-as-border-image-4b.html == http://10.7.205.221:8854/tests/layout/reftests/border-image/svg-as-border-image-4-ref.html
[task 2019-09-17T11:36:26.850Z] 11:36:26 INFO - REFTEST TEST-LOAD | http://10.7.205.221:8854/tests/layout/reftests/border-image/svg-as-border-image-4b.html | 231 / 1903 (12%)
[task 2019-09-17T11:36:26.850Z] 11:36:26 INFO - REFTEST TEST-UNEXPECTED-FAIL | http://10.7.205.221:8854/tests/layout/reftests/border-image/svg-as-border-image-4b.html == http://10.7.205.221:8854/tests/layout/reftests/border-image/svg-as-border-image-4-ref.html | image comparison, max difference: 16, number of differing pixels: 94
[task 2019-09-17T11:36:26.988Z] 11:36:26 INFO - REFTEST IMAGE 1 (TEST): data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAyAAAAPoCAYAAAA...
[task 2019-09-17T11:36:26.989Z] 11:36:26 INFO - REFTEST IMAGE 2 (REFERENCE): data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAyAAAAPoCAYAAAAmy5qxAAAgAElEQVR4...
[task 2019-09-17T11:36:26.989Z] 11:36:26 INFO - REFTEST INFO | Saved log: START http://10.7.205.221:8854/tests/layout/reftests/border-image/svg-as-border-image-4b.html
[task 2019-09-17T11:36:26.989Z] 11:36:26 INFO - REFTEST INFO | Saved log: [CONTENT] OnDocumentLoad triggering AfterOnLoadScripts
[task 2019-09-17T11:36:26.989Z] 11:36:26 INFO - REFTEST INFO | Saved log: Initializing canvas snapshot
[task 2019-09-17T11:36:26.989Z] 11:36:26 INFO - REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000
[task 2019-09-17T11:36:26.989Z] 11:36:26 INFO - REFTEST INFO | Saved log: [CONTENT] RecordResult fired
[task 2019-09-17T11:36:26.989Z] 11:36:26 INFO - REFTEST INFO | Saved log: RecordResult fired
[task 2019-09-17T11:36:26.989Z] 11:36:26 INFO - REFTEST INFO | Saved log: RecordResult fired
[task 2019-09-17T11:36:26.989Z] 11:36:26 INFO - REFTEST TEST-END | http://10.7.205.221:8854/tests/layout/reftests/border-image/svg-as-border-image-4b.html == http://10.7.205.221:8854/tests/layout/reftests/border-image/svg-as-border-image-4-ref.html

This seems to start with the changes from bug 1570081.

nical, could you, please, take a look?

Flags: needinfo?(nical.bugzilla)
Regressed by: 1570081

This looks like a subtle antialiasing difference. A fuzzy annotation seems like it'd be probably fine here.

Note that the testcase and reference case have nearly identical markup here -- the only difference is that the testcase has two CSS transforms that should effectively cancel each other out: translate(100px, 200px); on the divs, and transform: translate(-100px, -200px); on the body. Arguably those should 100% cancel out and match the reference case exactly, but if there are technical reasons that it subtly influences antialiasing (perhaps even just from the presence vs. absence of some transform), I'm not going to worry too much.

Seems less than ideal to introduce new things that require fuzz like this, if we can avoid it, though.

Fixed in bug 1581804.

Assignee: nobody → nical.bugzilla
Status: NEW → RESOLVED
Closed: 4 years ago
Flags: needinfo?(nical.bugzilla)
Resolution: --- → FIXED

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=267443839&repo=mozilla-inbound&lineNumber=3829

[task 2019-09-19T13:30:44.153Z] 13:30:44 INFO - REFTEST TEST-START | http://10.7.205.214:8854/tests/layout/reftests/border-image/svg-as-border-image-4b.html == http://10.7.205.214:8854/tests/layout/reftests/border-image/svg-as-border-image-4-ref.html
[task 2019-09-19T13:30:44.153Z] 13:30:44 INFO - REFTEST TEST-LOAD | http://10.7.205.214:8854/tests/layout/reftests/border-image/svg-as-border-image-4b.html | 231 / 1903 (12%)
[task 2019-09-19T13:30:44.153Z] 13:30:44 INFO - REFTEST INFO | REFTEST fuzzy test (16, 90) <= (16, 82) <= (16, 150)
[task 2019-09-19T13:30:44.153Z] 13:30:44 INFO - REFTEST TEST-UNEXPECTED-PASS | http://10.7.205.214:8854/tests/layout/reftests/border-image/svg-as-border-image-4b.html == http://10.7.205.214:8854/tests/layout/reftests/border-image/svg-as-border-image-4-ref.html | image comparison, max difference: 16, number of differing pixels: 82

Nicolas, can you please take a look? This perma failure turns into intermittent one.

https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&resultStatus=pending%2Crunning%2Csuccess%2Csuperseded%2Ctestfailed%2Cbusted%2Cexception%2Crunnable&searchStr=android%2C8.0%2Cpixel2%2Cquantumrender%2Copt%2Creftests%2Ctest-android-hw-p2-8-0-arm7-api-16-qr%2Fopt-geckoview-reftest-e10s-3%2Cr%28r3%29&selectedJob=267417740

Thank you

Status: RESOLVED → REOPENED
Flags: needinfo?(nical.bugzilla)
Resolution: FIXED → ---
Summary: Perma Tier 2 Android border-image/svg-as-border-image-4b.html == border-image/svg-as-border-image-4-ref.html | image comparison, max difference: X, number of differing pixels: Y → Intermittent Tier 2 Android border-image/svg-as-border-image-4b.html == border-image/svg-as-border-image-4-ref.html | image comparison, max difference: X, number of differing pixels: Y

The reftest.list parser is full of mystery and apparently ignores fuzzy-if(geckoview&&webrender,..) probably because of the other fuzzy-if. Short of having found documentation for the parsing rules, I'll simplify the fuzzy-if statement and avoid the footgun at the expense of broadening the fuzziness to other webrender configuration.

Flags: needinfo?(nical.bugzilla)

This seems to perma fail now: https://treeherder.mozilla.org/#/jobs?repo=autoland&selectedJob=269376115&revision=348fa5ed7407d53fd9e3661cf9f55cb187622fbc&searchStr=Android%2C8.0%2CPixel2%2CQuantumRender%2Copt%2CReftests%2Ctest-android-hw-p2-8-0-arm7-api-16-qr%2Fopt-geckoview-reftest-e10s-3%2CR%28R3%29

Summary: Intermittent Tier 2 Android border-image/svg-as-border-image-4b.html == border-image/svg-as-border-image-4-ref.html | image comparison, max difference: X, number of differing pixels: Y → Perma Fail Tier 2 Android border-image/svg-as-border-image-4b.html == border-image/svg-as-border-image-4-ref.html | image comparison, max difference: X, number of differing pixels: Y

More accurately: you mean perma orange, for TEST-UNEXPECTED-PASS.

lsalzman just adjusted the annotation in bug 1584760, and the perma-orange seems to be fallout from that.

Pushed by nsilva@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/0da03dadb7cb
Remove fuzziness from reftest svg-as-border-image-4b.html on android. r=jnicol

https://hg.mozilla.org/mozilla-central/rev/0da03dadb7cb

Status: REOPENED → RESOLVED
Closed: 4 years ago → 4 years ago
status-firefox71: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla71
Attachment #9094150 - Attachment is obsolete: true
Has Regression Range: --- → yes
You need to log in before you can comment on or make changes to this bug.