Closed Bug 1571222 Opened 6 years ago Closed 6 years ago

Permafail css-ui-invalid/select/select-disabled-fieldset-1.html == css-ui-invalid/select/select-fieldset-ref.html | image comparison, max difference: 9, number of differing pixels: 1

Categories

(Core :: Layout: Form Controls, defect, P5)

Product:
Core
Component:
Layout: Form Controls
Platform:
ARM
Android
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla70
Tracking Flags:
Tracking Status
firefox-esr60 --- unaffected
firefox-esr68 --- unaffected
firefox68 --- unaffected
firefox69 --- unaffected
firefox70 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: emilio)

References

(Regression)

Details

(Keywords: regression)

Filed by: btara [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=259782619&repo=mozilla-central
Full log: https://queue.taskcluster.net/v1/task/WrHwiE2mTnufkyYlBzlv7Q/runs/1/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/WrHwiE2mTnufkyYlBzlv7Q/runs/1/artifacts/public/logs/live_backing.log&only_show_unexpected=1

[task 2019-08-03T15:52:23.873Z] 15:52:23 INFO - REFTEST TEST-START | http://10.0.2.2:8854/tests/layout/reftests/css-ui-invalid/select/select-disabled-fieldset-1.html == http://10.0.2.2:8854/tests/layout/reftests/css-ui-invalid/select/select-fieldset-ref.html
[task 2019-08-03T15:52:23.874Z] 15:52:23 INFO - REFTEST TEST-LOAD | http://10.0.2.2:8854/tests/layout/reftests/css-ui-invalid/select/select-disabled-fieldset-1.html | 3 / 272 (1%)
[task 2019-08-03T15:52:34.091Z] 15:52:34 INFO - REFTEST TEST-LOAD | http://10.0.2.2:8854/tests/layout/reftests/css-ui-invalid/select/select-fieldset-ref.html | 3 / 272 (1%)
[task 2019-08-03T15:52:34.092Z] 15:52:34 INFO - REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/css-ui-invalid/select/select-disabled-fieldset-1.html == http://10.0.2.2:8854/tests/layout/reftests/css-ui-invalid/select/select-fieldset-ref.html | image comparison, max difference: 9, number of differing pixels: 1
[task 2019-08-03T15:52:34.101Z] 15:52:34 INFO - REFTEST IMAGE 1 (TEST): data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAyAAAAPoCAYAA...
[task 2019-08-03T15:52:34.107Z] 15:52:34 INFO - REFTEST IMAGE 2 (REFERENCE): data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAyAAAAPoCAYAAAAmy5qxAAAWY....
[task 2019-08-03T15:52:34.108Z] 15:52:34 INFO - REFTEST INFO | Saved log: START http://10.0.2.2:8854/tests/layout/reftests/css-ui-invalid/select/select-disabled-fieldset-1.html
[task 2019-08-03T15:52:34.108Z] 15:52:34 INFO - REFTEST INFO | Saved log: [CONTENT] OnDocumentLoad triggering WaitForTestEnd
[task 2019-08-03T15:52:34.108Z] 15:52:34 INFO - REFTEST INFO | Saved log: [CONTENT] WaitForTestEnd: Adding listeners
[task 2019-08-03T15:52:34.108Z] 15:52:34 INFO - REFTEST INFO | Saved log: Initializing canvas snapshot
[task 2019-08-03T15:52:34.109Z] 15:52:34 INFO - REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000

David, is this caused by your changes?

Flags: needinfo?(dbaron)
OS: Unspecified → Android
Hardware: Unspecified → ARM

It seems like this is the regression range known so far, though it looks like you've triggered test runs on autoland to narrow it. Is that a correct assumption about the data you're working off of?

I think my patches are pretty unlikely to cause a single pixel difference of that sort. But I also don't see anything else obvious.

Flags: needinfo?(dbaron)
No longer regressed by: 1404868
Pushed by emilio@crisal.io: https://hg.mozilla.org/mozilla-central/rev/22b33b20fba1 Fuzz a test on Android to keep central green. rpending=dbaron

(In reply to Pulsebot from comment #3)

Pushed by emilio@crisal.io:
https://hg.mozilla.org/mozilla-central/rev/22b33b20fba1
Fuzz a test on Android to keep central green. rpending=dbaron

Forgot an a=apavel, who asked me to do that to keep central green. David, mind going over that commit message and review that? If you want we can also back it out, but then your patches are going to be backed out too, probably :(

Flags: needinfo?(dbaron)

TLDR my theory so far is that bug 1404868 caused it by adding more reftests.

Emilio's theory from his commit message is plausible, though, given that the patch did add reftests which could change the reftest chunking:

This annotation matches the one in
layout/reftests/css-invalid/select/reftest.list for
select-disabled-fieldset-1.html (which should really produce the same
rendering, and have the same reference).

This is a one-pixel fuzz on the fieldset border near the bottom right corner.

It's unclear which patch of this pushlog, if any, caused this:

  https://hg.mozilla.org/mozilla-central/pushloghtml?changeset=4a49d88894d8d88f87760ac59ae35b2158fab7b2

Probably bug 1404868, which added reftests and tickled something. But it seems
this is not really that patches' fault, so to avoid speculatively backing it
out, with the chance of it not being the offender, let's mark it as fuzzy for
now.

I kept the exact same annotation as the other test, since this goes directly to
central, so it's less risky.

We could try to make it fuzzy-if(Android,9-9,1-1), maybe, though given it seems
to be affected to changes to adjacent reftests that may be unwise.

And yes, the patch looks fine.

Flags: needinfo?(dbaron)

I guess we can call this FIXED by emilio's patch?

Status: NEW → RESOLVED
Closed: 6 years ago
status-firefox70: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla70
Assignee: nobody → emilio

Retriggering the test shows that it was in fact bug 1404868. Prior to bug 1404868, the failing test was test 5 of 273 in the run; after the change it was test 3 of 272. So it seems like moving it earlier in the run may have triggered this somehow.

Has Regression Range: --- → yes
You need to log in before you can comment on or make changes to this bug.