Closed Bug 1439115 Opened 4 years ago Closed 4 years ago

Categories

(Core :: Layout: Form Controls, defect, P3)

defect

Tracking

()

RESOLVED FIXED
mozilla60
Tracking Status
firefox60 --- fixed

People

(Reporter: gbrown, Unassigned)

References

Details

As seen in https://bugzilla.mozilla.org/show_bug.cgi?id=1433163#c26, 2 css-disabled/select reftests become near perma-fail on Android when test tasks are modified to change the way tests are divided amongst tasks. It looks like these tests depend on state set up by earlier tests.


Push with failures: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=b2fe34d6f8a054fc0f4f64740ad4f984cf647510

Failure log for R13 test: https://treeherder.mozilla.org/logviewer.html#?job_id=162726354&repo=mozilla-inbound&lineNumber=1557
[task 2018-02-16T20:25:45.365Z] 20:25:45     INFO -  REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8888/tests/layout/reftests/css-disabled/select/select-fieldset-4.html == http://10.0.2.2:8888/tests/layout/reftests/css-disabled/select/select-fieldset-ref.html | image comparison, max difference: 12, number of differing pixels: 1

Failure log for R25 test: https://treeherder.mozilla.org/logviewer.html#?job_id=162726661&repo=mozilla-inbound&lineNumber=1571
[task 2018-02-16T20:33:49.469Z] 20:33:49     INFO -  REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8888/tests/layout/reftests/css-disabled/select/select-fieldset-2.html == http://10.0.2.2:8888/tests/layout/reftests/css-disabled/select/select-fieldset-ref-disabled.html | image comparison, max difference: 12, number of differing pixels: 1
https://treeherder.mozilla.org/#/jobs?repo=try&revision=93c46cd871a0ae4cc0f09ebf072b985ca98379f2 demonstrates that these tests fail test-verify: They cannot be run reliably on their own.
[ Triage 2017/02/20: P3 ]
Priority: -- → P3
There was a similar issue hit in bug 1295261, resolved by fuzzy-if(Android). I'll do the same here.

Passes test-verify with the fuzzy-if:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=008fd8950cf6cd862af90eacc7fdb3bea987e4d7

and unblocks bug 1433163:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=3f5a5cdf97f1a5996c09775f6e73e080f23ac206
Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/42248666ddf6
Annotate css-disabled/select reftests so they pass reliably on Android; r=me,a=test-only
Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/671b0faa7112
(follow-up) Annotate more css-disabled/select reftests so they pass reliably on Android; r=me,a=test-only
https://hg.mozilla.org/mozilla-central/rev/42248666ddf6
https://hg.mozilla.org/mozilla-central/rev/671b0faa7112
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla60
See Also: → 1442428
You need to log in before you can comment on or make changes to this bug.