1586512 - Perma fail Tier 2 Central TEST-UNEXPECTED-PASS | chrome://reftest/content/forms/textbox/accesskey-4.xul == chrome://reftest/content/forms/textbox/accesskey-4-ref.xul | image comparison, max difference: 0, number of differing pixels: 0

Status: RESOLVED FIXED

(Core :: Layout: Form Controls, defect, P5)

Description

Filed by: btara [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=269903123&repo=mozilla-central
Full log: https://queue.taskcluster.net/v1/task/L1-jm97FS2iC_9J4a1v2Dg/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/L1-jm97FS2iC_9J4a1v2Dg/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

---

[task 2019-10-05T12:19:31.628Z] 12:19:31 INFO - REFTEST TEST-START | chrome://reftest/content/forms/textbox/accesskey-4.xul == chrome://reftest/content/forms/textbox/accesskey-4-ref.xul
[task 2019-10-05T12:19:31.628Z] 12:19:31 INFO - REFTEST TEST-LOAD | chrome://reftest/content/forms/textbox/accesskey-4.xul | 1607 / 1973 (81%)
[task 2019-10-05T12:19:31.628Z] 12:19:31 INFO - REFTEST TEST-LOAD | chrome://reftest/content/forms/textbox/accesskey-4-ref.xul | 1607 / 1973 (81%)
[task 2019-10-05T12:19:31.628Z] 12:19:31 INFO - REFTEST TEST-UNEXPECTED-PASS | chrome://reftest/content/forms/textbox/accesskey-4.xul == chrome://reftest/content/forms/textbox/accesskey-4-ref.xul | image comparison, max difference: 0, number of differing pixels: 0
[task 2019-10-05T12:19:31.629Z] 12:19:31 INFO - REFTEST IMAGE: [edit by dholbert: omitting data URI since it's not useful and it's also in the log]
[task 2019-10-05T12:19:31.629Z] 12:19:31 INFO - REFTEST INFO | Saved log: START chrome://reftest/content/forms/textbox/accesskey-4.xul
[task 2019-10-05T12:19:31.629Z] 12:19:31 INFO - REFTEST INFO | Saved log: [CONTENT] OnDocumentLoad triggering AfterOnLoadScripts
[task 2019-10-05T12:19:31.629Z] 12:19:31 INFO - REFTEST INFO | Saved log: Initializing canvas snapshot
[task 2019-10-05T12:19:31.629Z] 12:19:31 INFO - REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000
[task 2019-10-05T12:19:31.629Z] 12:19:31 INFO - REFTEST INFO | Saved log: [CONTENT] RecordResult fired
[task 2019-10-05T12:19:31.629Z] 12:19:31 INFO - REFTEST INFO | Saved log: RecordResult fired
[task 2019-10-05T12:19:31.629Z] 12:19:31 INFO - REFTEST INFO | Saved log: START chrome://reftest/content/forms/textbox/accesskey-4-ref.xul
[task 2019-10-05T12:19:31.629Z] 12:19:31 INFO - REFTEST INFO | Saved log: [CONTENT] OnDocumentLoad triggering AfterOnLoadScripts
[task 2019-10-05T12:19:31.629Z] 12:19:31 INFO - REFTEST INFO | Saved log: Initializing canvas snapshot
[task 2019-10-05T12:19:31.629Z] 12:19:31 INFO - REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000
[task 2019-10-05T12:19:31.629Z] 12:19:31 INFO - REFTEST INFO | Saved log: [CONTENT] RecordResult fired
[task 2019-10-05T12:19:31.629Z] 12:19:31 INFO - REFTEST INFO | Saved log: RecordResult fired
[task 2019-10-05T12:19:31.629Z] 12:19:31 INFO - REFTEST TEST-END | chrome://reftest/content/forms/textbox/accesskey-4.xul == chrome://reftest/content/forms/textbox/accesskey-4-ref.xul

Comment 1

[Bogdan Tara[:bogdan_tara | bogdant]]

This seems to be introduced by bug 1586336.

Comment 2

[Intermittent Failures Robot]

20 failures in 196 pushes (0.102 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 20

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16-qr: 20

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1586512&startday=2019-10-05&endday=2019-10-05&tree=all

Comment 3

[Intermittent Failures Robot]

23 failures in 4292 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 20
• autoland: 3

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16-qr: 23

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1586512&startday=2019-09-30&endday=2019-10-06&tree=all

Comment 4

[Brian Grinstead [:bgrins]]

(In reply to Intermittent Failures Robot from comment #3)

23 failures in 4292 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 20
• autoland: 3

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16-qr: 23

I would have thought this test was skipped on Android via skip-if(cocoaWidget||Android): https://searchfox.org/mozilla-central/rev/5cb522c7baba24e55874809e0e206b001494c1e9/layout/reftests/forms/textbox/reftest.list#8-9. Daniel, could you help us figure out which conditions need to change in the reftest.list now that this test seems to be passing unexpectedly on "android-hw-p2-8-0-arm7-api-16-qr"?

Comment 6

[Emma Malysz]

Attachment: Bug 1586512, fix the reftest failure annotation to be more specific for Android failure

Comment 7

[Daniel Holbert [:dholbert]]

(In reply to Brian Grinstead [:bgrins] from comment #4)

I would have thought this test was skipped on Android via skip-if(cocoaWidget||Android)

That gets superceded by the "fails-if(webrender)" later in the line. The platform here is Android WebRender, so both of these annotations match it, and I believe we trust the last one to set the behavior/expectation. So we run the test and expect it to fail.

I think Emma's patch is correct, but Try can tell us for sure.

Comment 8

[Intermittent Failures Robot]

25 failures in 785 pushes (0.032 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 10
• mozilla-central: 8
• mozilla-inbound: 1
• try: 6

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16-qr: 25

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1586512&startday=2019-10-08&endday=2019-10-08&tree=all

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7046609ec00a
fix the reftest failure annotation to be more specific for Android failure r=dholbert

Comment 10

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/7046609ec00a

Comment 11

[Intermittent Failures Robot]

51 failures in 3801 pushes (0.013 failures/push) were associated with this bug in the last 7 days.

This is the #35 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 22
• mozilla-inbound: 3
• autoland: 18
• try: 8

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16-qr: 51

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1586512&startday=2019-10-07&endday=2019-10-13&tree=all