Closed Bug 1558816 Opened 5 years ago Closed 5 years ago

Intermittent TEST-UNEXPECTED-PASS | editor/reftests/spellcheck-hyphen-multiple-invalid.html != editor/reftests/spellcheck-hyphen-multiple-invalid-ref.html | image comparison, max difference: 1, number of differing pixels: 1

Categories

(Core :: DOM: Editor, defect, P5)

x86_64
Android
defect

Tracking

()

RESOLVED INCOMPLETE

People

(Reporter: intermittent-bug-filer, Unassigned)

References

(Regression)

Details

(Keywords: intermittent-failure, regression)

Filed by: malexandru [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=251383178&repo=autoland
Full log: https://queue.taskcluster.net/v1/task/XNiGAu7aSTCPXTycXEZbBg/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/XNiGAu7aSTCPXTycXEZbBg/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1


task 2019-06-12T13:16:02.394Z] 13:16:02 INFO - REFTEST TEST-START | http://10.0.2.2:8854/tests/editor/reftests/spellcheck-hyphen-multiple-invalid.html != http://10.0.2.2:8854/tests/editor/reftests/spellcheck-hyphen-multiple-invalid-ref.html
[task 2019-06-12T13:16:02.394Z] 13:16:02 INFO - REFTEST TEST-LOAD | http://10.0.2.2:8854/tests/editor/reftests/spellcheck-hyphen-multiple-invalid.html | 2815 / 3069 (91%)
[task 2019-06-12T13:16:02.396Z] 13:16:02 INFO - REFTEST TEST-LOAD | http://10.0.2.2:8854/tests/editor/reftests/spellcheck-hyphen-multiple-invalid-ref.html | 2815 / 3069 (91%)
[task 2019-06-12T13:16:02.398Z] 13:16:02 INFO - REFTEST TEST-UNEXPECTED-PASS | http://10.0.2.2:8854/tests/editor/reftests/spellcheck-hyphen-multiple-invalid.html != http://10.0.2.2:8854/tests/editor/reftests/spellcheck-hyphen-multiple-invalid-ref.html | image comparison, max difference: 1, number of differing pixels: 1
[task 2019-06-12T13:16:02.403Z] 13:16:02 INFO - REFTEST IMAGE 1 (TEST): ...
[task 2019-06-12T13:16:02.409Z] 13:16:02 INFO - REFTEST IMAGE 2 (REFERENCE): ...
[task 2019-06-12T13:16:02.410Z] 13:16:02 INFO - REFTEST INFO | Saved log: START http://10.0.2.2:8854/tests/editor/reftests/spellcheck-hyphen-multiple-invalid.html
[task 2019-06-12T13:16:02.411Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] OnDocumentLoad triggering WaitForTestEnd
[task 2019-06-12T13:16:02.411Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] WaitForTestEnd: Adding listeners
[task 2019-06-12T13:16:02.411Z] 13:16:02 INFO - REFTEST INFO | Saved log: Initializing canvas snapshot
[task 2019-06-12T13:16:02.412Z] 13:16:02 INFO - REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000
[task 2019-06-12T13:16:02.412Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_TO_FIRE_INVALIDATE_EVENT
[task 2019-06-12T13:16:02.412Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: waiting for MozAfterPaint
[task 2019-06-12T13:16:02.413Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] AfterPaintListener in http://10.0.2.2:8854/tests/editor/reftests/spellcheck-hyphen-multiple-invalid.html
[task 2019-06-12T13:16:02.413Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] SendUpdateCanvasForEvent with 1 rects
[task 2019-06-12T13:16:02.413Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] Rect: 0 0 800 1000
[task 2019-06-12T13:16:02.414Z] 13:16:02 INFO - REFTEST INFO | Saved log: Updating canvas for invalidation
[task 2019-06-12T13:16:02.414Z] 13:16:02 INFO - REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000
[task 2019-06-12T13:16:02.415Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_TO_FIRE_INVALIDATE_EVENT
[task 2019-06-12T13:16:02.415Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: dispatching MozReftestInvalidate
[task 2019-06-12T13:16:02.415Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_FOR_REFTEST_WAIT_REMOVAL
[task 2019-06-12T13:16:02.415Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_FOR_SPELL_CHECKS
[task 2019-06-12T13:16:02.415Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: waiting for spell checks to end
[task 2019-06-12T13:16:02.416Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_FOR_SPELL_CHECKS
[task 2019-06-12T13:16:02.416Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_FOR_APZ_FLUSH
[task 2019-06-12T13:16:02.417Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: done requesting APZ flush
[task 2019-06-12T13:16:02.417Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: apz-repaints-flushed fired
[task 2019-06-12T13:16:02.418Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_TO_FINISH
[task 2019-06-12T13:16:02.418Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: waiting for MozAfterPaint
[task 2019-06-12T13:16:02.419Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] AfterPaintListener in http://10.0.2.2:8854/tests/editor/reftests/spellcheck-hyphen-multiple-invalid.html
[task 2019-06-12T13:16:02.423Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] SendUpdateCanvasForEvent with 0 rects
[task 2019-06-12T13:16:02.423Z] 13:16:02 INFO - REFTEST INFO | Saved log: Updating canvas for invalidation
[task 2019-06-12T13:16:02.423Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_TO_FINISH
[task 2019-06-12T13:16:02.423Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: Completed
[task 2019-06-12T13:16:02.423Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] RecordResult fired
[task 2019-06-12T13:16:02.423Z] 13:16:02 INFO - REFTEST INFO | Saved log: RecordResult fired
[task 2019-06-12T13:16:02.424Z] 13:16:02 INFO - REFTEST INFO | Saved log: START http://10.0.2.2:8854/tests/editor/reftests/spellcheck-hyphen-multiple-invalid-ref.html
[task 2019-06-12T13:16:02.424Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] OnDocumentLoad triggering AfterOnLoadScripts
[task 2019-06-12T13:16:02.424Z] 13:16:02 INFO - REFTEST INFO | Saved log: Initializing canvas snapshot
[task 2019-06-12T13:16:02.424Z] 13:16:02 INFO - REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000
[task 2019-06-12T13:16:02.424Z] 13:16:02 INFO - REFTEST INFO | Saved log: [CONTENT] RecordResult fired
[task 2019-06-12T13:16:02.424Z] 13:16:02 INFO - REFTEST INFO | Saved log: RecordResult fired
[task 2019-06-12T13:16:02.425Z] 13:16:02 INFO - REFTEST TEST-END | http://10.0.2.2:8854/tests/editor/reftests/spellcheck-hyphen-multiple-invalid.html != http://10.0.2.2:8854/tests/editor/reftests/spellcheck-hyphen-multiple-invalid-ref.html
[task 2019-06-12T13:16:02.426Z] 13:16:02 INFO - REFTEST TEST-START | http://10.0.2.2:8854/tests/editor/reftests/spellcheck-dotafterquote-valid.html == http://10.0.2.2:8854/tests/editor/reftests/spellcheck-dotafterquote-valid-ref.html

Assignee: nobody → kats
Regressed by: geckoview_reftests
OS: Unspecified → Android
Hardware: Unspecified → x86_64

This one is weird.. it's a != test which is already marked as fails-if(Android). Which means on Android the test and ref images are coming out exactly the same. But on the emulator+geckoview there's intermittently a 1px fuzz which makes the test "pass" even though really it shouldn't. The reftest.list file, according to the documentation does support fuzzy-if with != so I guess we can annotate this and it should do the right thing. I've never used this combination before though.

Actually no, if I add a fuzzy-if(geckoview,0-1,0-1) here then it will override the fails-if(Android). I don't think there's anyway to annotate this test so it does what I want.

We should just make this test pass on Android, and then add the fuzzy-if(geckoview,0-1,0-1) annotation. Leaving for editor folks to look into.

Assignee: kats → nobody
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → INCOMPLETE
Has Regression Range: --- → yes
You need to log in before you can comment on or make changes to this bug.