Open Bug 1460502 Opened 2 years ago Updated 5 days ago

Intermittent reftest-sanity/async-scroll-1b.html != reftest-sanity/async-scroll-1-ref.html | image comparison, max difference: 0, number of differing pixels: 0

Categories

(Testing :: Reftest, defect, P5)

Version 3
x86_64
macOS
defect

Tracking

(Not tracked)

People

(Reporter: intermittent-bug-filer, Assigned: kats)

Details

(Keywords: intermittent-failure, Whiteboard: [stockwell unknown])

Filed by: rgurzau [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=177766266&repo=autoland

https://queue.taskcluster.net/v1/task/MqdyZ9GQQgKWLCrcAqieNw/runs/0/artifacts/public/logs/live_backing.log

https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/MqdyZ9GQQgKWLCrcAqieNw/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

16:42:03     INFO - REFTEST TEST-START | file:///Users/cltbld/tasks/task_1525907760/build/tests/reftest/tests/layout/reftests/reftest-sanity/async-scroll-1b.html != file:///Users/cltbld/tasks/task_1525907760/build/tests/reftest/tests/layout/reftests/reftest-sanity/async-scroll-1-ref.html
16:42:03     INFO - REFTEST TEST-LOAD | file:///Users/cltbld/tasks/task_1525907760/build/tests/reftest/tests/layout/reftests/reftest-sanity/async-scroll-1b.html | 102 / 123 (82%)
16:42:04     INFO - ++DOMWINDOW == 75 (0x124d84400) [pid = 1020] [serial = 208] [outer = 0x11273a600]
16:42:04     INFO - REFTEST TEST-LOAD | file:///Users/cltbld/tasks/task_1525907760/build/tests/reftest/tests/layout/reftests/reftest-sanity/async-scroll-1-ref.html | 102 / 123 (82%)
16:42:04     INFO - ++DOMWINDOW == 76 (0x124165c00) [pid = 1020] [serial = 209] [outer = 0x11273a600]
16:42:04     INFO - REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/tasks/task_1525907760/build/tests/reftest/tests/layout/reftests/reftest-sanity/async-scroll-1b.html != file:///Users/cltbld/tasks/task_1525907760/build/tests/reftest/tests/layout/reftests/reftest-sanity/async-scroll-1-ref.html | image comparison, max difference: 0, number of differing pixels: 0
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → INCOMPLETE

This seems to be a persistent failure throughout time, happening on mac builds: https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2019-10-11&endday=2020-02-08&tree=all&bug=1460502

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=288001527&repo=mozilla-central&lineNumber=3733

[task 2020-02-07T23:42:51.686Z] 23:42:51 INFO - REFTEST TEST-START | layout/reftests/reftest-sanity/async-scroll-1b.html != layout/reftests/reftest-sanity/async-scroll-1-ref.html
[task 2020-02-07T23:42:51.686Z] 23:42:51 INFO - REFTEST TEST-LOAD | file:///Users/cltbld/tasks/task_1581117414/build/tests/reftest/tests/layout/reftests/reftest-sanity/async-scroll-1b.html | 104 / 126 (82%)
[task 2020-02-07T23:42:51.697Z] 23:42:51 INFO - REFTEST TEST-LOAD | file:///Users/cltbld/tasks/task_1581117414/build/tests/reftest/tests/layout/reftests/reftest-sanity/async-scroll-1-ref.html | 104 / 126 (82%)
[task 2020-02-07T23:42:51.774Z] 23:42:51 INFO - REFTEST TEST-UNEXPECTED-FAIL | layout/reftests/reftest-sanity/async-scroll-1b.html != layout/reftests/reftest-sanity/async-scroll-1-ref.html | image comparison, max difference: 0, number of differing pixels: 0

Andrew can you assign someone to take a look?

Flags: needinfo?(ahal)
Whiteboard: [stockwell needswork:owner]

Hi David,

:kats had volunteered to take over reftest triage ownership duties from me when he gets back from his PTO. In the meantime, could you help find someone to take a look at this?

Thanks!

Flags: needinfo?(ahal) → needinfo?(dbaron)
Status: REOPENED → NEW
OS: Unspecified → macOS
Hardware: Unspecified → x86_64

There are 23 failures associated to this bug in the last 7 days. These are occurring on:

  • macosx1014-64, debug
  • macosx1014-64-qr, debug
  • macosx1014-64-shippable, opt

log: https://treeherder.mozilla.org/logviewer.html#?job_id=299774644&repo=autoland
[task 2020-04-28T12:27:56.851Z] 12:27:56 INFO - REFTEST TEST-START | layout/reftests/reftest-sanity/async-scroll-1b.html != layout/reftests/reftest-sanity/async-scroll-1-ref.html
[task 2020-04-28T12:27:56.851Z] 12:27:56 INFO - REFTEST TEST-LOAD | file:///Users/cltbld/tasks/task_1588076166/build/tests/reftest/tests/layout/reftests/reftest-sanity/async-scroll-1b.html | 104 / 126 (82%)
[task 2020-04-28T12:27:56.852Z] 12:27:56 INFO - REFTEST TEST-LOAD | file:///Users/cltbld/tasks/task_1588076166/build/tests/reftest/tests/layout/reftests/reftest-sanity/async-scroll-1-ref.html | 104 / 126 (82%)
[task 2020-04-28T12:27:56.923Z] 12:27:56 INFO - REFTEST TEST-UNEXPECTED-FAIL | layout/reftests/reftest-sanity/async-scroll-1b.html != layout/reftests/reftest-sanity/async-scroll-1-ref.html | image comparison, max difference: 0, number of differing pixels: 0
[task 2020-04-28T12:27:56.924Z] 12:27:56 INFO - REFTEST INFO | Saved log: START file:///Users/cltbld/tasks/task_1588076166/build/tests/reftest/tests/layout/reftests/reftest-sanity/async-scroll-1b.html
[task 2020-04-28T12:27:56.924Z] 12:27:56 INFO - REFTEST INFO | Saved log: [CONTENT] Setting displayport to <x=0, y=0, w=800, h=1000>
[task 2020-04-28T12:27:56.924Z] 12:27:56 INFO - REFTEST INFO | Saved log: [CONTENT] OnDocumentLoad triggering AfterOnLoadScripts
[task 2020-04-28T12:27:56.925Z] 12:27:56 INFO - REFTEST INFO | Saved log: Initializing canvas snapshot
[task 2020-04-28T12:27:56.925Z] 12:27:56 INFO - REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000
[task 2020-04-28T12:27:56.925Z] 12:27:56 INFO - REFTEST INFO | Saved log: [CONTENT] RecordResult fired
[task 2020-04-28T12:27:56.925Z] 12:27:56 INFO - REFTEST INFO | Saved log: RecordResult fired
[task 2020-04-28T12:27:56.926Z] 12:27:56 INFO - REFTEST INFO | Saved log: START file:///Users/cltbld/tasks/task_1588076166/build/tests/reftest/tests/layout/reftests/reftest-sanity/async-scroll-1-ref.html
[task 2020-04-28T12:27:56.926Z] 12:27:56 INFO - REFTEST INFO | Saved log: [CONTENT] OnDocumentLoad triggering AfterOnLoadScripts
[task 2020-04-28T12:27:56.926Z] 12:27:56 INFO - REFTEST INFO | Saved log: Initializing canvas snapshot
[task 2020-04-28T12:27:56.926Z] 12:27:56 INFO - REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000
[task 2020-04-28T12:27:56.926Z] 12:27:56 INFO - REFTEST INFO | Saved log: [CONTENT] RecordResult fired
[task 2020-04-28T12:27:56.926Z] 12:27:56 INFO - REFTEST INFO | Saved log: RecordResult fired
[task 2020-04-28T12:27:56.926Z] 12:27:56 INFO - REFTEST TEST-END | layout/reftests/reftest-sanity/async-scroll-1b.html != layout/reftests/reftest-sanity/async-scroll-1-ref.html

:kats can you take a look?

Flags: needinfo?(kats)

This is somewhat perplexing. Looking at the reftest analyzer comparisons shows two different failures. The -qr builds have a green rect all the way to the bottom (800x1000) while the non-qr builds are missing the bottom 50 pixels of green. As far as I can tell, what's supposed to happen is that the test page (-1b.html) is supposed to have the bottom 50 pixels cut off, due to being outside the displayport. And the reference page (-1-ref.html) should have the bottom 50 pixels of green, but with a yellow square.

So this means the -qr builds are rendering both test and reference wrong, and the non-qr builds are rendering the reference page wrong.

If the reference is rendering incorrectly, as it appears to be, that's a somewhat bigger problem, because it means the -1a.html test which also uses the same reference might be incorrectly passing.

I'll try to dig in a bit here, parking with me for now.

Assignee: nobody → kats
Flags: needinfo?(kats)
Flags: needinfo?(dbaron)
You need to log in before you can comment on or make changes to this bug.