1604338 - ubuntu1804 - layout/reftests/async-scrolling/reftest.list | multiple failures and unexpected passes

Status: RESOLVED FIXED

(Core :: Layout, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: egao [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=281172941&repo=try
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/ETYg40ybRz-2tUi6ORtpEA/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/ETYg40ybRz-2tUi6ORtpEA/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

---

suite: reftest
chunk: 2

notes: asan, qr

context:
[task 2019-12-14T09:54:10.996Z] 09:54:10 INFO - REFTEST TEST-START | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/async-scrolling/position-fixed-transformed-1.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/async-scrolling/position-fixed-transformed-1-ref.html
[task 2019-12-14T09:54:10.998Z] 09:54:10 INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/async-scrolling/position-fixed-transformed-1.html | 25 / 87 (28%)
[task 2019-12-14T09:54:11.349Z] 09:54:11 INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/async-scrolling/position-fixed-transformed-1-ref.html | 25 / 87 (28%)
[task 2019-12-14T09:54:11.524Z] 09:54:11 INFO - REFTEST INFO | REFTEST fuzzy test (60, 14) <= (32, 32) <= (60, 14)
[task 2019-12-14T09:54:11.744Z] 09:54:11 INFO - REFTEST TEST-UNEXPECTED-FAIL | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/async-scrolling/position-fixed-transformed-1.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/async-scrolling/position-fixed-transformed-1-ref.html | image comparison, max difference: 32, number of differing pixels: 32

Comment 1

[Edwin Takahashi (:egao | infrequent contributor)]

There appears to be ~18 or so failures related to the fact that test is run with -qr enabled.

Bunch of the new failures I spot-checked already have annotations marking webrender&&gtkWidget with fuzz values.

:svoisen - given that these 18 or so new failures are due to fuzz values deviating (in both directions, up and down) by a small margin, would it be acceptable to:

• temporarily loosen the values while ubuntu1604 and ubuntu1804 coexist; then
• tighten the fuzz values around the ubuntu1804 values once ubuntu1604 is switched off

Comment 2

[Edwin Takahashi (:egao | infrequent contributor)]

leave-open so that once ubuntu1804 migration is complete I can tighten up the ranges to the new values produced by ubuntu1804.

Comment 3

[Edwin Takahashi (:egao | infrequent contributor)]

Attachment: Bug 1604338 - annotate unexpected outcomes for layout/reftests/async-scrolling/reftest.list when run on ubuntu1804 with webrender

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7b62deb7c22b
annotate unexpected outcomes for layout/reftests/async-scrolling/reftest.list when run on ubuntu1804 with webrender r=jmaher

Comment 5

[Oana Pop-Rus]

https://hg.mozilla.org/mozilla-central/rev/7b62deb7c22b

Comment 6

[Sean Voisen (:svoisen)]

Clearing my old needinfo — Edwin and I discussed the proposal in comment 1, which seems fine.