1644632 - Intermittent tier2 bugs/456219-1a.html == bugs/456219-1-ref.html | image comparison, max difference: 60, number of differing pixels: 540

Status: RESOLVED FIXED

(Core :: Layout, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: nerli [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=305743681&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/MX-QHTIBSTC917Ggu-5-ZQ/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/MX-QHTIBSTC917Ggu-5-ZQ/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

---

[task 2020-06-10T03:33:35.376Z] 03:33:35 INFO - REFTEST TEST-START | layout/reftests/bugs/456219-1a.html == layout/reftests/bugs/456219-1-ref.html
[task 2020-06-10T03:33:35.376Z] 03:33:35 INFO - REFTEST TEST-LOAD | http://10.7.205.221:8854/tests/layout/reftests/bugs/456219-1a.html | 1325 / 2151 (61%)
[task 2020-06-10T03:33:35.376Z] 03:33:35 INFO - REFTEST TEST-LOAD | http://10.7.205.221:8854/tests/layout/reftests/bugs/456219-1-ref.html | 1325 / 2151 (61%)
[task 2020-06-10T03:33:35.376Z] 03:33:35 INFO - REFTEST INFO | REFTEST fuzzy test (56, 449) <= (60, 540) <= (60, 528)
[task 2020-06-10T03:33:35.477Z] 03:33:35 WARNING - REFTEST TEST-UNEXPECTED-FAIL | layout/reftests/bugs/456219-1a.html == layout/reftests/bugs/456219-1-ref.html | image comparison, max difference: 60, number of differing pixels: 540

Comment 1

[Noemi Erli[:noemi_erli]]

This is a high frequency failure as far as we can see here: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&resultStatus=success%2Ctestfailed%2Cbusted%2Cexception&fromchange=b7876423a11f066001bf5b0ce45423238b02d38d&searchStr=android%2C8.0%2Cpixel2%2Caarch64%2Cwebrender%2Copt%2Creftests%2Ctest-android-hw-p2-8-0-android-aarch64-qr%2Fopt-geckoview-reftest-qr-e10s-1%2Cr%28r1%29&selectedTaskRun=MX-QHTIBSTC917Ggu-5-ZQ-0
Seems to be triggered somehow from Bug 1641751
Glenn, could you please take a look at this?

Comment 2

[Glenn Watson [:gw]]

It looks like the fuzziness probably needs to be increased on these - I'll take a look now.

Comment 3

[Glenn Watson [:gw]]

Attachment: Bug 1644632 - Update fuzziness for android test on webrender.

The change in texture cache eviction policy made this slightly
more fuzzy on android hardware.

Comment 4

[Glenn Watson [:gw]]

Yup, that test is already fuzzy on all gfx backends.

The texture cache patch can result in items being allocated in slightly different texture cache locations, which can alter fuzziness on tests that have existing accuracy issues, so I added an extra 12 pixels fuzziness for these two tests.

I haven't tested that patch on try, but I think I got the annotations right - let me know if they seem wrong!

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f286f22e7dc7
Update fuzziness for android test on webrender. r=nical

Comment 6

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/f286f22e7dc7

Comment 8

[Alexandru Michis [:malexandru]]

Issue looks to still be present:
https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&resultStatus=success%2Ctestfailed%2Cbusted%2Cexception%2Crunnable&searchStr=android%2C8.0%2Cpixel2%2Caarch64%2Cwebrender%2Copt%2Creftests%2Ctest-android-hw-p2-8-0-android-aarch64-qr%2Fopt-geckoview-reftest-qr-e10s-1%2Cr%28r1%29&revision=c7dbcbcbb07745ad124640bc639c86a251c66300&selectedTaskRun=f1NrErXbRNirkRIuXvleDQ-0

Glenn, could you please take a look?

Comment 10

[Glenn Watson [:gw]]

It looks like I was too conservative with the fuzziness amount - it sometimes gets 544 fuzzy pixels instead of the 540 fuzzy pixels I adjusted it to. I'll post an updated patch here shortly.

Comment 11

[Glenn Watson [:gw]]

Attachment: Bug 1644632 - Add more fuzziness to android test.

The exact fuzziness amount depends on where it's placed in the
texture cache, which depends on previous evictions.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7381ed4dde17
Add more fuzziness to android test. r=jrmuizel

Comment 13

[Glenn Watson [:gw]]

Updated fuzziness patch uploaded and pushed.

Comment 14

[Intermittent Failures Robot]

18 failures in 1067 pushes (0.017 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 18

Platform breakdown:

• android-hw-p2-8-0-android-aarch64-qr: 18

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1644632&startday=2020-06-10&endday=2020-06-10&tree=all

Comment 15

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/7381ed4dde17

Comment 16

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1641751

Comment 17

[Intermittent Failures Robot]

29 failures in 5211 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 29

Platform breakdown:

• android-hw-p2-8-0-android-aarch64-qr: 29

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1644632&startday=2020-06-08&endday=2020-06-14&tree=all