1865654 - Intermittent pixel-rounding/border-image-width-4.html == pixel-rounding/border-image-width-0.html | single tracking bug

Status: RESOLVED FIXED

(Core :: Graphics: WebRender, defect)

Description

[Treeherder Bug Filer]

Filed by: smolnar [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=436846768&repo=try
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/X9Xz1yDIQ0uwg9PXoMC1yA/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/X9Xz1yDIQ0uwg9PXoMC1yA/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

---

REFTEST TEST-START | layout/reftests/pixel-rounding/border-image-width-4.html == layout/reftests/pixel-rounding/border-image-width-0.html
[task 2023-11-20T13:01:53.859Z] 13:01:53     INFO - REFTEST TEST-LOAD | file:///Z:/task_170048272076390/build/tests/reftest/tests/layout/reftests/pixel-rounding/border-image-width-4.html | 158 / 163 (96%)
[task 2023-11-20T13:01:53.938Z] 13:01:53     INFO - REFTEST INFO | REFTEST fuzzy test (0, 0) <= (112, 29) <= (112, 25)
[task 2023-11-20T13:01:54.215Z] 13:01:54     INFO - REFTEST TEST-UNEXPECTED-FAIL | layout/reftests/pixel-rounding/border-image-width-4.html == layout/reftests/pixel-rounding/border-image-width-0.html | image comparison, max difference: 112, number of differing pixels: 29
[task 2023-11-20T13:01:54.216Z] 13:01:54     INFO - REFTEST   IMAGE 1 (TEST)
INFO - REFTEST   IMAGE 2 (REFERENCE):
INFO - REFTEST TEST-END | layout/reftests/pixel-rounding/border-image-width-4.html == layout/reftests/pixel-rounding/border-image-width-0.html

Comment 1

[Daniel Holbert [:dholbert]]

This is just an incremental iteration of a known-fuzzy issue discussed in bug 1842623.

See bug 1842623 comment 9 in particular. We had an annotation with a differing-pixel-count of 21, and we bumped it to 25 in that bug (+4), and it looks like we need to bump it to 29 (+4 again) now.

Though also, it's interesting that we didn't hit this for months and now we've hit it 3 times in the past week somehow, based on the orange factor graph? All three of the failures seem to be on a single platform on a single try run:
Windows 11 x64 22H2 WebRender DevEdition opt
https://treeherder.mozilla.org/jobs?repo=try&revision=080000b5e337362a9b5887dad2b7f5ad31598111&searchStr=Windows%2C11%2Cx64%2C22H2%2CWebRender%2CDevEdition%2Copt%2CR3

...for the "staging release: 121.0b1" push. So maybe this is worse with some nightly-only feature preffed off, specifically on Windows, for some reason?

nical, do you know what might be going on (what preffable features might be turned off when we switch to beta) that makes extra pixels fuzzy here? And should we just update the annotation to reflect reality (s/25/29/), or do you want to poke at anything else?

Comment 2

[Daniel Holbert [:dholbert]]

(I can also imagine this being some sort of spooky-action-at-a-distance, where a pref-twiddle for some earlier test (to enable an otherwise nightly-only-feature) turns out to induce the intermittent behavior here. That sort of thing could explain the beta-vs-nightly release-train change in behavior.)

Comment 3

[Nicolas Silva [:nical]]

I can't think of recent changes that could have affected borders, but forwarding the needinfo to Glenn since I haven't touched webrender in a long time.

This test seems to exacerbate float arithmetic issues. There is a lot going on in the border shader so even a driver update might have made things change slightly.
We should probably just increase the reftest fuzziness.

Comment 4

[Glenn Watson [:gw]]

Nothing obvious I can think of that has changed recently in WR to cause that. I do recall that some of the Windows CI machines were recently updated (perhaps Windows version?) or something like that? Perhaps that might explain what changed?

Comment 5

[Daniel Holbert [:dholbert]]

This seems to be specific to this week's merge-to-beta, so it's something specific to that configuration, I think (presumably some feature or set of tests being disabled on beta, in a way that has a direct or indirect/downstream effect on this tests).

Here's the intermittent-failures view:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?startday=2023-11-14&endday=2023-11-21&tree=all&failurehash=all&bug=1865654

Note that all of the failures are on try "nightly-as-beta/staging" releases, or on actual-beta.

Anyway: seems not-concerning, so we should just bump up the fuzzy pixel count to 29 for border-image-width-4.html and border-image-width-9.html. (This bug was filed for the former test, but about half of the failure logs have the same failure in the latter test too, e.g. this one.)

Comment 6

[Intermittent Failures Robot]

15 failures in 3623 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 9
• mozilla-beta: 6

Platform and build breakdown:

• windows11-64-2009-devedition-qr: 15
  • opt: 15

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1865654&startday=2023-11-20&endday=2023-11-26&tree=all

Comment 7

[Intermittent Failures Robot]

13 failures in 4108 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 13

Platform and build breakdown:

• windows11-64-2009-devedition-qr: 13
  • opt: 13

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1865654&startday=2023-11-27&endday=2023-12-03&tree=all

Comment 8

[Intermittent Failures Robot]

13 failures in 3671 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 8
• try: 5

Platform and build breakdown:

• windows11-64-2009-devedition-qr: 13
  • opt: 13

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1865654&startday=2023-12-04&endday=2023-12-10&tree=all

Comment 9

[Intermittent Failures Robot]

5 failures in 4224 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 5

Platform and build breakdown:

• windows11-64-2009-devedition-qr: 5
  • opt: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1865654&startday=2023-12-11&endday=2023-12-17&tree=all

Comment 10

[Intermittent Failures Robot]

30 failures in 3349 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• try: 4
• mozilla-beta: 7
• autoland: 16
• mozilla-central: 3

Platform and build breakdown:

• windows11-64-2009-devedition-qr: 10
  • opt: 10
• windows11-32-2009-qr: 20
  • debug: 20

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1865654&startday=2023-12-18&endday=2023-12-24&tree=all

Comment 11

[Intermittent Failures Robot]

24 failures in 1162 pushes (0.021 failures/push) were associated with this bug in the last 7 days.

This is the #30 most frequent failure this week.

Repository breakdown:

• autoland: 12
• mozilla-central: 7
• mozilla-beta: 4
• try: 1

Platform and build breakdown:

• windows11-32-2009-qr: 20
  • debug: 20
• windows11-64-2009-devedition-qr: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1865654&startday=2023-12-25&endday=2023-12-31&tree=all

Comment 12

[Intermittent Failures Robot]

24 failures in 2274 pushes (0.011 failures/push) were associated with this bug in the last 7 days.

This is the #45 most frequent failure this week.

Repository breakdown:

• mozilla-beta: 7
• mozilla-central: 8
• autoland: 8
• try: 1

Platform and build breakdown:

• windows11-64-2009-devedition-qr: 7
  • opt: 7
• windows11-32-2009-qr: 17
  • debug: 17

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1865654&startday=2024-01-01&endday=2024-01-07&tree=all

Comment 13

[Intermittent Failures Robot]

41 failures in 3217 pushes (0.013 failures/push) were associated with this bug in the last 7 days.

This is the #25 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 6
• try: 5
• autoland: 21
• mozilla-beta: 9

Platform and build breakdown:

• windows11-32-2009-qr: 32
  • debug: 32
• windows11-64-2009-devedition-qr: 9
  • opt: 9

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1865654&startday=2024-01-08&endday=2024-01-14&tree=all

Comment 14

[Intermittent Failures Robot]

31 failures in 3599 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

This is the #30 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 16
• mozilla-central: 7
• mozilla-beta: 2
• try: 5
• pine: 1

Platform and build breakdown:

• windows11-32-2009-qr: 29
  • debug: 29
• windows11-64-2009-devedition-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1865654&startday=2024-01-15&endday=2024-01-21&tree=all

Comment 15

[Daniel Holbert [:dholbert]]

These failures still show 28 or 29 pixels mismatching, for border-image-width-4.html and/or border-image-width-9.html

Recent examples of each:
28px: https://treeherder.mozilla.org/logviewer?job_id=444046236&repo=mozilla-central&lineNumber=6278
29px: https://treeherder.mozilla.org/logviewer?job_id=443346131&repo=mozilla-beta&lineNumber=9596

Let's just bump the threshold to allow 29, since this increase-in-fuzziness is small and seems to be unconcerning as discussed above.

Comment 16

[Daniel Holbert [:dholbert]]

Attachment: Bug 1865654: Bump fuzzy threshold to reflect reality, for reftests border-image-width-4.html and -9.html. r?gw,nical

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4a31b5feaf19
Bump fuzzy threshold to reflect reality, for reftests border-image-width-4.html and -9.html. r=TYLin

Comment 18

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/4a31b5feaf19

Comment 19

[Intermittent Failures Robot]

10 failures in 4034 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2
• autoland: 5
• mozilla-beta: 3

Platform and build breakdown:

• windows11-32-2009-qr: 10
  • debug: 10

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1865654&startday=2024-01-22&endday=2024-01-28&tree=all

Comment 20

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-beta/rev/6da99d13d244