1554342 - Perma [tier 2] /css/css-contain/contain-layout-baseline-005.html | Testing http://web-platform.test:8000/css/css-contain/contain-layout-baseline-005.html == http://web-platform.test:8000/css/css-contain/reference/contain-layout-baseline-005-ref.html

Status: RESOLVED FIXED

(Core :: CSS Parsing and Computation, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: cbrindusan [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=248284445&repo=mozilla-central
Full log: https://queue.taskcluster.net/v1/task/LlEap-54QKeDAi0-QcorMA/runs/1/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/LlEap-54QKeDAi0-QcorMA/runs/1/artifacts/public/logs/live_backing.log&only_show_unexpected=1

---

[task 2019-05-24T23:56:39.250Z] 23:56:39 INFO - TEST-START | /css/css-contain/contain-layout-baseline-005.html
[task 2019-05-24T23:56:40.152Z] 23:56:40 INFO - TEST-UNEXPECTED-FAIL | /css/css-contain/contain-layout-baseline-005.html | Testing http://web-platform.test:8000/css/css-contain/contain-layout-baseline-005.html == http://web-platform.test:8000/css/css-contain/reference/contain-layout-baseline-005-ref.html

Comment 1

[Intermittent Failures Robot]

34 failures in 4730 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 33
• try: 1

Platform breakdown:

• android-em-7-0-x86: 10
• android-em-7-0-x86_64: 24

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1554342&startday=2019-05-20&endday=2019-05-26&tree=all

Comment 2

[Intermittent Failures Robot]

19 failures in 841 pushes (0.023 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 18
• autoland: 1

Platform breakdown:

• android-em-7-0-x86: 7
• android-em-7-0-x86_64: 12

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1554342&startday=2019-05-28&endday=2019-05-28&tree=all

Comment 3

[Intermittent Failures Robot]

56 failures in 4337 pushes (0.013 failures/push) were associated with this bug in the last 7 days.

This is the #35 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 54
• autoland: 1
• try: 1

Platform breakdown:

• android-em-7-0-x86: 36
• android-em-7-0-x86_64: 20

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1554342&startday=2019-05-27&endday=2019-06-02&tree=all

Comment 4

[Intermittent Failures Robot]

15 failures in 997 pushes (0.015 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 5
• try: 10

Platform breakdown:

• android-em-7-0-x86: 5
• android-em-7-0-x86_64: 10

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1554342&startday=2019-06-05&endday=2019-06-05&tree=all

Comment 5

[Intermittent Failures Robot]

21 failures in 1101 pushes (0.019 failures/push) were associated with this bug yesterday.

Repository breakdown:

• try: 20
• mozilla-central: 1

Platform breakdown:

• android-em-7-0-x86_64: 20
• android-em-7-0-x86: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1554342&startday=2019-06-06&endday=2019-06-06&tree=all

Comment 6

[Intermittent Failures Robot]

48 failures in 5197 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

This is the #40 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 16
• try: 32

Platform breakdown:

• android-em-7-0-x86: 18
• android-em-7-0-x86_64: 30

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1554342&startday=2019-06-03&endday=2019-06-09&tree=all

Comment 7

[Intermittent Failures Robot]

1 failures in 4753 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform breakdown:

• android-em-7-0-x86: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1554342&startday=2019-06-10&endday=2019-06-16&tree=all

Comment 8

[Intermittent Failures Robot]

1 failures in 4409 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform breakdown:

• android-em-7-0-x86: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1554342&startday=2019-06-24&endday=2019-06-30&tree=all

Comment 9

[Daniel Holbert [:dholbert]]

This looks basically like a form of bug 1480862. (2 fuzzy pixels at the corner of a native widget -- a <select> dropdown button in this case)

We should annotate this test as having 2 pixels fuzzy, rather than annotating it as failing as we currently have done.

Comment 10

[Daniel Holbert [:dholbert]]

Right now it's annotated as failing on android, which is why we haven't been getting reports recently. But, per comment 9, we should just annotate it as fuzzy instead.

Comment 11

[Daniel Holbert [:dholbert]]

So in an x86_64 emulator log as well as an x86 emulator log (both from before we annotated this test as failing), I see the following for this test failure:

  INFO    Found 2 pixels different, maximum difference per channel 9

I'll just use those numbers in the annotation. For now, I'll expect precisely that much to differ, so we can tell if the issue goes away. (This is a reliable failure, given that we've annotated this test as reliably failing right now without incurring "unexpected-pass" oranges yet as far as I'm aware. And it seems likely that we're always failing in that same way.)

Comment 12

[Daniel Holbert [:dholbert]]

Attachment: Bug 1554342: Annotate web-platform-test contain-layout-baseline-005.html as fuzzy rather than failing. (no review, just adjusting a test-annotation)

This test reliably fails (at least, it's currently annotated as such), but the
failure appears to always only have 2px differing with maximum color-channel
difference of 9. This patch tightens the annotation to express precisely that
expectation.

Comment 13

[Daniel Holbert [:dholbert]]

Try run: https://treeherder.mozilla.org/#/jobs?repo=try&revision=923269a38517e23453e8e51f1982b9fe6e107512

(Try run is only exercising the x86_64 android emulator, but that should be sufficient. I couldn't figure out how to get ./mach try fuzzy to offer me x86 android emulator web-platform-test-runs)

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/3dfc7260f081
Annotate web-platform-test contain-layout-baseline-005.html as fuzzy rather than failing. (no review, just adjusting a test-annotation)

Comment 15

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/3dfc7260f081