1509242 - Intermittent /css/CSS2/colors/color-001.xht | Testing http://web-platform.test:8000/css/CSS2/colors/color-001.xht == http://web-platform.test:8000/css/CSS2/colors/color-001-ref.xht

Status: RESOLVED INCOMPLETE

(Core :: CSS Parsing and Computation, defect, P5)

Description

Filed by: aciure [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=213318098&repo=autoland

https://queue.taskcluster.net/v1/task/Y3Ge_svCR36Z-1yB04PBOg/runs/0/artifacts/public/logs/live_backing.log

https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/Y3Ge_svCR36Z-1yB04PBOg/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

20:25:45     INFO - TEST-START | /css/CSS2/colors/color-001.xht
[...]
20:25:45     INFO - PID 707 | 1542860745578	Marionette	INFO	No differences allowed
20:25:45     INFO - PID 707 | 1542860745578	Marionette	INFO	Found 8735 pixels different, maximum difference per channel 255
20:25:45     INFO - TEST-UNEXPECTED-FAIL | /css/CSS2/colors/color-001.xht | Testing http://web-platform.test:8000/css/CSS2/colors/color-001.xht == http://web-platform.test:8000/css/CSS2/colors/color-001-ref.xht
20:25:45     INFO - TEST-INFO took 540ms

Comment 1

[Daniel Holbert [:dholbert]]

The screenshot shows a lot of large "X" characters that are expected to render as squares (but are instead rendering as "X").

Looks like the Ahem font failed to load.  This is OS X where Ahem+WPT was having too many issues to be usable at all, until some recent change, IIRC -- this might be some lingering bit of whatever was broken before, perhaps?

Comment 2

[Intermittent Failures Robot]

1 failures in 2387 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 1

Platform breakdown:
* osx-10-10: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1509242&startday=2018-11-19&endday=2018-11-25&tree=all

Comment 3

[Intermittent Failures Robot]

4 failures in 2648 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 1
* mozilla-beta: 2
* mozilla-inbound: 1

Platform breakdown:
* macosx64-nightly: 1
* osx-10-10: 2
* macosx64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1509242&startday=2018-12-17&endday=2018-12-23&tree=all

Comment 4

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Comment 5

[Daniel Holbert [:dholbert]]

I believe the bug-reopening here was for this recent failure:
https://treeherder.mozilla.org/logviewer.html#?job_id=252329030&repo=mozilla-beta
...which is a case where we failed every css/CSS2/colors/color-*.xht testcase it seems, due to Ahem failing to load on mac, as described in comment 1.

(I'm not sure if we failed "comprehensively" like this [every test] in comment 0 as well, though I'm betting we probably did. The log from comment 0 has expired now, so it's hard to know.)

Comment 6

[Intermittent Failures Robot]

1 failures in 2581 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform breakdown:

• macosx1010-64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1509242&startday=2019-06-17&endday=2019-06-23&tree=all

Comment 7

[Cristina Coroiu [:ccoroiu]]

Started retriggers:
https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=os%2Cx%2C10.14%2Cshippable%2Copt%2Cweb%2Cplatform%2Ctests%2Ctest-macosx1014-64-shippable%2Fopt-web-platform-tests-reftests-e10s-3%2Cw%28wr3%29&tochange=49a65a87a08f0394d79cb3aa699814f1efb0c632&fromchange=9f0dcec2bc31b04ca2998c7b592c01f064038a76&selectedJob=255322936

Comment 8

[Andreea Pavel [:apavel]]

James recent occurrences seems to be from bug 1505739, can you take a look?

Comment 9

[Daniel Holbert [:dholbert]]

(In reply to Andreea Pavel [:apavel] from comment #8)

James recent occurrences seems to be from bug 1505739, can you take a look?

I'm not James, but could you clarify what you mean? That bug's patch landed 8 months ago, which is not really "recent". :)

(It is possible that that bug may be where this started, since it's probably where these tests became enabled. But I'm curious if there's something else I'm misunderstanding RE recent failures and/or recent landings.)

Comment 10

[James Graham [:jgraham]]

AFAICT there is a possible, apparently low-frequency, intermittent failure mode whereby the Ahem font isn't loading at all on macOS for some reason. There has been work to move away from using the system font and instead prefer to use webfonts, so I think this issue will go away on its own at some point in the near future.

Comment 11

[Andreea Pavel [:apavel]]

(In reply to Daniel Holbert [:dholbert] from comment #9)

(In reply to Andreea Pavel [:apavel] from comment #8)

James recent occurrences seems to be from bug 1505739, can you take a look?

I'm not James, but could you clarify what you mean? That bug's patch landed 8 months ago, which is not really "recent". :)

(It is possible that that bug may be where this started, since it's probably where these tests became enabled. But I'm curious if there's something else I'm misunderstanding RE recent failures and/or recent landings.)

i just looked for the last changes on that file in mozilla-files: https://hg.mozilla.org/mozilla-central/log?rev=css%2FCSS2%2Fcolors%2Fcolor-001.xht

Comment 12

[Intermittent Failures Robot]

25 failures in 4710 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 2
• autoland: 18
• mozilla-central: 1
• mozilla-inbound: 3
• try: 1

Platform breakdown:

• macosx1014-64: 4
• macosx1014-64-shippable: 13
• android-em-7-0-x86_64: 8

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1509242&startday=2019-07-08&endday=2019-07-14&tree=all

Comment 13

[Intermittent Failures Robot]

1 failures in 4298 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• macosx1014-64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1509242&startday=2019-07-15&endday=2019-07-21&tree=all

Comment 14

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup