Open Bug 1557438 Opened 6 years ago Updated 3 years ago

OSX Mojave (10.14) - testing/web-platform/tests/css/CSS2/colors | permafail

Categories

(Core :: CSS Parsing and Computation, defect, P5)

Product:
Core
Component:
CSS Parsing and Computation
Type:
defect

Tracking

()

People

(Reporter: intermittent-bug-filer, Unassigned)

References

Details

(Keywords: regression)

Filed by: egao [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=250418039&repo=try
Full log: https://queue.taskcluster.net/v1/task/WmnUHDcQREy0WK9MI2a92Q/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/WmnUHDcQREy0WK9MI2a92Q/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

Test: web-platform-tests-reftest
Chunk: 3
Platform: macosx1014-64-shippable/opt

Consistent failures of many test files under the directory testing/web-platform/tests/css/CSS2/colors observed on macosx1014-64-shippable/opt platform.

Context (one example):
17:04:40 INFO - TEST-UNEXPECTED-FAIL | /css/CSS2/colors/color-001.xht | Testing http://web-platform.test:8000/css/CSS2/colors/color-001.xht == http://web-platform.test:8000/css/CSS2/colors/color-001-ref.xht

Context (another example):
17:04:48 INFO - PID 1589 | 1559840688294 Marionette INFO Found 8685 pixels different, maximum difference per channel 254
17:04:48 INFO - TEST-UNEXPECTED-FAIL | /css/CSS2/colors/color-003.xht | Testing http://web-platform.test:8000/css/CSS2/colors/color-003.xht == http://web-platform.test:8000/css/CSS2/colors/color-003-ref.xht

:svoisen - this failure cropped up recently as I attempted to move web-platform-tests-reftest from macosx1010 to macosx1014. The failures appear to affect all tests in this directory. Could you pass the ni to someone who may be able to investigate?

Flags: needinfo?(svoisen)

:egao – This looks like the Ahem font is not installed on the machines running 10.14. The tests that show an "X" should actually be a unit square.
:jfkthame – Can you confirm diagnosis and help here?

Flags: needinfo?(svoisen) → needinfo?(jfkthame)

Yes, it certainly looks like the test machines are lacking the Ahem font expected by those tests. Something like this came up previously in bug 1425698, where :jgraham and :dhouse discussed and tested a bunch of stuff; maybe they can look into what's failing here?

Flags: needinfo?(jfkthame) → needinfo?(james)

So, fun story. We install the Ahem font to the user font directory on macOS. However something about how our machines are configured means that this wasn't working reliably. When we were investigating, it just started working and we never really figured out what changed. I wonder if the directory had to already exist when the user initally logged in, or something, but it wasn't clear. Nevertheless things kept working and we stopped digging into what had changed.

So the good news is that this kind of failure has precedent and may be fixable. The bad news is that we don't know exactly what steps are required to fix it.

However there's a further twist in the tale. It turns out that Google can't load Ahem in their CI for <reasons> so they've started work to move to loading Ahem as a webfont in all cases. So in some wpt update in the near future this problem should go away entirely.

There is an open question about whether we will need to add different system fonts to wpt to test things that work differently between system fonts and webfonts, but at this point no one seems quite sure what the requirements are there.

Flags: needinfo?(james)

(In reply to James Graham [:jgraham] from comment #5)

However there's a further twist in the tale. It turns out that Google can't load Ahem in their CI for <reasons> so they've started work to move to loading Ahem as a webfont in all cases. So in some wpt update in the near future this problem should go away entirely.

Hold on, where's this discussion happening? In the past people were opposed to this because @font-face loads async and thus can cover up layout issues.

Flags: needinfo?(james)
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.