Closed Bug 1451123 Opened 6 years ago Closed 6 years ago

Perma-orange tier2 TVw /css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html | Testing http

Categories

(Core :: CSS Parsing and Computation, defect, P5)

Product:
Core
Component:
CSS Parsing and Computation
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla61
Tracking Flags:
Tracking Status
firefox61 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: bradwerth)

References

Details

(Keywords: intermittent-failure)

Attachments

(1 file, 1 obsolete file)

Bug 1451123: Disable some shape-outside image web-platform tests on macOS, since they fail without Ahem font support.
59 bytes, text/x-review-board-request
dholbert
: review+
Details 
Filed by: apavel [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=171710486&repo=mozilla-inbound

https://queue.taskcluster.net/v1/task/S4u57933REiFkOMexVSeMw/runs/0/artifacts/public/logs/live_backing.log

13:17:29     INFO - PID 1714 | Unable to read VR Path Registry from /Users/cltbld/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath
13:17:30     INFO - PID 1714 | Unable to read VR Path Registry from /Users/cltbld/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath
13:17:30     INFO - PID 1714 | 2018-04-03 13:17:30.911 plugin-container[1719:7793] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x9543, name = 'com.apple.tsm.portname'
13:17:30     INFO - PID 1714 | See /usr/include/servers/bootstrap_defs.h for the error codes.
13:17:31     INFO - PID 1714 | 1522786651177	Marionette	INFO	Listening on port 2828
13:17:31     INFO - TEST-START | /css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html
13:17:31     INFO - PID 1714 | Unable to read VR Path Registry from /Users/cltbld/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath
13:17:31     INFO - PID 1714 | 1522786651932	Marionette	INFO	Testing http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html == http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/reference/shape-outside-linear-gradient-001-ref.html
13:17:31     INFO - PID 1714 | 2018-04-03 13:17:31.973 plugin-container[1720:7886] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x9543, name = 'com.apple.tsm.portname'
13:17:31     INFO - PID 1714 | See /usr/include/servers/bootstrap_defs.h for the error codes.
13:17:32     INFO - PID 1714 | 1522786652094	Marionette	INFO	Found 20859 pixels different, maximum difference per channel 255
13:17:32     INFO - TEST-UNEXPECTED-FAIL | /css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html | Testing http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html == http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/reference/shape-outside-linear-gradient-001-ref.html

13:17:32     INFO - TEST-INFO took 647ms
13:17:32     INFO - PID 1714 | *** UTM:SVC TimerManager:registerTimer called after profile-before-change notification. Ignoring timer registration for id: telemetry_modules_ping
13:17:32     INFO - PID 1714 | 
13:17:32     INFO - PID 1714 | ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
13:17:32     INFO - PID 1714 | 
13:17:32  WARNING - u'runner_teardown': ()
13:17:32     INFO - certutil | 
13:17:32     INFO - certutil | 
13:17:32     INFO - certutil | 
13:17:32     INFO - Certificate Nickname                                         Trust Attributes
13:17:32     INFO -                                                              SSL,S/MIME,JAR/XPI
13:17:32     INFO - 
13:17:32     INFO - web-platform-tests                                           CT,, 
13:17:32     INFO - 
13:17:33     INFO - PID 1724 | Unable to read VR Path Registry from /Users/cltbld/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath
13:17:33     INFO - PID 1724 | Unable to read VR Path Registry from /Users/cltbld/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath
13:17:34     INFO - PID 1724 | 2018-04-03 13:17:34.280 plugin-container[1726:7997] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x973b, name = 'com.apple.tsm.portname'
13:17:34     INFO - PID 1724 | See /usr/include/servers/bootstrap_defs.h for the error codes.
13:17:34     INFO - PID 1724 | 1522786654586	Marionette	INFO	Listening on port 2828
13:17:34     INFO - TEST-START | /css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html
13:17:35     INFO - PID 1724 | Unable to read VR Path Registry from /Users/cltbld/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath
13:17:35     INFO - PID 1724 | 1522786655287	Marionette	INFO	Testing http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html == http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/reference/shape-outside-linear-gradient-001-ref.html
13:17:35     INFO - PID 1724 | 2018-04-03 13:17:35.327 plugin-container[1727:8078] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x974b, name = 'com.apple.tsm.portname'
13:17:35     INFO - PID 1724 | See /usr/include/servers/bootstrap_defs.h for the error codes.
13:17:35     INFO - PID 1724 | 1522786655440	Marionette	INFO	Found 20859 pixels different, maximum difference per channel 255
13:17:35     INFO - TEST-UNEXPECTED-FAIL | /css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html | Testing http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html == http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/reference/shape-outside-linear-gradient-001-ref.html
Summary: Intermittent /css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html | Testing http → Perma-orange tier2 TVw /css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html | Testing http
Looks like this is a real failure, caused by bug 1404222.  (The final cset there removed some failure annotations for some of these tests, but it looks like it still fails on this platform at least.)
Blocks: 1404222
(ni=bradwerth to figure out how best to sort this out)
Flags: needinfo?(bwerth)
Ahem is not loading for the new wpt reftests added in Bug 1404222. Both the test page and the reference page rely upon Ahem, and the failure log https://treeherder.mozilla.org/logviewer.html#?job_id=171710486&repo=mozilla-inbound&lineNumber=815 shows that the font is not rendering in either image.

There's evidence that this has appeared before on this platform. https://searchfox.org/mozilla-central/source/testing/web-platform/meta/infrastructure/assumptions/ahem.html.ini shows that a recently-added wpt reftest that uses Ahem had to be marked expected fail on this platform.

It's unclear why this is affecting new wpt reftests and not existing tests. Since Ahem.tff is available in the wpt subtree, my proposed approach is to add an explicit font-face declaration to the new tests; something like:

@font-face { font-family: "Ahem"; src: url(/fonts/Ahem.ttf);
Assignee: nobody → bwerth
jgraham, see comment 3 -- do you know what might be going on here RE Ahem on this platform?

(See the ahem.html.ini file that Brad linked, which seems to have been added in the "Update wpt metadata" commit in bug 1410245.)
[needinfo for comment 4]
Flags: needinfo?(james)
Or it may be as simple as adding this declaration to the tests, since they appear in older, working tests:

<meta name="flags" content="ahem"/>
So, Ahem on OSX is a problem.

In theory wpt tests have system Ahem (I think this is not true for Mozilla reftests). And wptrunner has a --install-fonts option that's applied in prod. On OSX this copies Ahem.ttf to ~/Library/Fonts/ when the runner starts. On a local machine, this seems to work fine, but on our CI system Ahem still doesn't load. I have no idea what's going on, even though I got a loaner to investigate. A solution is to add Ahem directly to the puppet configs, but when I looked at that it was hard to figure out where it would go.
Flags: needinfo?(james)
Maybe we should just skip these tests on OSX for the time being (via annotations in .ini files), then?

I'm reluctant to add paper-over hacks to the test files themselves (particularly in the files written by other vendors, e.g. shape-outside-linear-gradient-001-ref.html), as the current Part 1 is kind of doing...
(jgraham, would you mind filing a bug to capture comment 11, if there isn't already one? Then we can reference that bug from .ini file annotations, perhaps, for tests like these & future ones that depend strongly on Ahem.)
Flags: needinfo?(james)
Attachment #8964718 - Attachment is obsolete: true
Attachment #8964721 - Flags: review?(dholbert)
(In reply to Daniel Holbert [:dholbert] (away 4/24 - 5/11) from comment #13)
> (jgraham, would you mind filing a bug to capture comment 11, if there isn't
> already one? Then we can reference that bug from .ini file annotations,
> perhaps, for tests like these & future ones that depend strongly on Ahem.)

Ah, I prematurely requested review for my fix, when the .ini files in there still contain a comment referencing THIS bug, which doesn't make any sense. I'll update those comments before landing, once we have a proper bug to reference.
Comment on attachment 8964721 [details]
Bug 1451123: Disable some shape-outside image web-platform tests on macOS, since they fail without Ahem font support.

https://reviewboard.mozilla.org/r/233446/#review239498
Attachment #8964721 - Flags: review?(dholbert) → review+
(In reply to Brad Werth [:bradwerth] from comment #16)
> I'll update those comments before landing, once we have a proper bug to
> reference.

No need; this bug is fine to reference for now, I think.  Better to fix the orange sooner, IMO; and curious people who stumble on the .ini file will still have a route to discovering whatever bug we end up spinning off of this one. (by reading comments here)
Pushed by bwerth@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/4b833df6ec89
Disable some shape-outside image web-platform tests on macOS, since they fail without Ahem font support. r=dholbert
https://hg.mozilla.org/mozilla-central/rev/4b833df6ec89
Status: NEW → RESOLVED
Closed: 6 years ago
status-firefox61: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla61
Looks like all is well until we solve the root cause noted in comment 11.
Flags: needinfo?(bwerth)
Bug 1425698 is the Ahem-on-OSX issue.

It's not 100% clear to me why you disabled the tests rather than marking them expected fail. The former makes it much less likely they we will rememeber to re-enable them.
Flags: needinfo?(james)
Yes, please, let's mark it as failing instead?
Flags: needinfo?(bwerth)
(In reply to James Graham [:jgraham] from comment #22)
> Bug 1425698 is the Ahem-on-OSX issue.

Thanks!

> It's not 100% clear to me why you disabled the tests rather than marking
> them expected fail.

That was my suggestion (comment 12) -- I was thinking that these were only failing on some OSX versions, since I only saw one row of test failures in the link from comment 0.

If this is a known (and 100% reliable) issue on all OSX versions that we test in CI [and it sounds like it is?], then agreed, we should do expected-fail instead.
(I'd thought we still ran tests on multiple versions of OSX, but I guess we don't anymore -- only 10.10 now -- based on paging through treeherder backlog.)
Blocks: 1452184
(In reply to Emilio Cobos Álvarez [:emilio] from comment #23)
> Yes, please, let's mark it as failing instead?

Bug 1452184 has been opened to address this.
Flags: needinfo?(bwerth)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: