1451123 - Perma-orange tier2 TVw /css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html | Testing http

Status: RESOLVED FIXED

(Core :: CSS Parsing and Computation, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: apavel [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=171710486&repo=mozilla-inbound

https://queue.taskcluster.net/v1/task/S4u57933REiFkOMexVSeMw/runs/0/artifacts/public/logs/live_backing.log

13:17:29     INFO - PID 1714 | Unable to read VR Path Registry from /Users/cltbld/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath
13:17:30     INFO - PID 1714 | Unable to read VR Path Registry from /Users/cltbld/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath
13:17:30     INFO - PID 1714 | 2018-04-03 13:17:30.911 plugin-container[1719:7793] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x9543, name = 'com.apple.tsm.portname'
13:17:30     INFO - PID 1714 | See /usr/include/servers/bootstrap_defs.h for the error codes.
13:17:31     INFO - PID 1714 | 1522786651177	Marionette	INFO	Listening on port 2828
13:17:31     INFO - TEST-START | /css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html
13:17:31     INFO - PID 1714 | Unable to read VR Path Registry from /Users/cltbld/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath
13:17:31     INFO - PID 1714 | 1522786651932	Marionette	INFO	Testing http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html == http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/reference/shape-outside-linear-gradient-001-ref.html
13:17:31     INFO - PID 1714 | 2018-04-03 13:17:31.973 plugin-container[1720:7886] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x9543, name = 'com.apple.tsm.portname'
13:17:31     INFO - PID 1714 | See /usr/include/servers/bootstrap_defs.h for the error codes.
13:17:32     INFO - PID 1714 | 1522786652094	Marionette	INFO	Found 20859 pixels different, maximum difference per channel 255
13:17:32     INFO - TEST-UNEXPECTED-FAIL | /css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html | Testing http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html == http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/reference/shape-outside-linear-gradient-001-ref.html

13:17:32     INFO - TEST-INFO took 647ms
13:17:32     INFO - PID 1714 | *** UTM:SVC TimerManager:registerTimer called after profile-before-change notification. Ignoring timer registration for id: telemetry_modules_ping
13:17:32     INFO - PID 1714 | 
13:17:32     INFO - PID 1714 | ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
13:17:32     INFO - PID 1714 | 
13:17:32  WARNING - u'runner_teardown': ()
13:17:32     INFO - certutil | 
13:17:32     INFO - certutil | 
13:17:32     INFO - certutil | 
13:17:32     INFO - Certificate Nickname                                         Trust Attributes
13:17:32     INFO -                                                              SSL,S/MIME,JAR/XPI
13:17:32     INFO - 
13:17:32     INFO - web-platform-tests                                           CT,, 
13:17:32     INFO - 
13:17:33     INFO - PID 1724 | Unable to read VR Path Registry from /Users/cltbld/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath
13:17:33     INFO - PID 1724 | Unable to read VR Path Registry from /Users/cltbld/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath
13:17:34     INFO - PID 1724 | 2018-04-03 13:17:34.280 plugin-container[1726:7997] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x973b, name = 'com.apple.tsm.portname'
13:17:34     INFO - PID 1724 | See /usr/include/servers/bootstrap_defs.h for the error codes.
13:17:34     INFO - PID 1724 | 1522786654586	Marionette	INFO	Listening on port 2828
13:17:34     INFO - TEST-START | /css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html
13:17:35     INFO - PID 1724 | Unable to read VR Path Registry from /Users/cltbld/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath
13:17:35     INFO - PID 1724 | 1522786655287	Marionette	INFO	Testing http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html == http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/reference/shape-outside-linear-gradient-001-ref.html
13:17:35     INFO - PID 1724 | 2018-04-03 13:17:35.327 plugin-container[1727:8078] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x974b, name = 'com.apple.tsm.portname'
13:17:35     INFO - PID 1724 | See /usr/include/servers/bootstrap_defs.h for the error codes.
13:17:35     INFO - PID 1724 | 1522786655440	Marionette	INFO	Found 20859 pixels different, maximum difference per channel 255
13:17:35     INFO - TEST-UNEXPECTED-FAIL | /css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html | Testing http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html == http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/reference/shape-outside-linear-gradient-001-ref.html

Comment 1

[Daniel Holbert [:dholbert]]

Looks like this is a real failure, caused by bug 1404222.  (The final cset there removed some failure annotations for some of these tests, but it looks like it still fails on this platform at least.)

Comment 2

[Daniel Holbert [:dholbert]]

(ni=bradwerth to figure out how best to sort this out)

Comment 3

[Brad Werth [:bradwerth]]

Ahem is not loading for the new wpt reftests added in Bug 1404222. Both the test page and the reference page rely upon Ahem, and the failure log https://treeherder.mozilla.org/logviewer.html#?job_id=171710486&repo=mozilla-inbound&lineNumber=815 shows that the font is not rendering in either image.

There's evidence that this has appeared before on this platform. https://searchfox.org/mozilla-central/source/testing/web-platform/meta/infrastructure/assumptions/ahem.html.ini shows that a recently-added wpt reftest that uses Ahem had to be marked expected fail on this platform.

It's unclear why this is affecting new wpt reftests and not existing tests. Since Ahem.tff is available in the wpt subtree, my proposed approach is to add an explicit font-face declaration to the new tests; something like:

@font-face { font-family: "Ahem"; src: url(/fonts/Ahem.ttf);

Comment 4

[Daniel Holbert [:dholbert]]

jgraham, see comment 3 -- do you know what might be going on here RE Ahem on this platform?

(See the ahem.html.ini file that Brad linked, which seems to have been added in the "Update wpt metadata" commit in bug 1410245.)

Comment 5

[Daniel Holbert [:dholbert]]

[needinfo for comment 4]

Comment 6

[Brad Werth [:bradwerth]]

Or it may be as simple as adding this declaration to the tests, since they appear in older, working tests:

<meta name="flags" content="ahem"/>

Comment 7

[Brad Werth [:bradwerth]]

Attachment: Bug 1451123 Part 2: Add @font-face to a failing wpt infrastructure reftest.

Review commit: https://reviewboard.mozilla.org/r/233444/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/233444/

Comment 8

[Brad Werth [:bradwerth]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=73ba7cb8342a8e325663db3ef00b5e616f4ae286

Comment 9

[Brad Werth [:bradwerth]]

Comment on attachment 8964718 [details]
Bug 1451123 Part 2: Add @font-face to a failing wpt infrastructure reftest.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/233444/diff/1-2/

Comment 10

[Brad Werth [:bradwerth]]

Attachment: Bug 1451123: Disable some shape-outside image web-platform tests on macOS, since they fail without Ahem font support.

Review commit: https://reviewboard.mozilla.org/r/233446/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/233446/

Comment 11

[James Graham [:jgraham]]

So, Ahem on OSX is a problem.

In theory wpt tests have system Ahem (I think this is not true for Mozilla reftests). And wptrunner has a --install-fonts option that's applied in prod. On OSX this copies Ahem.ttf to ~/Library/Fonts/ when the runner starts. On a local machine, this seems to work fine, but on our CI system Ahem still doesn't load. I have no idea what's going on, even though I got a loaner to investigate. A solution is to add Ahem directly to the puppet configs, but when I looked at that it was hard to figure out where it would go.

Comment 12

[Daniel Holbert [:dholbert]]

Maybe we should just skip these tests on OSX for the time being (via annotations in .ini files), then?

I'm reluctant to add paper-over hacks to the test files themselves (particularly in the files written by other vendors, e.g. shape-outside-linear-gradient-001-ref.html), as the current Part 1 is kind of doing...

Comment 13

[Daniel Holbert [:dholbert]]

(jgraham, would you mind filing a bug to capture comment 11, if there isn't already one? Then we can reference that bug from .ini file annotations, perhaps, for tests like these & future ones that depend strongly on Ahem.)

Comment 14

[Brad Werth [:bradwerth]]

Comment on attachment 8964721 [details]
Bug 1451123: Disable some shape-outside image web-platform tests on macOS, since they fail without Ahem font support.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/233446/diff/1-2/

Comment 15

[Brad Werth [:bradwerth]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=535460ea358f2f854ebcd8be97a68e25d7fe9460

Comment 16

[Brad Werth [:bradwerth]]

(In reply to Daniel Holbert [:dholbert] (away 4/24 - 5/11) from comment #13)
> (jgraham, would you mind filing a bug to capture comment 11, if there isn't
> already one? Then we can reference that bug from .ini file annotations,
> perhaps, for tests like these & future ones that depend strongly on Ahem.)

Ah, I prematurely requested review for my fix, when the .ini files in there still contain a comment referencing THIS bug, which doesn't make any sense. I'll update those comments before landing, once we have a proper bug to reference.

Comment 17

[Daniel Holbert [:dholbert]]

Comment on attachment 8964721 [details]
Bug 1451123: Disable some shape-outside image web-platform tests on macOS, since they fail without Ahem font support.

https://reviewboard.mozilla.org/r/233446/#review239498

Comment 18

[Daniel Holbert [:dholbert]]

(In reply to Brad Werth [:bradwerth] from comment #16)
> I'll update those comments before landing, once we have a proper bug to
> reference.

No need; this bug is fine to reference for now, I think.  Better to fix the orange sooner, IMO; and curious people who stumble on the .ini file will still have a route to discovering whatever bug we end up spinning off of this one. (by reading comments here)

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4b833df6ec89
Disable some shape-outside image web-platform tests on macOS, since they fail without Ahem font support. r=dholbert

Comment 20

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/4b833df6ec89

Comment 21

[Brad Werth [:bradwerth]]

Looks like all is well until we solve the root cause noted in comment 11.

Comment 22

[James Graham [:jgraham]]

Bug 1425698 is the Ahem-on-OSX issue.

It's not 100% clear to me why you disabled the tests rather than marking them expected fail. The former makes it much less likely they we will rememeber to re-enable them.

Comment 23

[Emilio Cobos Álvarez (:emilio)]

Yes, please, let's mark it as failing instead?

Comment 24

[Daniel Holbert [:dholbert]]

(In reply to James Graham [:jgraham] from comment #22)
> Bug 1425698 is the Ahem-on-OSX issue.

Thanks!

> It's not 100% clear to me why you disabled the tests rather than marking
> them expected fail.

That was my suggestion (comment 12) -- I was thinking that these were only failing on some OSX versions, since I only saw one row of test failures in the link from comment 0.

If this is a known (and 100% reliable) issue on all OSX versions that we test in CI [and it sounds like it is?], then agreed, we should do expected-fail instead.

Comment 25

[Daniel Holbert [:dholbert]]

(I'd thought we still ran tests on multiple versions of OSX, but I guess we don't anymore -- only 10.10 now -- based on paging through treeherder backlog.)

Comment 26

[Brad Werth [:bradwerth]]

(In reply to Emilio Cobos Álvarez [:emilio] from comment #23)
> Yes, please, let's mark it as failing instead?

Bug 1452184 has been opened to address this.

Comment 27

[Intermittent Failures Robot]

12 failures in 765 pushes (0.016 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 7
* mozilla-central: 3
* autoland: 2

Platform breakdown:
* osx-10-10: 11
* macosx64-nightly: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1451123&startday=2018-04-02&endday=2018-04-08&tree=trunk