1551611 - OSX Mojave (10.14) - 2 osx wrench reftests fails when run on 10.14

Status: RESOLVED FIXED

(Core :: Graphics: WebRender, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

I am trying to get all tests running on osx 10.14 in CI- this throws a wrench in some plans when tests fail.

here is a try push with a failure:
https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=246347549&revision=01feb26a1a4726cba1f54b1f7421e694e3e87ab5

commenting out both of these tests result in a green try push:
REFTEST TEST-UNEXPECTED-FAIL | reftests/text/white-opacity.yaml == reftests/text/white-opacity.png | image comparison, max difference: 9, number of differing pixels: 3095
REFTEST TEST-UNEXPECTED-FAIL | reftests/text/allow-subpixel.yaml != reftests/text/allow-subpixel-ref.yaml | image comparison

It seems like a lot of pixels on white-opacity.yaml to accept- and I am not sure what to do about the subpixel failure.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

:kats, could you take a look at this or help redirect me to someone who can help figure out a fix or acceptable manifest hack

Comment 2

[Kartikaya Gupta (email:[e-mail])]

I'm not sure offhand why the test results would be different on macOS 10.14. Redirecting to :kvark and /cc some people who might have more useful information.

Comment 3

[Dzmitry Malyshau [:kvark]]

I wonder if this wonderful supports_subpixel_aa function started return false on 10.14 out of a sudden? The comment appears relevant:

// According to the Skia source code, there's no public API to
// determine if subpixel AA is supported. So jrmuizel ported
// this function from Skia which is used to check if a glyph
// can be rendered with subpixel AA.

We can find out on CI if we set RUST_LOG=webrender=debug envionment variable to see the following output:

debug!("Test for subpixel AA support: {}", supports_subpixel_aa());

If it's a legitimate "false", we'd need some notion of "run this test only if subpixel AA is supported" semantics.

Comment 4

[Markus Stange [:mstange]]

In the General section of the system settings there's a checkbox at the bottom that says "Use Font Smoothing when available". Is this checkbox checked on the 10.14 machines? My guess is that it is not checked, and checking it might make these tests pass.

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

:dividehex- can you help get an answer to comment 4 ^ ?

Comment 6

[Kartikaya Gupta (email:[e-mail])]

I can repro these two reftest failures locally on my 10.14 machine, and the "Use Font Smoothing when available" checkbox is checked. Unchecking it makes no difference.

Comment 7

[Markus Stange [:mstange]]

Ah.

Comment 8

[Kartikaya Gupta (email:[e-mail])]

(In reply to Dzmitry Malyshau [:kvark] from comment #3)

We can find out on CI if we set RUST_LOG=webrender=debug envionment variable to see the following output:

debug!("Test for subpixel AA support: {}", supports_subpixel_aa());

This returns false for me on 10.14. I don't know what it returns on older macOS but presumably we can do a try push to find out.

Comment 9

[Lee Salzman [:lsalzman]]

As of macOS 10.14, requesting font smoothing now causes a dilation, but not actual subpixel AA, due to Apple's decision to discontinue it. So probably some tests will need their PNG refs regenerated or need to be disabled for macOS 10.14.

Comment 10

[Kartikaya Gupta (email:[e-mail])]

Note that these tests run on older macOS (and non-macOS) as well as part of the github automation. So it seems better to disable the tests on macOS 10.14 rather than regenerate the PNGs.

Comment 11

[Kartikaya Gupta (email:[e-mail])]

I can write the patch unless somebody else really wants to.

Comment 12

[Joel Maher ( :jmaher ) (UTC -8)]

based on a few retriggers:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=7daf8948e0d0a30a984e6cb09844ba0082922661

I see a need to look at:
REFTEST TEST-UNEXPECTED-FAIL | reftests/text/split-batch.yaml == reftests/text/split-batch-ref.yaml | image comparison, max difference: 1, number of differing pixels: 14

Comment 13

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1551611 - Introduce a ReftestEnvironment. r?kvark

This moves the existing constants into a ReftestEnvironment which
encapsulates it a bit better. Also this fixes the incorrect "debug" cfg
check to "debug_assertions" which is more correct.

Comment 14

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1551611 - Add macOS version information to the ReftestEnvironment. r?kvark

Depends on D31181

Comment 15

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1551611 - Skip subpixel AA tests on macOS 10.14. r?kvark

Depends on D31182

Comment 16

[Kartikaya Gupta (email:[e-mail])]

(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #12)

based on a few retriggers:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=7daf8948e0d0a30a984e6cb09844ba0082922661

I see a need to look at:
REFTEST TEST-UNEXPECTED-FAIL | reftests/text/split-batch.yaml == reftests/text/split-batch-ref.yaml | image comparison, max difference: 1, number of differing pixels: 14

Let's split this into another bug. This looks like nondeterminism that we might want to track down.

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/df4a76b3e6ce
Introduce a ReftestEnvironment. r=Gankro
https://hg.mozilla.org/integration/autoland/rev/85ac4e68a63e
Add macOS version information to the ReftestEnvironment. r=Gankro
https://hg.mozilla.org/integration/autoland/rev/f60b2e760d1a
Skip subpixel AA tests on macOS 10.14. r=Gankro

Comment 18

[Bogdan Tara[:bogdan_tara | bogdant]]

Backed out 3 changesets (Bug 1551611) for tidy bustage

Push with failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&fromchange=f60b2e760d1adf5be0aae2485ab685dc8b2fb039&tochange=bfddd86062409ba22bf888f09d478041cf4391e7&selectedJob=246608087

Backout link: https://hg.mozilla.org/integration/autoland/rev/bfddd86062409ba22bf888f09d478041cf4391e7

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=246608087&repo=autoland&lineNumber=993

[task 2019-05-15T14:54:00.066Z] Progress: 87% (806/925)
[task 2019-05-15T14:54:00.085Z] Progress: 87% (807/925)
[task 2019-05-15T14:54:00.105Z] Progress: 87% (808/925)
[task 2019-05-15T14:54:00.105Z] Progress: 87% (809/925)
[task 2019-05-15T14:54:00.187Z] Progress: 87% (810/925)
[task 2019-05-15T14:54:00.194Z] Progress: 87% (811/925)
[task 2019-05-15T14:54:00.196Z] Progress: 87% (812/925)
[task 2019-05-15T14:54:00.196Z] ./wrench/src/reftest.rs:412: Line is longer than 120 characters
[task 2019-05-15T14:54:00.255Z]
[task 2019-05-15T14:54:00.286Z] Progress: 87% (813/925)
[task 2019-05-15T14:54:00.298Z] Progress: 88% (814/925)
[task 2019-05-15T14:54:00.362Z] Progress: 88% (815/925)
[task 2019-05-15T14:54:00.363Z] Progress: 88% (816/925)

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/74151c255b07
Introduce a ReftestEnvironment. r=Gankro
https://hg.mozilla.org/integration/autoland/rev/a7f0ac2141b1
Add macOS version information to the ReftestEnvironment. r=Gankro
https://hg.mozilla.org/integration/autoland/rev/c2102cbe5593
Skip subpixel AA tests on macOS 10.14. r=Gankro

Comment 20

[Daniel Varga [:dvarga]]

https://hg.mozilla.org/mozilla-central/rev/74151c255b07
https://hg.mozilla.org/mozilla-central/rev/a7f0ac2141b1
https://hg.mozilla.org/mozilla-central/rev/c2102cbe5593