Closed Bug 1551611 Opened 5 months ago Closed 5 months ago

OSX Mojave (10.14) - 2 osx wrench reftests fails when run on 10.14

Categories

(Core :: Graphics: WebRender, defect)

defect
Not set

Tracking

()

RESOLVED FIXED
mozilla68
Tracking Status
firefox68 --- fixed

People

(Reporter: jmaher, Assigned: kats)

References

(Blocks 1 open bug)

Details

Attachments

(3 files)

I am trying to get all tests running on osx 10.14 in CI- this throws a wrench in some plans when tests fail.

here is a try push with a failure:
https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=246347549&revision=01feb26a1a4726cba1f54b1f7421e694e3e87ab5

commenting out both of these tests result in a green try push:
REFTEST TEST-UNEXPECTED-FAIL | reftests/text/white-opacity.yaml == reftests/text/white-opacity.png | image comparison, max difference: 9, number of differing pixels: 3095
REFTEST TEST-UNEXPECTED-FAIL | reftests/text/allow-subpixel.yaml != reftests/text/allow-subpixel-ref.yaml | image comparison

It seems like a lot of pixels on white-opacity.yaml to accept- and I am not sure what to do about the subpixel failure.

:kats, could you take a look at this or help redirect me to someone who can help figure out a fix or acceptable manifest hack

Flags: needinfo?(kats)

I'm not sure offhand why the test results would be different on macOS 10.14. Redirecting to :kvark and /cc some people who might have more useful information.

Flags: needinfo?(kats) → needinfo?(dmalyshau)

I wonder if this wonderful supports_subpixel_aa function started return false on 10.14 out of a sudden? The comment appears relevant:

// According to the Skia source code, there's no public API to
// determine if subpixel AA is supported. So jrmuizel ported
// this function from Skia which is used to check if a glyph
// can be rendered with subpixel AA.

We can find out on CI if we set RUST_LOG=webrender=debug envionment variable to see the following output:

debug!("Test for subpixel AA support: {}", supports_subpixel_aa());

If it's a legitimate "false", we'd need some notion of "run this test only if subpixel AA is supported" semantics.

Flags: needinfo?(dmalyshau)

In the General section of the system settings there's a checkbox at the bottom that says "Use Font Smoothing when available". Is this checkbox checked on the 10.14 machines? My guess is that it is not checked, and checking it might make these tests pass.

Flags: needinfo?(jmaher)

:dividehex- can you help get an answer to comment 4 ^ ?

Flags: needinfo?(jmaher) → needinfo?(jwatkins)

I can repro these two reftest failures locally on my 10.14 machine, and the "Use Font Smoothing when available" checkbox is checked. Unchecking it makes no difference.

Ah.

Flags: needinfo?(jwatkins)

(In reply to Dzmitry Malyshau [:kvark] from comment #3)

We can find out on CI if we set RUST_LOG=webrender=debug envionment variable to see the following output:

debug!("Test for subpixel AA support: {}", supports_subpixel_aa());

This returns false for me on 10.14. I don't know what it returns on older macOS but presumably we can do a try push to find out.

As of macOS 10.14, requesting font smoothing now causes a dilation, but not actual subpixel AA, due to Apple's decision to discontinue it. So probably some tests will need their PNG refs regenerated or need to be disabled for macOS 10.14.

Note that these tests run on older macOS (and non-macOS) as well as part of the github automation. So it seems better to disable the tests on macOS 10.14 rather than regenerate the PNGs.

I can write the patch unless somebody else really wants to.

Assignee: nobody → kats

based on a few retriggers:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=7daf8948e0d0a30a984e6cb09844ba0082922661

I see a need to look at:
REFTEST TEST-UNEXPECTED-FAIL | reftests/text/split-batch.yaml == reftests/text/split-batch-ref.yaml | image comparison, max difference: 1, number of differing pixels: 14

This moves the existing constants into a ReftestEnvironment which
encapsulates it a bit better. Also this fixes the incorrect "debug" cfg
check to "debug_assertions" which is more correct.

(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #12)

based on a few retriggers:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=7daf8948e0d0a30a984e6cb09844ba0082922661

I see a need to look at:
REFTEST TEST-UNEXPECTED-FAIL | reftests/text/split-batch.yaml == reftests/text/split-batch-ref.yaml | image comparison, max difference: 1, number of differing pixels: 14

Let's split this into another bug. This looks like nondeterminism that we might want to track down.

Pushed by kgupta@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/df4a76b3e6ce
Introduce a ReftestEnvironment. r=Gankro
https://hg.mozilla.org/integration/autoland/rev/85ac4e68a63e
Add macOS version information to the ReftestEnvironment. r=Gankro
https://hg.mozilla.org/integration/autoland/rev/f60b2e760d1a
Skip subpixel AA tests on macOS 10.14. r=Gankro

Backed out 3 changesets (Bug 1551611) for tidy bustage

Push with failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&fromchange=f60b2e760d1adf5be0aae2485ab685dc8b2fb039&tochange=bfddd86062409ba22bf888f09d478041cf4391e7&selectedJob=246608087

Backout link: https://hg.mozilla.org/integration/autoland/rev/bfddd86062409ba22bf888f09d478041cf4391e7

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=246608087&repo=autoland&lineNumber=993

[task 2019-05-15T14:54:00.066Z] Progress: 87% (806/925)
[task 2019-05-15T14:54:00.085Z] Progress: 87% (807/925)
[task 2019-05-15T14:54:00.105Z] Progress: 87% (808/925)
[task 2019-05-15T14:54:00.105Z] Progress: 87% (809/925)
[task 2019-05-15T14:54:00.187Z] Progress: 87% (810/925)
[task 2019-05-15T14:54:00.194Z] Progress: 87% (811/925)
[task 2019-05-15T14:54:00.196Z] Progress: 87% (812/925)
[task 2019-05-15T14:54:00.196Z] ./wrench/src/reftest.rs:412: Line is longer than 120 characters
[task 2019-05-15T14:54:00.255Z]
[task 2019-05-15T14:54:00.286Z] Progress: 87% (813/925)
[task 2019-05-15T14:54:00.298Z] Progress: 88% (814/925)
[task 2019-05-15T14:54:00.362Z] Progress: 88% (815/925)
[task 2019-05-15T14:54:00.363Z] Progress: 88% (816/925)

Flags: needinfo?(kats)
Flags: needinfo?(kats)
Pushed by kgupta@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/74151c255b07
Introduce a ReftestEnvironment. r=Gankro
https://hg.mozilla.org/integration/autoland/rev/a7f0ac2141b1
Add macOS version information to the ReftestEnvironment. r=Gankro
https://hg.mozilla.org/integration/autoland/rev/c2102cbe5593
Skip subpixel AA tests on macOS 10.14. r=Gankro
Status: NEW → RESOLVED
Closed: 5 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla68
You need to log in before you can comment on or make changes to this bug.