1982909 - a few wpt reftests in mathml/presentation-markup/fractions fail on new android 14 emulator

Status: NEW

(Core :: MathML, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

while testing the new android 14 os version on emulators, many reftests fail as seen on try:
https://treeherder.mozilla.org/jobs?repo=try&tier=1%2C2%2C3&revision=fdea14904f15d8fe3a01e1277a1ad60054bf5e8e&test_paths=%2Fmathml%2Fpresentation-markup%2Ffractions&selectedTaskRun=FYjz1ooWRxCRzmUoypwt1g.0

the reftest-compare shows we have different fonts for the variables used in the mathml:
https://hg-edge.mozilla.org/mozilla-central/raw-file/default/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/FYjz1ooWRxCRzmUoypwt1g/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

here are the tests that have this problem:

• mathml/presentation-markup/fractions/frac-created-dynamically-4.html
• mathml/presentation-markup/fractions/frac-linethickness-005.html
• mathml/presentation-markup/fractions/frac-linethickness-006.html
• mathml/presentation-markup/fractions/frac-linethickness-007.html
• mathml/presentation-markup/fractions/frac-linethickness-008.html

this might end up being a layout/font thing instead of mathml, let me ask :jfkthame for his expert opinion.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

same bugzilla component, but different types of tests and failures:

• /mathml/presentation-markup/operators/embellished-op-4-2.html
• /mathml/presentation-markup/operators/embellished-op-4-3.html

you can see these on the wr6 job:
https://treeherder.mozilla.org/jobs?repo=try&tier=1%2C2%2C3&revision=fdea14904f15d8fe3a01e1277a1ad60054bf5e8e&test_paths=%2Fmathml%2Fpresentation-markup%2Foperators&selectedTaskRun=L0_V9XxrTFaXFgMVIH4xQQ.0

and in reftest-compare:
https://hg-edge.mozilla.org/mozilla-central/raw-file/default/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/L0_V9XxrTFaXFgMVIH4xQQ/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

there is a g on the image, but the ref has a black square instead.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

finally on the wr5 job (on the ^^ try push), you can see in reftest-compare:
https://hg-edge.mozilla.org/mozilla-central/raw-file/default/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Wru7UycxRAWqXYHpDOo9SQ/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

the image is regular and the reference is bold. we have this behavior also in bug 1982904.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

adding onto the mathml failures:

• _mozilla/mathml/mathspace_names/negative-namedspace.html
• _mozilla/mathml/mathspace_names/positive-namedspace.html

it appears we have a different font and the reference image is bolded.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

possibly the last mathml and font issue:

• mathml/scripts/scriptlevel-movablelimits-accent.html

and here:

• _mozilla/mathml/tables/mtable-align-negative-rownumber-2.html
• _mozilla/mathml/tables/mtable-align-negative-rownumber.html

Comment 5

[Jonathan Kew [:jfkthame]]

(In reply to Joel Maher ( :jmaher ) (UTC -8) from comment #2)

finally on the wr5 job (on the ^^ try push), you can see in reftest-compare:
https://hg-edge.mozilla.org/mozilla-central/raw-file/default/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Wru7UycxRAWqXYHpDOo9SQ/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

the image is regular and the reference is bold. we have this behavior also in bug 1982904.

I don't think it's actually "bold", it's just an entirely different font family. The test has a serif font (maybe Latin Modern? STIX?) for the superscripts, while the reference is a sans-serif face (maybe Roboto or Noto Sans?).

One thing that might be worth trying would be to set gfx.font_rendering.fallback.async to false for all these mathml tests (it's true by default, but that means we may transiently get a fallback font rendering when full font metadata hasn't yet been loaded).

(If that doesn't help, we should probably ping :fredw to see if he has ideas of what may be going on.)

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

ok, this worked for all of these, I will add these prefs in my Bug 1982454 patch.

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

as this seemed to work (testing on android 7 as well so I can land and run both in parallel until all is ready to switch), I wonder if other of the reftests in Bug 1982454 patch would benefit from this.

Comment 8

[Jonathan Kew [:jfkthame]]

(In reply to Joel Maher ( :jmaher ) (UTC -8) from comment #7)

I wonder if other of the reftests in Bug 1982454 patch would benefit from this.

Yes, probably. I looked at some of the failures on the try run there, and I suspect that quite a few of them may be the same issue. If they succeed with the pref disabled, that's much preferable to adding lots of fuzzy or expected-failure annotations.

Comment 9

[Joel Maher ( :jmaher ) (UTC -8)]

picked up a few more tests with the pref set! thanks for pointing me in the direction. I will leave this bug alone, currently I will just add the pref do a few __dir__.ini files, which will account for all the tests in this bug