1389187 - stylo: Stylo web-platform-test regressions compared to Gecko

Status: RESOLVED WORKSFORME

(Core :: CSS Parsing and Computation, enhancement, P5)

Description

[Chris Peterson [:cpeterson]]

James Graham reports the following web-platform-test differences between Stylo and Gecko. Since Stylo now passes some web-platform-test cases that Gecko fails, we should make sure we're not unintentionally shipping new CSS features in Stylo (bug 1337166).

https://pastebin.mozilla.org/9029321

The following test cases PASS with Gecko but FAIL or TIMEOUT with Stylo:

/css/css-fonts-3/alternates-order.html
/css/css-fonts-3/font-variant-alternates-03.html
/css/css-fonts-3/font-variant-alternates-04.html
/css/css-fonts-3/font-variant-alternates-05.html
/css/css-fonts-3/font-variant-alternates-06.html
/css/css-fonts-3/font-variant-alternates-07.html
/css/css-fonts-3/font-variant-alternates-08.html
/css/css-fonts-3/font-variant-alternates-09.html
/css/css-fonts-3/font-variant-alternates-10.html
/css/css-fonts-3/font-variant-alternates-11.html
/css/css-fonts-3/font-variant-alternates-12.html
/css/css-fonts-3/font-variant-alternates-13.html
/css/css-fonts-3/font-variant-alternates-14.html
/css/css-fonts-3/font-variant-alternates-15.html
/css/css-fonts-3/font-variant-alternates-16.html
/css/css-fonts-3/font-variant-alternates-17.html
/css/css-transforms-1/transform-rounding-001.html
/css/css-writing-modes-3/text-orientation-script-001.html
/css/CSS2/cascade/at-import-009.xht
/css/CSS2/cascade/at-import-010.xht
/css/CSS2/syntax/at-rule-008.xht
/css/CSS2/syntax/at-rule-009.xht

The following test cases PASS with Stylo but FAIL with Gecko:

/css/css-transforms-1/transform-inherit-001.html
/css/css-ui-3/outline-009.html
/css/css-values-3/calc-in-media-queries-001.html
/css/css-values-3/calc-in-media-queries-002.html
/css/CSS2/backgrounds/background-position-001.xht
/css/CSS2/backgrounds/background-position-002.xht
/css/CSS2/fonts/font-size-124.xht
/css/CSS2/selectors/first-letter-dynamic-001.xht
/css/CSS2/selectors/first-line-floats-004.xht
/css/CSS2/ui/outline-width-096.xht
/css/mediaqueries-3/mq-calc-001.html
/css/mediaqueries-3/mq-calc-002.html
/css/mediaqueries-3/mq-calc-003.html
/css/mediaqueries-3/mq-calc-004.html
/css/mediaqueries-3/mq-calc-005.html

Comment 1

[Xidorn Quan [:xidorn] UTC+10]

@font-feature-values rule support should fix the css-fonts-3 failures.

Comment 2

[Xidorn Quan [:xidorn] UTC+10]

I believe the transform-rounding-001.html failure has the same underlying issue as floating-point rounding issue.

Comment 3

[Xidorn Quan [:xidorn] UTC+10]

The 3 (out of 4) wpt related to @import can be fixed trivially:
/css/CSS2/cascade/at-import-010.xht
/css/CSS2/syntax/at-rule-008.xht
/css/CSS2/syntax/at-rule-009.xht

To fix /css/CSS2/cascade/at-import-009.xht, we need the same mechanism as bug 1388911.

Comment 4

[Xidorn Quan [:xidorn] UTC+10]

Oh, hmmm, it doesn't really.

So the test coverage actually isn't good enough :)

Comment 5

[Xidorn Quan [:xidorn] UTC+10]

Servo PR for the four @import-related tests: https://github.com/servo/servo/pull/18139

Comment 6

[Xidorn Quan [:xidorn] UTC+10]

Oh, I thought the @import-related tests are trivial to fix... actually they are not... Fixing them is much more involving...

I've probably met the hardest lifetime problem I've ever tried to deal :(

Comment 7

[Xidorn Quan [:xidorn] UTC+10]

(In reply to Xidorn Quan [:xidorn] UTC+10 from comment #2)
> I believe the transform-rounding-001.html failure has the same underlying
> issue as floating-point rounding issue.

No. I was misled by the name. There is nothing about the serialization in this test at all.

Comment 8

[Xidorn Quan [:xidorn] UTC+10]

For /css/css-writing-modes-3/text-orientation-script-001.html, w3c/web-platform-tests#6949 should make it much faster, and not timeout for stylo anymore, but we may still want to know why Stylo is so much slower than Gecko on this case. We can do that in bug 1392170.

Comment 9

[Xidorn Quan [:xidorn] UTC+10]

* font-variant-alternates-related tests are bug 1355721
* transform-rounding-001.html is bug 1392161
* text-orientation-script-001.html is w3c/web-platform-tests#6949 and bug 1392170
* @import-related tests have been fixed

I'm not too concerned about test that stylo passes while gecko fails. We may want to file bugs for some of them against bug 1365771, though.

I don't think there is any code work in this bug. It should basically just a meta, so downgrade to P5.

Comment 10

[Xidorn Quan [:xidorn] UTC+10]

I found that there are some stylo-specific issues remaining in shape-outside tests. IIRC they are related to in unit. We may need to investigate what happens there.

Comment 11

[Boris Chiou [:boris]]

All the dependencies are fixed?

Comment 12

[Xidorn Quan [:xidorn] UTC+10]

There are still some stylo-specific failures:

Known ones:
* 2dcontext/text-styles/2d.text.font.parse.system.html: this is because of bug 1374885, where Gecko's behavior (returning empty) is definitely undesired, but happens to satisfy this test.
* css/css-align-3/content-distribution/place-content-shorthand-00{2,4,6}.html: this is bug 1363971 where we need to update the impl to match the new spec.
* css/css-fonts-3/test_font_feature_values_parsing.html: seems to be some parsing / serialization disagreement.

Need investigation:
* css/css-variables-1/variable-animation-substitute-within-keyframe*.html
* css/css-variables-1/variable-substitution-variable-declaration.html
* cssom/getComputedStyle-pseudo.html

Shadow DOM bug 1293844 related failures:
* dom/interfaces.html
* html/semantics/forms/the-label-element/label-attributes.html
* intersection-observer/shadow-content.html
* shadow-dom/*

Comment 13

[Emilio Cobos Álvarez (:emilio)]

I don't think there's any point in keeping this open.