1950721 - Firefox fails WPT css/css-writing-modes/wm-propagation-body-044.html due to a test bug

Status: NEW

(Core :: Layout, task)

Description

[Daniel Holbert [:dholbert]]

We fail this WPT:
wpt.fyi: https://wpt.fyi/results/css/css-writing-modes/wm-propagation-body-044.html
wpt.live: https://wpt.live/css/css-writing-modes/wm-propagation-body-044.html

Failure annotation, showing that we only fail on linux: https://searchfox.org/mozilla-central/source/testing/web-platform/meta/css/css-writing-modes/wm-propagation-body-044.html.ini

The test-failure comparison screenshot on wpt.fyi (linked here currently shows that we're just sizing the scrollbar to be slightly different between the testcase and reference case. The scrollbar is only there in the first place because the descriptive text barely runs off the bottom of the screen...

I can reproduce that failure locally, in Firefox as well as Chrome, if I make the viewport the right height. So probably the only reason this shows up on wpt.fyi as a Firefox failure is because our choice of font happens to be one that triggers a little bit of overflow with the default WPT viewport-size, whereas Chrome's choice of font is slightly smaller or something.

Given that this doesn't actually seem to be a Firefox-specific behavior, I think this is probably a bug in the test's expectations -- we should just amend the test to avoid triggering this overflow (or avoid triggering it in a way that differs in overflow-amount between testcase and reference case).

Comment 1

[Daniel Holbert [:dholbert]]

Attachment: screenshot of testcase in Chrome with 516px tall viewport

Comment 2

[Daniel Holbert [:dholbert]]

Attachment: screenshot of reference case in Chrome with 516px tall viewport

Comment 3

[Daniel Holbert [:dholbert]]

If you compare those two screenshots^ and look at the scrollbar at the bottom right corner, you can see that Chrome has the exact same rendering difference that Firefox does -- the testcase has less scrollable overflow (and hence a taller scrollbar-thumb) than the reference case.

This demonstrates that Firefox's behavior here is interoperable with Chrome at least, and we're just at the mercy of whether the font that we use happens to trigger this situation with the standard WPT viewport.

The issue is worse if you make the window even shorter -- the text starts linewrapping in opposite directions in the testcase vs. the reference case, because one has veritcal-lr and the other has vertical-rl. Probably no font would run into that, but it's another sort of flakiness that the current test design is potentially subject to.

One simple way to address both issues would just be to replace that long "This text must be written vertically" string with a single shorter unwrappable string, e.g VerticalTextBelowOrangeSquare

Comment 4

[Daniel Holbert [:dholbert]]

(Alternately, we could just add body { margin-bottom: 0 }. That margin-bottom seems to be the thing that's causing the difference in scrollable overflow here, due to subtleties around the relationship between this text and the body/html element and the writing-modes that are involved in the testcase vs. reference case.)

Comment 5

[Daniel Holbert [:dholbert]]

(Whatever we do probably needs a test-change-proposal to follow the official procedure, filed at https://github.com/web-platform-tests/interop/issues .)

Comment 6

[Jonathan Kew [:jfkthame]]

I've filed a test change proposal at https://github.com/web-platform-tests/interop/issues/936 to address this.