Closed Bug 1514265 Opened 5 years ago Closed 1 year ago

[meta] [CSS Grid] Firefox only web platform test failures

Categories

(Core :: Layout: Grid, task, P3)

Product:
Core
Component:
Layout: Grid
Type:
task

Tracking

()

Status:
RESOLVED INCOMPLETE

People

(Reporter: automatedtester, Unassigned)

References

(Depends on 4 open bugs, Blocks 1 open bug)

Details

(Keywords: meta, parity-chrome, parity-safari) 

The following web platform tests are failing only in firefox as documented in https://foolip.github.io/ad-hoc-wpt-results-analysis/firefox-lone-failures.html

/css/css-grid/abspos/grid-positioned-items-implicit-grid-line-001.html
/css/css-grid/abspos/grid-positioned-items-padding-001.html
/css/css-grid/abspos/grid-positioned-items-unknown-named-grid-line-001.html
/css/css-grid/alignment/grid-alignment-style-changes-003.html
/css/css-grid/alignment/grid-alignment-style-changes-004.html
/css/css-grid/alignment/grid-alignment-style-changes-005.html
/css/css-grid/alignment/grid-alignment-style-changes-006.html
/css/css-grid/alignment/grid-alignment-style-changes-007.html
/css/css-grid/alignment/grid-alignment-style-changes-008.html
/css/css-grid/alignment/grid-content-alignment-second-pass-001.html
/css/css-grid/alignment/grid-self-alignment-non-static-positioned-items-008.html
/css/css-grid/alignment/grid-self-alignment-positioned-items-with-margin-border-padding-016.html
/css/css-grid/alignment/grid-self-baseline-not-applied-if-sizing-cyclic-dependency-001.html
/css/css-grid/alignment/grid-self-baseline-not-applied-if-sizing-cyclic-dependency-002.html
/css/css-grid/grid-definition/grid-change-fit-content-argument-001.html
/css/css-grid/grid-definition/grid-inline-template-columns-rows-resolved-values-001.html
/css/css-grid/grid-definition/grid-template-columns-rows-resolved-values-001.html
/css/css-grid/grid-layout-properties.html
/css/css-grid/grid-model/grid-container-ignores-first-letter-001.html
/css/css-grid/grid-model/grid-container-ignores-first-line-001.html
/css/css-grid/grid-model/grid-container-scrollbar-vertical-lr-001.html
/css/css-grid/grid-model/grid-container-sizing-constraints-001.html
/css/css-grid/grid-model/grid-first-letter-003.html
/css/css-grid/grid-model/grid-first-line-003.html
/css/css-grid/grid-model/grid-inline-first-letter-003.html
/css/css-grid/grid-model/grid-inline-first-line-003.html
/css/css-grid/implicit-grids/grid-support-grid-auto-columns-rows-002.html
/css/css-grid/layout-algorithm/grid-intrinsic-size-with-orthogonal-items.html
/css/css-grid/parsing/grid-auto-columns-valid.html
/css/css-grid/parsing/grid-auto-rows-valid.html
/css/css-grid/parsing/grid-template-areas-valid.html
I triaged some of these failures in Oct 2018.
Here are the comments I made back then (they're probably still valid):

====
/css/css-grid/abspos/grid-positioned-items-implicit-grid-line-001.html

This is a bogus test.  I'm guessing other UAs haven't implemented
bug 1490730 yet.  I filed bug 1500622 to fix this test.

/css/css-grid/alignment/grid-alignment-style-changes-003.html
/css/css-grid/alignment/grid-alignment-style-changes-004.html

Our rendering is correct AFAICT - these tests are bogus since
they expect the buggy layout in Chrome.  I filed bug 1500630
to fix the tests. (FYI, writing-mode:vertical-rl is rather buggy
in Chrome, so if you see failures involving that then you should
be skeptical - it's very likely that our rendering is correct)

/css/css-grid/grid-model/grid-first-letter-003.html
/css/css-grid/grid-model/grid-first-line-003.html
/css/css-grid/grid-model/grid-inline-first-letter-003.html
/css/css-grid/grid-model/grid-inline-first-line-003.html

These are valid bugs, but they have nothing to do with Grid layout.
It's bug 214004 / bug 317081 (::first-letter/line w. nested blocks).

/css/css-grid/implicit-grids/grid-support-grid-auto-columns-rows-002.html

This is a valid Grid bug, but we're already aware of it: 1339672
These tests are also bogus:
/css/css-grid/abspos/grid-positioned-items-padding-001.html
/css/css-grid/abspos/grid-positioned-items-unknown-named-grid-line-001.html

I filed a Chrome bug https://bugs.chromium.org/p/chromium/issues/detail?id=915551
/css/css-grid/implicit-grids/grid-support-grid-auto-columns-rows-002.html
/css/css-grid/parsing/grid-auto-columns-valid.html
/css/css-grid/parsing/grid-auto-rows-valid.html

all fail due to bug 1339672.
Depends on: 1500590
The following tests are bogus because they depend on line-height metrics,
which differs between UAs:
/css/css-grid/alignment/grid-alignment-style-changes-005.html
/css/css-grid/alignment/grid-alignment-style-changes-006.html
/css/css-grid/alignment/grid-alignment-style-changes-007.html
/css/css-grid/alignment/grid-alignment-style-changes-008.html

Adding 'line-height:1' makes us pass.  Filed bug 1514658 to fix those tests.
Depends on: 1514658
Mats, could you set a priority?
Flags: needinfo?(mats)
Keywords: parity-chrome, parity-safari

http://w3c-test.org/css/CSS2/backgrounds/background-bg-pos-206.xht

is passed by Firefox 66 even though firefox-lone-failures.html and wpt lists it as failed by Firefox 66 on Jan 3, 2019.

I just noticed this bug report is for CSS-grid only ... nevertheless, I think there must be a problem somewhere with some of the test results found to be failed by Firefox!

Gerard,

All of these tests are disabled in web platform tests[1] for Firefox so there appears to be some issue. Could you help diagnose what that might be?

[1] https://searchfox.org/mozilla-central/source/testing/web-platform/meta/css/CSS2/backgrounds

Flags: needinfo?(bugzilla)

Could you help diagnose what that might be?

Okay... I think I found why they have been reported as failures...

https://searchfox.org/mozilla-central/source/testing/web-platform/meta/css/css-backgrounds/background-334.html.ini
states
[background-334.html]
disabled: antialiasing

https://searchfox.org/mozilla-central/source/testing/web-platform/meta/css/css-backgrounds/background-attachment-local/attachment-local-clipping-color-3.html.ini
states
[attachment-local-clipping-color-3.html]
expected:
if debug and not webrender and e10s and (os == "win") and (version == "6.1.7601") and (processor == "x86") and (bits == 32): FAIL
if not debug and not webrender and e10s and (os == "win") and (version == "6.1.7601") and (processor == "x86") and (bits == 32): FAIL
if debug and not webrender and e10s and (os == "win") and (version == "10.0.15063") and (processor == "x86_64") and (bits == 64): FAIL
if not debug and not webrender and e10s and (os == "win") and (version == "10.0.15063") and (processor == "x86_64") and (bits == 64): FAIL

https://searchfox.org/mozilla-central/source/testing/web-platform/meta/css/css-backgrounds/background-attachment-local/attachment-local-positioning-2.html.ini
states
expected:
if debug and not webrender and e10s and (os == "win") and (version == "6.1.7601") and (processor == "x86") and (bits == 32): FAIL
if not debug and not webrender and e10s and (os == "win") and (version == "6.1.7601") and (processor == "x86") and (bits == 32): FAIL
if debug and not webrender and e10s and (os == "win") and (version == "10.0.15063") and (processor == "x86_64") and (bits == 64): FAIL
if not debug and not webrender and e10s and (os == "win") and (version == "10.0.15063") and (processor == "x86_64") and (bits == 64): FAIL

https://searchfox.org/mozilla-central/source/testing/web-platform/meta/css/css-writing-modes/mongolian-orientation-002.html.ini
states
expected:
if os == "android": PASS
FAIL

https://searchfox.org/mozilla-central/source/testing/web-platform/meta/css/CSS2/backgrounds/background-bg-pos-206.xht.ini
states
expected:
if debug and not webrender and e10s and (os == "win") and (version == "6.1.7601") and (processor == "x86") and (bits == 32): FAIL
if not debug and not webrender and e10s and (os == "win") and (version == "6.1.7601") and (processor == "x86") and (bits == 32): FAIL
if debug and not webrender and e10s and (os == "win") and (version == "10.0.15063") and (processor == "x86_64") and (bits == 64): FAIL
if not debug and not webrender and e10s and (os == "win") and (version == "10.0.15063") and (processor == "x86_64") and (bits == 64): FAIL
if os == "android": FAIL

I do not use Android and I do not use Windows. Under Linux, I see no problem with those tests.

attachment-local-clipping-color-3.html appears to be slightly different from the one submitted over at wpt:

https://dxr.mozilla.org/mozilla-central/source/layout/reftests/backgrounds/attachment-local-clipping-color-3.html
uses a 32px blue square
while the one submitted
http://w3c-test.org/css/css-backgrounds/background-attachment-local/attachment-local-clipping-color-3.html
uses "Text" ... but that may be unrelated.

Flags: needinfo?(bugzilla)

The aqua-yellow-32x32.png image in

http://test.csswg.org/suites/css-backgrounds-3_dev/nightly-unstable/html4/attachment-local-positioning-2.htm

is not fetched and is 404 Not Found. But it is fetched and loaded in

http://w3c-test.org/css/css-backgrounds/background-attachment-local/attachment-local-positioning-2.html

I'll ask Peter Linss about this...

Depends on: 290125
Blocks: 1546389
No longer blocks: wpt.fyi-firefox-fails

THe updated failing tests for css grid are

/css/css-grid/abspos/grid-positioned-items-implicit-grid-line-001.html [2 subtests]
▶ /css/css-grid/alignment/grid-alignment-style-changes-001.html [3 subtests]
▶ /css/css-grid/alignment/grid-alignment-style-changes-002.html [3 subtests]
▶ /css/css-grid/alignment/grid-alignment-style-changes-003.html [5 subtests]
▶ /css/css-grid/alignment/grid-alignment-style-changes-004.html [5 subtests]
▶ /css/css-grid/alignment/grid-column-axis-self-baseline-synthesized-001.html [4 subtests]
▶ /css/css-grid/alignment/grid-column-axis-self-baseline-synthesized-002.html [3 subtests]
▶ /css/css-grid/alignment/grid-column-axis-self-baseline-synthesized-003.html [3 subtests]
▶ /css/css-grid/alignment/grid-column-axis-self-baseline-synthesized-004.html [3 subtests]
▶ /css/css-grid/alignment/grid-container-baseline-001.html [7 subtests]
▶ /css/css-grid/alignment/grid-content-alignment-second-pass-001.html [6 subtests]
▶ /css/css-grid/alignment/grid-content-alignment-second-pass-002.html [8 subtests]
▶ /css/css-grid/alignment/grid-row-axis-self-baseline-synthesized-001.html [4 subtests]
▶ /css/css-grid/alignment/grid-row-axis-self-baseline-synthesized-002.html [4 subtests]
▶ /css/css-grid/alignment/grid-row-axis-self-baseline-synthesized-003.html [4 subtests]
▶ /css/css-grid/alignment/grid-row-axis-self-baseline-synthesized-004.html [4 subtests]
▶ /css/css-grid/alignment/grid-self-alignment-non-static-positioned-items-008.html [3 subtests]
▶ /css/css-grid/alignment/grid-self-alignment-positioned-items-with-margin-border-padding-016.html [1 subtests]
▶ /css/css-grid/alignment/grid-self-baseline-not-applied-if-sizing-cyclic-dependency-001.html [3 subtests]
▶ /css/css-grid/alignment/grid-self-baseline-not-applied-if-sizing-cyclic-dependency-002.html [24 subtests]
▶ /css/css-grid/alignment/grid-self-baseline-not-applied-if-sizing-cyclic-dependency-003.html [6 subtests]
▶ /css/css-grid/grid-definition/grid-auto-repeat-max-size-001.html [2 subtests]
▶ /css/css-grid/grid-definition/grid-auto-repeat-min-size-001.html [4 subtests]
▶ /css/css-grid/grid-definition/grid-change-fit-content-argument-001.html [2 subtests]
▶ /css/css-grid/grid-definition/grid-inline-template-columns-rows-resolved-values-001.html [15 subtests]
▶ /css/css-grid/grid-definition/grid-percentage-rows-indefinite-height-001.html [104 subtests]
▶ /css/css-grid/grid-definition/grid-percentage-rows-indefinite-height-002.html [4 subtests]
▶ /css/css-grid/grid-definition/grid-template-columns-rows-resolved-values-001.html [15 subtests]
▶ /css/css-grid/grid-layout-properties.html [5 subtests]
▶ /css/css-grid/grid-model/grid-container-ignores-first-letter-001.html [4 subtests]
▶ /css/css-grid/grid-model/grid-container-ignores-first-line-001.html [4 subtests]
▶ /css/css-grid/grid-model/grid-first-letter-003.html [1 subtests]
▶ /css/css-grid/grid-model/grid-first-line-003.html [1 subtests]
▶ /css/css-grid/grid-model/grid-inline-first-letter-003.html [1 subtests]
▶ /css/css-grid/grid-model/grid-inline-first-line-003.html [1 subtests]
▶ /css/css-grid/implicit-grids/grid-support-grid-auto-columns-rows-002.html [1 subtests]
▶ /css/css-grid/inheritance.html [20 subtests]
▶ /css/css-grid/layout-algorithm/grid-content-distribution-must-account-for-track-sizing-002.html [1 subtests]
▶ /css/css-grid/layout-algorithm/grid-intrinsic-size-with-orthogonal-items.html [7 subtests]
▶ /css/css-grid/layout-algorithm/grid-minimum-contribution-baseline-shim-vertical-lr.html [2 subtests]
▶ /css/css-grid/layout-algorithm/grid-minimum-contribution-baseline-shim-vertical-rl.html [3 subtests]
▶ /css/css-grid/layout-algorithm/grid-minimum-contribution-baseline-shim.html [2 subtests]
▶ /css/css-grid/layout-algorithm/grid-percent-rows-filled-shrinkwrap-001.html [1 subtests]
▶ /css/css-grid/layout-algorithm/grid-percent-rows-spanned-shrinkwrap-001.html [1 subtests]
▶ /css/css-grid/parsing/grid-area-computed.html [25 subtests]
▶ /css/css-grid/parsing/grid-auto-columns-computed.html [25 subtests]
▶ /css/css-grid/parsing/grid-auto-columns-valid.html [2 subtests]
▶ /css/css-grid/parsing/grid-auto-flow-computed.html [4 subtests]
▶ /css/css-grid/parsing/grid-auto-rows-computed.html [25 subtests]
▶ /css/css-grid/parsing/grid-auto-rows-valid.html [2 subtests]
▶ /css/css-grid/parsing/grid-template-areas-computed.html [9 subtests]
▶ /css/css-grid/parsing/grid-template-areas-valid.html [3 subtests]

There is a lot more failing tests for this since the original issue

All the parsing and inheritance ones are bogus per https://github.com/web-platform-tests/wpt/pull/16892.

Priority: -- → P3

Just notice there are some shorthand and getComputedStyle() issues on this Firefox-only wpt failure:

css/css-grid/parsing/grid-area-computed.html [13 subtests]

and the spec issues are 1033 and 2529.

I'll turn this into a meta bug since many of the test failures here seems to be from bogus tests.
Please file any real bugs separately and mark it as blocking this bug. Thanks.

Type: defect → task
Flags: needinfo?(mats)
Keywords: meta
Summary: [CSS Grid] Firefox only web platform test failures → [meta] [CSS Grid] Firefox only web platform test failures
Severity: normal → S3

I don't think this metabug serves a useful purpose at this point; it looks like we fixed some of these, and for remaining failures, we can track them as-needed on a case by case basis (and comment 0's analysis site https://foolip.github.io/ad-hoc-wpt-results-analysis/firefox-lone-failures.html doesn't load anymore).

Status: NEW → RESOLVED
Closed: 1 year ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.