1594937 - Remove w3c-css/received/css-multicol reftests

Status: RESOLVED FIXED

(Core :: Layout, task)

Comment 1

[Ting-Yu Lin [:TYLin] (PDT, UTC-7)]

Attachment: Remove fuzzy notation and add <style> to properly load Ahem for received multicol reftests.

Note that this patch is used to analyze the fail/pass state in
w3c-css/css-multicol, not for check-in.

The following test are annotated "fails-if(!OSX) random-if(OSX)" due to
not load Ahem properly (a shortcoming in import-tests.py). The can be
fixed by manually adding proper <style> to load Ahem.

multicol-count-002.xht
multicol-inherit-003.xht
multicol-rule-001.xht
multicol-rule-002.xht
multicol-rule-px-001.xht
multicol-rule-stacking-001.xht
multicol-shorthand-001.xht
multicol-span-all-003.xht
multicol-width-count-002.xht
multicol-width-negative-001.xht

The rest of the failed tests match the state in wpt. Either they have
ini marked as failures in wpt, or they're fixed in wpt.

Here a try run of this patch.
https://treeherder.mozilla.org/#/jobs?repo=try&collapsedPushes=532734&revision=d4d2f9c2cb49a2366a80ebf5472e63c63ccf42f0

multicol-span-all-block-sibling-003.xht fails on Windows because of
anti-alias issue. wpt doesn't have the same problem, but I think it's
OK.

Comment 2

[Ting-Yu Lin [:TYLin] (PDT, UTC-7)]

Attachment: Remove fuzzy notation and add <style> to properly load Ahem for received multicol reftests.

Note that this patch is used to analyze the fail/pass state in
w3c-css/css-multicol, not for check-in.

The following test are annotated "fails-if(!OSX) random-if(OSX)" due to
not load Ahem properly (a shortcoming in import-tests.py). The can be
fixed by manually adding proper <style> to load Ahem.

multicol-count-002.xht
multicol-inherit-003.xht
multicol-rule-001.xht
multicol-rule-002.xht
multicol-rule-px-001.xht
multicol-rule-stacking-001.xht
multicol-shorthand-001.xht
multicol-span-all-003.xht
multicol-width-count-002.xht
multicol-width-negative-001.xht

The rest of the failed tests match the state in wpt. Either they have
ini marked as failures in wpt, or they're fixed in wpt.

Here a try run of this patch.
https://treeherder.mozilla.org/#/jobs?repo=try&collapsedPushes=532734&revision=d4d2f9c2cb49a2366a80ebf5472e63c63ccf42f0

multicol-span-all-block-sibling-003.xht fails on Windows because of
anti-alias issue. wpt doesn't have the same problem, but I think it's
OK.

Comment 3

[Ting-Yu Lin [:TYLin] (PDT, UTC-7)]

Comment on attachment 9107379 [details] [diff] [review]
Remove fuzzy notation and add <style> to properly load Ahem for received multicol reftests.

David, if the analysis in comment 2 looks reasonable, I'll write a patch to remove `received/css-multicol`

Comment 4

[Ting-Yu Lin [:TYLin] (PDT, UTC-7)]

Attachment: Bug 1594937 - Remove w3c-css/received/css-multicol reftests. r=dbaron

Bug 1594737 comment 2 provides an analysis on why it is OK to remove
css-multicol reftests.

The only manual modification in this patch is removing
os.path.join("css-multicol") in import-tests.py. Others parts are
generated by running import-tests.py on a wpt repository with commit
15f199c91a72b0d51bf0a12b3b77827ecb5051ff.

Comment 5

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Comment on attachment 9107379 [details] [diff] [review]
Remove fuzzy notation and add <style> to properly load Ahem for received multicol reftests.

This makes sense.

I think this points out that:
* we have a bunch of old `skip` and `fuzzy(0-X,0-Y)` annotations that ought to be cleaned up (since it would increase our test coverage)
* we have a bunch of `prefs: [layout.css.column-span.enabled:true]` in WPT ini files that should probably be cleaned up soon (although maybe not yet)

Comment 6

[Ting-Yu Lin [:TYLin] (PDT, UTC-7)]

I think this points out that:

• we have a bunch of old skip and fuzzy(0-X,0-Y) annotations that ought to be cleaned up (since it would increase our test coverage)

Right, we should remove those annotations when investigating other css modules.

• we have a bunch of prefs: [layout.css.column-span.enabled:true] in WPT ini files that should probably be cleaned up soon (although maybe not yet)

The pref can be removed in bug 1499281 after column-span reaches the release channel in 71.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/148081b861c8
Remove w3c-css/received/css-multicol reftests. r=dbaron

Comment 8

[Cosmin Sabou [:CosminS]]

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&selectedJob=277781054&resultStatus=testfailed%2Cbusted%2Cexception&revision=148081b861c81c70c9e79bdb06a395106ab3750e

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=277781054&repo=autoland

Backout link: https://hg.mozilla.org/integration/autoland/rev/4a51a464d8efabde6a41af89fbb6aace77613f98

Comment 9

[Ting-Yu Lin [:TYLin] (PDT, UTC-7)]

Hmm, my patch is unlikely to cause dom/tests/reftest/bug453105.html fail because it just removes a lot of reftests, without changing any gecko behavior.

Comment 10

[Ting-Yu Lin [:TYLin] (PDT, UTC-7)]

Pushed a try, and it seems bug453105.html on Android becomes nearly perma-orange for 1 pixel diff for some unknown reason, though some R2 runs did pass on Android 7.0 x86-64 opt. I've filed bug 1600534 for this, and will add a fuzzy-if for bug453105.html.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f4f9195bd628
Remove w3c-css/received/css-multicol reftests. r=dbaron

Comment 12

[Razvan Maries]

https://hg.mozilla.org/mozilla-central/rev/f4f9195bd628