1472172 - Old css/css-transitions tests flaky

Status: RESOLVED FIXED

(Core :: CSS Parsing and Computation, defect, P3)

Description

[Sam Sneddon [:gsnedders]]

These were disabled en-masse in bug 1356627 #c1, but when the big CSS rename in WPT happened, the directory disable didn't get copied over in bug 1419296.

See bug 1356627 #c3/4 for more discussion as to why these are flaky.

Tests in wpt/css/css-transitions using the old flaky test harness:

before-DOMContentLoaded-001.html
before-load-001.html
changing-while-transition.html
detached-container-001.html
hidden-container-001.html
properties-value-001.html
properties-value-002.html
properties-value-003.html
properties-value-auto-001.html
properties-value-implicit-001.html
properties-value-inherit-001.html
properties-value-inherit-002.html
properties-value-inherit-003.html
pseudo-elements-001.html

Comment 1

[Sam Sneddon [:gsnedders]]

So tests we've never had bugs filed for:

before-load-001.html
detached-container-001.html
hidden-container-001.html
properties-value-002.html
properties-value-auto-001.html
properties-value-inherit-003.html
pseudo-elements-001.html

Brian, you wouldn't happen to know off-hand if these are safe from being intermittent?

On the whole, I'm inclined to believe they can all be intermittent and (re)disable all of them.

Comment 2

[Sam Sneddon [:gsnedders]]

Attachment: Bug 1472172: Re-disable old css-transitions tests

The meta file in css-transitions-1 should've been, but wasn't, moved
in bug 1419296, causing these tests to be re-enabled. This re-disables
only the tests using the old, flaky harness.

Review commit: https://reviewboard.mozilla.org/r/253930/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/253930/

Comment 3

[Sam Sneddon [:gsnedders]]

Comment on attachment 8988726 [details]
Bug 1472172: Re-disable old css-transitions tests

https://reviewboard.mozilla.org/r/253930/#review260660

Comment 4

[Brian Birtles (:birtles)]

Comment on attachment 8988726 [details]
Bug 1472172: Re-disable old css-transitions tests

https://reviewboard.mozilla.org/r/253930/#review260858

Yes, let's disable this and write these tests properly.

Comment 5

[Brian Birtles (:birtles)]

(In reply to Geoffrey Sneddon [:gsnedders] from comment #1)
> So tests we've never had bugs filed for:
> 
> before-load-001.html
> detached-container-001.html
> hidden-container-001.html
> properties-value-002.html
> properties-value-auto-001.html
> properties-value-inherit-003.html
> pseudo-elements-001.html
> 
> Brian, you wouldn't happen to know off-hand if these are safe from being
> intermittent?
> 
> On the whole, I'm inclined to believe they can all be intermittent and
> (re)disable all of them.

I agree. I don't recognize any of those as being ones we've recently fixed.

Comment 6

[James Graham [:jgraham]]

Comment on attachment 8988726 [details]
Bug 1472172: Re-disable old css-transitions tests

https://reviewboard.mozilla.org/r/253930/#review260970

Why don't we delete these tests if they ouht to be disabled?

Comment 7

[Brian Birtles (:birtles)]

Because I plan to rewrite them (to make them sane and not flaky) and I'd like to use them as a guide to what should be covered.

Comment 8

[Sam Sneddon [:gsnedders]]

(In reply to James Graham [:jgraham] from comment #6)
> Comment on attachment 8988726 [details]
> Bug 1472172: Re-disable old css-transitions tests
> 
> https://reviewboard.mozilla.org/r/253930/#review260970
> 
> Why don't we delete these tests if they ouht to be disabled?

I was simply disabling them because that's what was done previously. Regardless of which we want to do, we've been running the flaky tests for another month now.

Comment 9

[Mozilla Autoland]

We're sorry, Autoland could not rebase your commits for you automatically. Please manually rebase your commits and try again.

hg error in cmd: hg rebase -s e59e05ae36be8559616fcd748cf4107e9687ad29 -d 8d88272387d1: rebasing 479229:e59e05ae36be "Bug 1472172: Re-disable old css-transitions tests r=birtles,jgraham" (tip)
merging testing/web-platform/meta/css/css-transitions/properties-value-001.html.ini
merging testing/web-platform/meta/css/css-transitions/properties-value-002.html.ini
merging testing/web-platform/meta/css/css-transitions/properties-value-003.html.ini
merging testing/web-platform/meta/css/css-transitions/properties-value-inherit-001.html.ini
merging testing/web-platform/meta/css/css-transitions/properties-value-inherit-002.html.ini
merging testing/web-platform/meta/css/css-transitions/properties-value-inherit-003.html.ini
warning: conflicts while merging testing/web-platform/meta/css/css-transitions/properties-value-001.html.ini! (edit, then use 'hg resolve --mark')
warning: conflicts while merging testing/web-platform/meta/css/css-transitions/properties-value-002.html.ini! (edit, then use 'hg resolve --mark')
warning: conflicts while merging testing/web-platform/meta/css/css-transitions/properties-value-003.html.ini! (edit, then use 'hg resolve --mark')
warning: conflicts while merging testing/web-platform/meta/css/css-transitions/properties-value-inherit-001.html.ini! (edit, then use 'hg resolve --mark')
warning: conflicts while merging testing/web-platform/meta/css/css-transitions/properties-value-inherit-002.html.ini! (edit, then use 'hg resolve --mark')
warning: conflicts while merging testing/web-platform/meta/css/css-transitions/properties-value-inherit-003.html.ini! (edit, then use 'hg resolve --mark')
unresolved conflicts (see hg resolve, then hg rebase --continue)

Comment 10

[Sam Sneddon [:gsnedders]]

Attachment: 1472172.patch

Here's an updated patch applies cleanly; I'm not going to spend half an hour trying to figure out how to install new tools to upload an updated patch that's already been reviewed, that's not a worthwhile use of my time.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/3c2793d1514f
Re-disable old css-transitions tests

Comment 12

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/3c2793d1514f

Comment 13

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/releases/mozilla-beta/rev/17abec484136