1351409 - Intermittent TEST-UNEXPECTED-TIMEOUT | testing/web-platform/tests/dom/events/EventListener-invoke-legacy.html | Listener of animationiteration - Test timed out

Status: RESOLVED FIXED

(Testing :: web-platform-tests, defect)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://treeherder.mozilla.org/logviewer.html#?job_id=86993022&repo=autoland

Not the first dom event related failure today, so there might be a real issue.

08:05:36     INFO - TEST-START | /dom/events/EventListener-invoke-legacy.html
08:05:46     INFO - PID 1596 | MARIONETTE LOG: INFO: Timeout fired
08:05:46     INFO - 
08:05:46     INFO - TEST-PASS | /dom/events/EventListener-invoke-legacy.html | Listener of transitionend 
08:05:46     INFO - TEST-PASS | /dom/events/EventListener-invoke-legacy.html | Legacy listener of transitionend 
08:05:46     INFO - TEST-PASS | /dom/events/EventListener-invoke-legacy.html | Listener of animationend 
08:05:46     INFO - TEST-PASS | /dom/events/EventListener-invoke-legacy.html | Legacy listener of animationend 
08:05:46     INFO - TEST-UNEXPECTED-TIMEOUT | /dom/events/EventListener-invoke-legacy.html | Listener of animationiteration - Test timed out
08:05:46     INFO - 
08:05:46     INFO - TEST-UNEXPECTED-TIMEOUT | /dom/events/EventListener-invoke-legacy.html | Legacy listener of animationiteration - Test timed out
08:05:46     INFO - TEST-UNEXPECTED-TIMEOUT | /dom/events/EventListener-invoke-legacy.html | expected OK

Comment 1

[Intermittent Failures Robot]

18 failures in 188 pushes (0.096 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* autoland: 10
* mozilla-inbound: 7
* try: 1

Platform breakdown:
* linux32: 5
* windows8-64: 4
* windows7-32-vm: 4
* osx-10-10: 3
* linux64: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1351409&startday=2017-03-29&endday=2017-03-29&tree=all

Comment 2

[Intermittent Failures Robot]

26 failures in 146 pushes (0.178 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* autoland: 13
* mozilla-inbound: 9
* try: 1
* oak: 1
* mozilla-central: 1
* graphics: 1

Platform breakdown:
* linux32: 9
* osx-10-10: 6
* linux64: 6
* windows8-64: 3
* windows7-32-vm: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1351409&startday=2017-03-31&endday=2017-03-31&tree=all

Comment 3

[Intermittent Failures Robot]

91 failures in 845 pushes (0.108 failures/push) were associated with this bug in the last 7 days. 

This is the #15 most frequent failure this week. 

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **  

Repository breakdown:
* autoland: 45
* mozilla-inbound: 22
* mozilla-central: 15
* try: 5
* oak: 2
* graphics: 2

Platform breakdown:
* linux32: 27
* osx-10-10: 18
* linux64: 18
* windows7-32-vm: 16
* windows8-64: 12

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1351409&startday=2017-03-27&endday=2017-04-02&tree=all

Comment 4

[Intermittent Failures Robot]

22 failures in 147 pushes (0.15 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* autoland: 12
* mozilla-inbound: 7
* try: 1
* oak: 1
* mozilla-central: 1

Platform breakdown:
* linux64: 8
* windows7-32-vm: 5
* linux32: 5
* osx-10-10: 4

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1351409&startday=2017-04-03&endday=2017-04-03&tree=all

Comment 5

[Geoff Brown [:gbrown]]

Failures affect all desktop platforms.

This test was added 2017-02-17 as part of a wpt update - bug 1340474. I don't see any failures for this test prior to March 28, but that's not a long history of stability.

Comment 6

[Geoff Brown [:gbrown]]

I see now: The test was disabled initially and re-enabled just before these failures started - bug 1340504.

:xidorn -- Is this good enough evidence for disabling now?! Do you want to try to fix it?

Comment 7

[Intermittent Failures Robot]

17 failures in 134 pushes (0.127 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* autoland: 8
* mozilla-inbound: 6
* try: 2
* mozilla-central: 1

Platform breakdown:
* linux32: 7
* windows7-32-vm: 5
* linux64: 3
* windows8-64: 1
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1351409&startday=2017-04-04&endday=2017-04-04&tree=all

Comment 8

[Xidorn Quan [:xidorn] UTC+10]

OK, then disable it, although I think we really want to figure out what's happening here, but I don't have that time right now...

Comment 9

[Intermittent Failures Robot]

28 failures in 151 pushes (0.185 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* try: 8
* autoland: 8
* mozilla-inbound: 6
* graphics: 3
* mozilla-central: 2
* oak: 1

Platform breakdown:
* linux32: 10
* windows7-32-vm: 9
* linux64: 5
* windows8-64: 2
* osx-10-10: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1351409&startday=2017-04-05&endday=2017-04-05&tree=all

Comment 10

[Intermittent Failures Robot]

17 failures in 170 pushes (0.1 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* mozilla-inbound: 6
* autoland: 6
* graphics: 2
* try: 1
* pine: 1
* mozilla-central: 1

Platform breakdown:
* linux64: 7
* windows7-32-vm: 3
* linux32: 3
* windows8-64: 2
* osx-10-10: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1351409&startday=2017-04-06&endday=2017-04-06&tree=all

Comment 11

[Intermittent Failures Robot]

18 failures in 185 pushes (0.097 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* autoland: 6
* oak: 4
* graphics: 4
* mozilla-inbound: 2
* mozilla-central: 2

Platform breakdown:
* linux64: 5
* linux32: 5
* windows8-64: 3
* windows7-32-vm: 3
* osx-10-10: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1351409&startday=2017-04-07&endday=2017-04-07&tree=all

Comment 12

[Intermittent Failures Robot]

142 failures in 867 pushes (0.164 failures/push) were associated with this bug in the last 7 days. 

This is the #10 most frequent failure this week. 

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **  

Repository breakdown:
* autoland: 58
* mozilla-inbound: 37
* try: 17
* mozilla-central: 15
* graphics: 8
* oak: 6
* pine: 1

Platform breakdown:
* linux32: 40
* windows7-32-vm: 34
* linux64: 32
* osx-10-10: 22
* windows8-64: 14

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1351409&startday=2017-04-03&endday=2017-04-09&tree=all

Comment 13

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: disable this temporarily

ok, lets disable this for now.

Comment 14

[James Graham [:jgraham]]

Comment on attachment 8856528 [details] [diff] [review]
disable this temporarily

Review of attachment 8856528 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/web-platform/meta/dom/events/EventListener-invoke-legacy.html.ini
@@ +1,2 @@
> +[EventListener-invoke-legacy.html]
> +  disabled: https://bugzilla.mozilla.org/show_bug.cgi?id=1351409

Can we make this:

  disabled:
    if (os=="win"): https://bugzilla.mozilla.org/show_bug.cgi?id=1351409

Comment 15

[Geoff Brown [:gbrown]]

(In reply to James Graham [:jgraham] from comment #14)
> Can we make this:
> 
>   disabled:
>     if (os=="win"): https://bugzilla.mozilla.org/show_bug.cgi?id=1351409

I don't think so - that would eliminate less than 50% of the failures (based on, say, comment 12).

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/b0f0ddeb1bcf
Intermittent /dom/events/EventListener-invoke-legacy.html. disable test. r=gbrown

Comment 17

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/b0f0ddeb1bcf

Comment 18

[Intermittent Failures Robot]

23 failures in 150 pushes (0.153 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* autoland: 10
* try: 6
* mozilla-central: 3
* graphics: 2
* oak: 1
* mozilla-inbound: 1

Platform breakdown:
* linux64: 9
* linux32: 7
* windows7-32-vm: 4
* windows8-64: 2
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1351409&startday=2017-04-10&endday=2017-04-10&tree=all

Comment 19

[Intermittent Failures Robot]

27 failures in 894 pushes (0.03 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 11
* try: 8
* mozilla-central: 3
* graphics: 3
* oak: 1
* mozilla-inbound: 1

Platform breakdown:
* linux64: 10
* linux32: 8
* windows7-32-vm: 5
* windows8-64: 3
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1351409&startday=2017-04-10&endday=2017-04-16&tree=all

Comment 20

[Xidorn Quan [:xidorn] UTC+10]

Attachment: Bug 1351409 - Remove animationiteration from legacy event listener test, make the transition tests more robust, and re-enable that test.

According to :birtles, it is not guaranteed that animationiteration
event will be fired. This event is sample-based rather than event-based,
and such behavior has been clarified in CSS Animations Level 2:
https://drafts.csswg.org/css-animations-2/#event-dispatch

Also, Chromium has the same issue with this test:
https://bugs.chromium.org/p/chromium/issues/detail?id=701445

Review commit: https://reviewboard.mozilla.org/r/209026/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/209026/

Comment 21

[Xidorn Quan [:xidorn] UTC+10]

The patch should fix this intermittent.

Comment 22

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8938260 [details]
Bug 1351409 - Remove animationiteration from legacy event listener test, make the transition tests more robust, and re-enable that test.

https://reviewboard.mozilla.org/r/209026/#review214780

Comment 23

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8c912df2f3a5
Remove animationiteration from legacy event listener test and re-enable that test. r=jmaher

Comment 24

[Natalia Csoregi [:nataliaCs]]

Backed out for failing dom/events/EventListener-invoke-legacy.html
Backout: https://hg.mozilla.org/integration/autoland/rev/3c1c19235a373f3f80ec685f2356824da4b40db1

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=8c912df2f3a5bdb67b4abcac25bb9e6e65de848a

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=152878459&repo=autoland&lineNumber=1206

[task 2017-12-21T11:13:28.932Z] 11:13:28     INFO - 
1206
[task 2017-12-21T11:13:28.932Z] 11:13:28     INFO - TEST-UNEXPECTED-FAIL | /dom/events/EventListener-invoke-legacy.html | Listener of transitionend - assert_false: unexpected transitionend event expected false got true
1207
[task 2017-12-21T11:13:28.933Z] 11:13:28     INFO - runLegacyEventTest/</<@http://web-platform.test:8000/dom/events/EventListener-invoke-legacy.html:30:7
1208
[task 2017-12-21T11:13:28.933Z] 11:13:28     INFO - Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:1489:20

Comment 25

[Xidorn Quan [:xidorn] UTC+10]

Comment on attachment 8938260 [details]
Bug 1351409 - Remove animationiteration from legacy event listener test, make the transition tests more robust, and re-enable that test.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/209026/diff/1-2/

Comment 26

[Xidorn Quan [:xidorn] UTC+10]

Comment on attachment 8938260 [details]
Bug 1351409 - Remove animationiteration from legacy event listener test, make the transition tests more robust, and re-enable that test.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/209026/diff/2-3/

Comment 27

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fa855df2e888
Remove animationiteration from legacy event listener test, make the transition tests more robust, and re-enable that test. r=jmaher

Comment 28

[Arthur Iakab [arthur_iakab]]

https://hg.mozilla.org/mozilla-central/rev/fa855df2e888