1298762 - Intermittent media tests FAIL test_video_playback.py TestVideoPlayback.test_playback_starts | after AssertionError: Error loading page, timed out (onDOMContentLoaded)

Status: RESOLVED INCOMPLETE

(Core :: Audio/Video: Playback, defect, P5)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

+++ This bug was initially created as a clone of Bug #1211959 +++

https://treeherder.mozilla.org/logviewer.html#?job_id=2669523&repo=autoland

18:48:32     INFO -   0:12.26 TEST_START: MainThread test_video_playback.py TestVideoPlayback.test_playback_starts
18:48:42     INFO -   0:22.27 LOG: MainThread INFO https://youtu.be/AbAACm1IQE0
18:48:51     INFO -   0:31.40 LOG: MainThread INFO https://www.youtube.com/watch?v=yOQQCoxs8-k
18:49:28     INFO -   1:07.85 LOG: MainThread INFO https://www.youtube.com/watch?v=1visYpIREUM
18:50:20    ERROR -   1:59.76 TEST_END: MainThread FAIL, expected PASS
18:50:20     INFO -  Traceback (most recent call last):
18:50:20     INFO -    File "C:\slave\test\build\venv\lib\site-packages\marionette\marionette_test.py", line 351, in run
18:50:20     INFO -      testMethod()
18:50:20     INFO -    File "C:\slave\test\build\venv\lib\site-packages\external_media_harness\testcase.py", line 169, in test_playback_starts
18:50:20    ERROR -      raise self.failureException(e)
18:50:20    ERROR -  AssertionError: Error loading page, timed out (onDOMContentLoaded)
18:50:28     INFO -   2:08.68 TEST_START: MainThread test_video_playback.py TestVideoPlayback.test_video_playback_partial
18:50:35     INFO -   2:15.05 LOG: MainThread INFO https://youtu.be/AbAACm1IQE0
18:50:55     INFO -   2:34.86 LOG: MainThread INFO https://www.youtube.com/watch?v=yOQQCoxs8-k
18:52:04     INFO -   3:44.42 LOG: MainThread INFO https://www.youtube.com/watch?v=1visYpIREUM
18:52:12     INFO -   3:52.54 LOG: MainThread INFO https://www.youtube.com/watch?v=rjmuKV9BTkE
18:53:12     INFO -   4:52.68 TEST_END: MainThread PASS
18:53:12     INFO -   4:52.68 LOG: MainThread INFO START LOG:
18:53:12     INFO -   4:52.68 LOG: MainThread INFO INFO TEST-START: C:\slave\test\build\venv\lib\site-packages\external_media_tests\playback\test_video_playback.py:test_playback_starts Sun Aug 28 2016 18:48:33 GMT-0700 (Pacific Standard Time)
18:53:12     INFO -   4:52.68 LOG: MainThread INFO INFO URL: https://youtu.be/AbAACm1IQE0 Sun Aug 28 2016 18:48:39 GMT-0700 (Pacific Standard Time)
18:53:12     INFO -   4:52.68 LOG: MainThread INFO INFO video element obtained Sun Aug 28 2016 18:48:40 GMT-0700 (Pacific Standard Time)
18:53:12     INFO -   4:52.68 LOG: MainThread INFO WARNING video is not mediasource: blob:https://www.youtube.com/b2b4e8ec-0196-4ca4-a982-c9c614555f11 Sun Aug 28 2016 18:48:43 GMT-0700 (Pacific Standard Time)
18:53:12     INFO -   4:52.68 LOG: MainThread INFO INFO URL: https://www.youtube.com/watch?v=yOQQCoxs8-k Sun Aug 28 2016 18:48:46 GMT-0700 (Pacific Standard Time)
18:53:12     INFO -   4:52.68 LOG: MainThread INFO INFO video element obtained Sun Aug 28 2016 18:48:49 GMT-0700 (Pacific Standard Time)
18:53:12     INFO -   4:52.68 LOG: MainThread INFO WARNING video is not mediasource: blob:https://www.youtube.com/5af996ff-3599-4720-bb9e-7c8cce03c3fb Sun Aug 28 2016 18:48:52 GMT-0700 (Pacific Standard Time)
18:53:12     INFO -   4:52.68 LOG: MainThread INFO INFO URL: https://www.youtube.com/watch?v=1visYpIREUM Sun Aug 28 2016 18:48:57 GMT-0700 (Pacific Standard Time)
18:53:12     INFO -   4:52.68 LOG: MainThread INFO INFO video element obtained Sun Aug 28 2016 18:48:57 GMT-0700 (Pacific Standard Time)
18:53:12     INFO -   4:52.68 LOG: MainThread INFO WARNING video is not mediasource: blob:https://www.youtube.com/6ecd62cd-2e30-4afe-89c9-4b44519f3923 Sun Aug 28 2016 18:49:37 GMT-0700 (Pacific Standard Time)
18:53:12     INFO -   4:52.68 LOG: MainThread INFO INFO TEST-END: C:\slave\test\build\venv\lib\site-packages\external_media_tests\playback\test_video_playback.py:test_playback_starts Sun Aug 28 2016 18:50:28 GMT-0700 (Pacific Standard Time)
18:53:12     INFO -   4:52.68 LOG: MainThread INFO END LOG:
18:53:12     INFO -   4:52.68 LOG: MainThread INFO START LOG:
18:53:12     INFO -   4:52.68 LOG: MainThread INFO INFO TEST-START: C:\slave\test\build\venv\lib\site-packages\external_media_tests\playback\test_video_playback.py:test_video_playback_partial Sun Aug 28 2016 18:50:29 GMT-0700 (Pacific Standard Time)
18:53:12     INFO -   4:52.68 LOG: MainThread INFO INFO URL: https://youtu.be/AbAACm1IQE0 Sun Aug 28 2016 18:50:33 GMT-0700 (Pacific Standard Time)
18:53:12     INFO -   4:52.68 LOG: MainThread INFO INFO video element obtained Sun Aug 28 2016 18:50:33 GMT-0700 (Pacific Standard Time)
18:53:12     INFO -   4:52.68 LOG: MainThread INFO INFO URL: https://www.youtube.com/watch?v=yOQQCoxs8-k Sun Aug 28 2016 18:50:53 GMT-0700 (Pacific Standard Time)
18:53:12     INFO -   4:52.68 LOG: MainThread INFO INFO video element obtained Sun Aug 28 2016 18:50:53 GMT-0700 (Pacific Standard Time)
18:53:12     INFO -   4:52.68 LOG: MainThread INFO INFO URL: https://www.youtube.com/watch?v=1visYpIREUM Sun Aug 28 2016 18:51:57 GMT-0700 (Pacific Standard Time)
18:53:12     INFO -   4:52.68 LOG: MainThread INFO INFO video element obtained Sun Aug 28 2016 18:51:58 GMT-0700 (Pacific Standard Time)
18:53:12     INFO -   4:52.68 LOG: MainThread INFO INFO URL: https://www.youtube.com/watch?v=rjmuKV9BTkE Sun Aug 28 2016 18:52:10 GMT-0700 (Pacific Standard Time)
18:53:12     INFO -   4:52.68 LOG: MainThread INFO INFO video element obtained Sun Aug 28 2016 18:52:11 GMT-0700 (Pacific Standard Time)
18:53:12     INFO -   4:52.68 LOG: MainThread INFO INFO TEST-END: C:\slave\test\build\venv\lib\site-packages\external_media_tests\playback\test_video_playback.py:test_video_playback_partial Sun Aug 28 2016 18:53:12 GMT-0700 (Pacific Standard Time)
18:53:12     INFO -   4:52.68 LOG: MainThread INFO END LOG:
18:53:12     INFO -   4:52.68 LOG: MainThread INFO
18:53:12     INFO -  SUMMARY
18:53:12     INFO -  -------
18:53:12     INFO -   4:52.68 LOG: MainThread INFO passed: 1
18:53:12     INFO -   4:52.68 LOG: MainThread INFO failed: 1
18:53:12     INFO -   4:52.68 LOG: MainThread INFO todo: 0
18:53:12     INFO -   4:52.68 LOG: MainThread INFO
18:53:12    ERROR -  FAILED TESTS
18:53:12     INFO -  -------
18:53:12     INFO -   4:52.68 LOG: MainThread INFO test_video_playback.py test_video_playback.TestVideoPlayback.test_playback_starts
18:53:12     INFO -   4:52.68 LOG: MainThread INFO mode: non-e10s
18:53:12     INFO -   4:52.68 SUITE_END: MainThread
18:53:12     INFO -  Summary
18:53:12     INFO -  =======
18:53:12     INFO -  Ran 4 tests
18:53:12     INFO -  Expected results: 2
18:53:12    ERROR -  Unexpected results: 2 (FAIL: 2)
18:53:12     INFO -  Unexpected Results
18:53:12     INFO -  ==================
18:53:12    ERROR -  FAIL test_video_playback.py TestVideoPlayback.test_playback_starts
18:53:12    ERROR -  FAIL test_video_playback.py TestVideoPlayback.test_playback_starts

Comment 1

[Intermittent Failures Robot]

15 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-inbound: 5
* autoland: 5
* try: 2
* fx-team: 2
* mozilla-central: 1

Platform breakdown:
* windows8-64: 5
* windows7-32: 5
* linux64: 5

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1298762&startday=2016-08-29&endday=2016-08-29&tree=all

Comment 2

[Intermittent Failures Robot]

15 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-inbound: 6
* autoland: 3
* mozilla-central: 2
* mozilla-beta: 2
* fx-team: 2

Platform breakdown:
* windows7-32: 6
* linux64: 6
* windows8-64: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1298762&startday=2016-08-30&endday=2016-08-30&tree=all

Comment 3

[Intermittent Failures Robot]

56 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 18
* autoland: 13
* fx-team: 8
* try: 6
* mozilla-central: 5
* mozilla-beta: 5
* mozilla-aurora: 1

Platform breakdown:
* linux64: 23
* windows7-32: 20
* windows8-64: 13

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1298762&startday=2016-08-29&endday=2016-09-04&tree=all

Comment 4

[Intermittent Failures Robot]

9 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 3
* fx-team: 3
* mozilla-release: 2
* mozilla-beta: 1

Platform breakdown:
* linux64: 6
* windows7-32: 2
* windows8-64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1298762&startday=2016-09-12&endday=2016-09-18&tree=all

Comment 5

[Intermittent Failures Robot]

5 failures in 526 pushes (0.01 failures/push) were associated with this bug in the last 7 days.  

Repository breakdown:
* mozilla-release: 4
* mozilla-esr45: 1

Platform breakdown:
* windows8-64: 2
* windows7-32: 1
* osx-10-10: 1
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1298762&startday=2016-12-12&endday=2016-12-18&tree=all

Comment 6

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Reworking of the tests has yielded a pretty big reduction in the false positives on this one. However, it is hard to eliminate them all due to the nature of our media test harness.

Looking over the last week of fails it looks like a number of timing related false positives (potentially compounded by autoplay). I'll keep an eye on what the robot has to say and will attempt to make sure the number of fails remains low.

Comment 7

[Intermittent Failures Robot]

7 failures in 836 pushes (0.008 failures/push) were associated with this bug in the last 7 days.  
Repository breakdown:
* mozilla-esr45: 3
* try: 1
* mozilla-inbound: 1
* mozilla-central: 1
* autoland: 1

Platform breakdown:
* windows7-32: 4
* linux64: 2
* windows8-64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1298762&startday=2017-02-06&endday=2017-02-12&tree=all

Comment 8

[Intermittent Failures Robot]

5 failures in 833 pushes (0.006 failures/push) were associated with this bug in the last 7 days.  
Repository breakdown:
* mozilla-central: 2
* mozilla-inbound: 1
* mozilla-esr45: 1
* autoland: 1

Platform breakdown:
* windows8-64: 2
* windows7-32: 2
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1298762&startday=2017-02-13&endday=2017-02-19&tree=all

Comment 9

[Intermittent Failures Robot]

5 failures in 812 pushes (0.006 failures/push) were associated with this bug in the last 7 days.  
Repository breakdown:
* mozilla-esr45: 2
* autoland: 2
* mozilla-inbound: 1

Platform breakdown:
* osx-10-10: 3
* windows8-64: 1
* windows7-32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1298762&startday=2017-02-20&endday=2017-02-26&tree=all

Comment 10

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.