Closed Bug 1213731 Opened 4 years ago Closed 3 years ago

Intermittent media tests ERROR test_video_playback.py TestVideoPlayback.test_video_playback_partial

Categories

(Core :: Audio/Video: Playback, defect, P5)

defect

Tracking

()

RESOLVED FIXED
mozilla51
Tracking Status
firefox50 --- fixed
firefox51 --- fixed

People

(Reporter: nigelb, Assigned: bryce)

References

()

Details

(Keywords: intermittent-failure)

Attachments

(2 files)

https://treeherder.mozilla.org/logviewer.html#?job_id=15495436&repo=mozilla-inbound


 20:10:02    ERROR -   2:13.18 TEST_END: MainThread ERROR, expected PASS
 20:10:02     INFO -  Traceback (most recent call last):
 20:10:02     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/marionette/marionette_test.py", line 296, in run
 20:10:02     INFO -      testMethod()
 20:10:02     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/firefox_media_tests/playback/test_video_playback.py", line 44, in test_video_playback_partial
 20:10:02     INFO -      set_duration=60)
 20:10:02     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/media_utils/video_puppeteer.py", line 73, in __init__
 20:10:02     INFO -      self.marionette.navigate(self.test_url)
 20:10:02     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/marionette_driver/marionette.py", line 1318, in navigate
 20:10:02     INFO -      self._send_message("get", {"url": url})
 20:10:02     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/marionette_driver/decorators.py", line 36, in _
 20:10:02     INFO -      return func(*args, **kwargs)
 20:10:02     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/marionette_driver/marionette.py", line 711, in _send_message
 20:10:02     INFO -      self._handle_error(resp)
 20:10:02     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/marionette_driver/marionette.py", line 752, in _handle_error
 20:10:02     INFO -      raise errors.lookup(error)(message, stacktrace=stacktrace)
 20:10:02    ERROR -  TimeoutException: TimeoutException: Error loading page, timed out (checkLoad)
 20:10:02     INFO -   2:13.33 LOG: MainThread INFO START LOG:
 20:10:02     INFO -   2:13.33 LOG: MainThread INFO INFO TEST-START: /builds/slave/test/build/venv/local/lib/python2.7/site-packages/firefox_media_tests/playback/test_video_playback.py:test_playback_starts Sun Oct 11 2015 20:07:54 GMT-0700 (PDT)
 20:10:02     INFO -   2:13.33 LOG: MainThread INFO INFO URL: https://youtu.be/AbAACm1IQE0 Sun Oct 11 2015 20:07:57 GMT-0700 (PDT)
 20:10:02     INFO -   2:13.33 LOG: MainThread INFO INFO video element obtained Sun Oct 11 2015 20:07:59 GMT-0700 (PDT)
 20:10:02     INFO -   2:13.33 LOG: MainThread INFO INFO URL: https://www.youtube.com/watch?v=yOQQCoxs8-k Sun Oct 11 2015 20:08:02 GMT-0700 (PDT)
 20:10:02     INFO -   2:13.33 LOG: MainThread INFO INFO video element obtained Sun Oct 11 2015 20:08:03 GMT-0700 (PDT)
 20:10:02     INFO -   2:13.34 LOG: MainThread INFO INFO URL: https://www.youtube.com/watch?v=1visYpIREUM Sun Oct 11 2015 20:08:07 GMT-0700 (PDT)
 20:10:02     INFO -   2:13.34 LOG: MainThread INFO INFO video element obtained Sun Oct 11 2015 20:08:08 GMT-0700 (PDT)
 20:10:02     INFO -   2:13.34 LOG: MainThread INFO INFO URL: https://www.youtube.com/watch?v=rjmuKV9BTkE Sun Oct 11 2015 20:08:12 GMT-0700 (PDT)
 20:10:02     INFO -   2:13.34 LOG: MainThread INFO INFO video element obtained Sun Oct 11 2015 20:08:14 GMT-0700 (PDT)
 20:10:02     INFO -   2:13.34 LOG: MainThread INFO INFO TEST-END: /builds/slave/test/build/venv/local/lib/python2.7/site-packages/firefox_media_tests/playback/test_video_playback.py:test_playback_starts Sun Oct 11 2015 20:08:15 GMT-0700 (PDT)
 20:10:02     INFO -   2:13.34 LOG: MainThread INFO END LOG:
 20:10:02     INFO -   2:13.34 LOG: MainThread INFO START LOG:
 20:10:02     INFO -   2:13.34 LOG: MainThread INFO INFO TEST-START: /builds/slave/test/build/venv/local/lib/python2.7/site-packages/firefox_media_tests/playback/test_video_playback.py:test_video_playback_partial Sun Oct 11 2015 20:08:15 GMT-0700 (PDT)
 20:10:02     INFO -   2:13.34 LOG: MainThread INFO INFO URL: https://youtu.be/AbAACm1IQE0 Sun Oct 11 2015 20:08:17 GMT-0700 (PDT)
 20:10:02     INFO -   2:13.34 LOG: MainThread INFO INFO video element obtained Sun Oct 11 2015 20:08:19 GMT-0700 (PDT)
 20:10:02     INFO -   2:13.34 LOG: MainThread INFO INFO URL: https://www.youtube.com/watch?v=yOQQCoxs8-k Sun Oct 11 2015 20:08:31 GMT-0700 (PDT)
 20:10:02     INFO -   2:13.34 LOG: MainThread INFO INFO video element obtained Sun Oct 11 2015 20:08:32 GMT-0700 (PDT)
 20:10:02     INFO -   2:13.34 LOG: MainThread INFO INFO TEST-END: /builds/slave/test/build/venv/local/lib/python2.7/site-packages/firefox_media_tests/playback/test_video_playback.py:test_video_playback_partial Sun Oct 11 2015 20:10:02 GMT-0700 (PDT)
 20:10:02     INFO -   2:13.34 LOG: MainThread INFO END LOG:
 20:10:02     INFO -   2:13.34 LOG: MainThread INFO
 20:10:02     INFO -  SUMMARY
 20:10:02     INFO -  -------
 20:10:02     INFO -   2:13.34 LOG: MainThread INFO passed: 1
 20:10:02     INFO -   2:13.34 LOG: MainThread INFO failed: 1
 20:10:02     INFO -   2:13.34 LOG: MainThread INFO todo: 0
 20:10:02     INFO -   2:13.34 LOG: MainThread INFO
 20:10:02    ERROR -  FAILED TESTS
 20:10:02     INFO -  -------
 20:10:02     INFO -   2:13.34 LOG: MainThread INFO test_video_playback.py test_video_playback.TestVideoPlayback.test_video_playback_partial
 20:10:03     INFO -   2:13.49 SUITE_END: MainThread
 20:10:03     INFO -  Summary
 20:10:03     INFO -  =======
 20:10:03     INFO -  Ran 4 tests
 20:10:03     INFO -  Expected results: 2
 20:10:03    ERROR -  Unexpected results: 2 (ERROR: 2)
 20:10:03     INFO -  Unexpected Results
 20:10:03     INFO -  ==================
 20:10:03    ERROR -  ERROR test_video_playback.py TestVideoPlayback.test_video_playback_partial
 20:10:03    ERROR -  ERROR test_video_playback.py TestVideoPlayback.test_video_playback_partial
 20:10:03    ERROR - Return code: 10
 20:10:03  WARNING - # TBPL WARNING #
Priority: -- → P5
Include Benjamin.
Looks like there at least two issues getting registered against this in :
- The good old autoplay starts the next video and the test freaks out because it's now at another URL/video. There may be a greater timing issue here, based on seeing some of the tests timing out, while still appearing to be on the correct URL and having started playback.
- Another issue is the tests not registering that a page has loaded. It looks like there's a DOM exception being fired.

I'm going to try and address this first issue by changing the problematic videos for the first case to embedded version to prevent autoplay taking the tests to another video. If there is more than just that going on, then this will help isolate the issue.
Assignee: nobody → bvandyk
Status: NEW → ASSIGNED
Comment on attachment 8786238 [details]
Bug 1213731 - Rework Youtube puppeteer to work with both youtube and embedded youtube.

https://reviewboard.mozilla.org/r/75240/#review73702
Attachment #8786238 - Flags: review?(mjzffr) → review+
Comment on attachment 8786239 [details]
Bug 1213731 - Remove unused Youtube URLs, use embedded URLs where possible.

https://reviewboard.mozilla.org/r/75242/#review73704
Attachment #8786239 - Flags: review?(mjzffr) → review+
Pushed by bvandyk@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/98465d11d072
Rework Youtube puppeteer to work with both youtube and embedded youtube. r=maja_zf
https://hg.mozilla.org/integration/autoland/rev/50dd3cba071e
Remove unused Youtube URLs, use embedded URLs where possible. r=maja_zf
I'm going to start tracking changes to address with other bugs (which I should have done from the start), rather than continually piling patches here.
Depends on: 1299756
https://hg.mozilla.org/mozilla-central/rev/98465d11d072
https://hg.mozilla.org/mozilla-central/rev/50dd3cba071e
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla51
You need to log in before you can comment on or make changes to this bug.