1149964 - TEST-UNEXPECTED-FAIL | test_1_browser_call.py Test1BrowserCall.test_1_browser_call | AssertionError: media start time should be uninitialized before link clicker enters room

Status: RESOLVED FIXED

(Hello (Loop) :: Client, defect)

Description

[Mark Banner (:standard8)]

This is new on the functional tests since last week ish.

TEST-UNEXPECTED-FAIL | test_1_browser_call.py Test1BrowserCall.test_1_browser_call | AssertionError: media start time should be uninitialized before link clicker enters room
Traceback (most recent call last):
  File "/home/mozilla/jenkins/workspace/hello-e2e-marionette/venv/local/lib/python2.7/site-packages/marionette_client-0.9.2-py2.7.egg/marionette/marionette_test.py", line 296, in run
    testMethod()
  File "/home/mozilla/jenkins/workspace/hello-e2e-marionette/marionette/tests/browser/components/loop/test/functional/test_1_browser_call.py", line 235, in test_1_browser_call
    self.local_check_media_start_time_uninitialized()
  File "/home/mozilla/jenkins/workspace/hello-e2e-marionette/marionette/tests/browser/components/loop/test/functional/test_1_browser_call.py", line 208, in local_check_media_start_time_uninitialized
    "media start time should be uninitialized before "

Comment 1

[Mark Banner (:standard8)]

Without extra debugging, I'd almost be willing to bet on this:

- local_check_room_self_video() doesn't actually check that media is flowing to the video container, i.e. we've at least got to the stage where the window has finished loading and has started initialising.
- local_check_media_start_time_uninitialized() then goes and tests the start time straight away - the element might be there, but connectSession() might not have been called on the otSdkDriver.

Waiting for the video to actually be streaming might help this. This would at least ensure most of the initial steps have happened. There could still be a bit of a race as video streaming isn't necessarily the same as connectSession having been called, but I think it'd be close enough for a try.

Comment 2

[Mark Banner (:standard8)]

Attachment: Fix an intermittent error in Loop's functional tests (media start time should be uninitialized) and wait for the content server before starting tests. NPOTB

This fixes the failures locally for me. As my previous analysis thought, this was due to us not waiting for things to happen (e.g. video streams, or end of call), before trying to test things. Sometimes we're just a bit too slow or quick and the intermittent hits.

There was a second intermittent after the closing of the call - so I've added a check in for that as well.

Lastly, for some reason git/make is being a bit slow on my repo, so the SOURCE_STAMP & SOURCE_DATE variables mean that its taking a while for make runserver to happen. There's probably some fixes I can do for that, but waiting for some output from the content server seems the right thing to do anyway.

Comment 3

[Dan Mosedale (:dmosedale, :dmose)]

Comment on attachment 8597226 [details] [diff] [review]
Fix an intermittent error in Loop's functional tests (media start time should be uninitialized) and wait for the content server before starting tests. NPOTB

Review of attachment 8597226 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good; thanks for doing this!

r=dmose

Comment 4

[Pulsebot]

https://hg.mozilla.org/integration/fx-team/rev/eb203de9655f

Comment 5

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/eb203de9655f