1003408 - Mozbase unit test mozrunner/tests/test_start.py fails on debug builds when run from test package

Status: RESOLVED FIXED

(Testing :: Mozbase, defect)

Description

[Dan Minor [:dminor]]

Relevant portion of log:
13:16:46     INFO -  ======================================================================
13:16:46     INFO -  FAIL: test_start_with_outputTimeout (test_start.MozrunnerStartTestCase)
13:16:46     INFO -  Start the process and set a timeout
13:16:46     INFO -  ----------------------------------------------------------------------
13:16:46     INFO -  Traceback (most recent call last):
13:16:46     INFO -    File "/builds/slave/test/build/tests/mozbase/mozrunner/tests/test_start.py", line 47, in test_start_with_outputTimeout
13:16:46     INFO -      self.assertFalse(self.runner.is_running())
13:16:46     INFO -  AssertionError: True is not false

Comment 1

[Dan Minor [:dminor]]

Unfortunately, this doesn't reproduce on my local machine so it is likely something to do with the test machine environment.

Comment 2

[Dan Minor [:dminor]]

It looks like we just need a longer timeout here [1]. Increasing from 5 to 15 seconds seems to give a consistent pass on my linux64 loaner. I'll give it a shot on cedar to see if that value works for the other test platforms.

I think the issue is that the debug builds are chattier, and so the browser needs to run for longer to reach a point of two seconds without output.

[1] http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/mozrunner/tests/test_start.py#45

Comment 3

[Dan Minor [:dminor]]

It looks like 15 seconds is sufficient on all platforms other than Windows 8, which seems to have its own set of mozrunner related problems:

https://hg.mozilla.org/projects/cedar/rev/6796d15eba33

Comment 4

[Dan Minor [:dminor]]

Attachment: Increase timeout

Comment 5

[William Lachance (:wlach)]

Comment on attachment 8553837 [details] [diff] [review]
Increase timeout

Review of attachment 8553837 [details] [diff] [review]:
-----------------------------------------------------------------

I am suspicious of any test which requires an arbitrary sleep in order to work, but I guess this can't be any worse than what we have now.

Comment 6

[Dan Minor [:dminor]]

Thanks, I agree these tests should be rewritten :)

https://hg.mozilla.org/integration/mozilla-inbound/rev/40a5dd69da69

Comment 7

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/40a5dd69da69