Closed Bug 1003408 Opened 8 years ago Closed 7 years ago

Mozbase unit test mozrunner/tests/test_start.py fails on debug builds when run from test package

Categories

(Testing :: Mozbase, defect)

x86_64
Linux
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED
mozilla38

People

(Reporter: dminor, Assigned: dminor)

References

Details

Attachments

(1 file)

Relevant portion of log:
13:16:46     INFO -  ======================================================================
13:16:46     INFO -  FAIL: test_start_with_outputTimeout (test_start.MozrunnerStartTestCase)
13:16:46     INFO -  Start the process and set a timeout
13:16:46     INFO -  ----------------------------------------------------------------------
13:16:46     INFO -  Traceback (most recent call last):
13:16:46     INFO -    File "/builds/slave/test/build/tests/mozbase/mozrunner/tests/test_start.py", line 47, in test_start_with_outputTimeout
13:16:46     INFO -      self.assertFalse(self.runner.is_running())
13:16:46     INFO -  AssertionError: True is not false
Unfortunately, this doesn't reproduce on my local machine so it is likely something to do with the test machine environment.
It looks like we just need a longer timeout here [1]. Increasing from 5 to 15 seconds seems to give a consistent pass on my linux64 loaner. I'll give it a shot on cedar to see if that value works for the other test platforms.

I think the issue is that the debug builds are chattier, and so the browser needs to run for longer to reach a point of two seconds without output.

[1] http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/mozrunner/tests/test_start.py#45
Assignee: nobody → dminor
Status: NEW → ASSIGNED
It looks like 15 seconds is sufficient on all platforms other than Windows 8, which seems to have its own set of mozrunner related problems:

https://hg.mozilla.org/projects/cedar/rev/6796d15eba33
Attached patch Increase timeoutSplinter Review
Attachment #8553837 - Flags: review?(wlachance)
Comment on attachment 8553837 [details] [diff] [review]
Increase timeout

Review of attachment 8553837 [details] [diff] [review]:
-----------------------------------------------------------------

I am suspicious of any test which requires an arbitrary sleep in order to work, but I guess this can't be any worse than what we have now.
Attachment #8553837 - Flags: review?(wlachance) → review+
Thanks, I agree these tests should be rewritten :)

https://hg.mozilla.org/integration/mozilla-inbound/rev/40a5dd69da69
https://hg.mozilla.org/mozilla-central/rev/40a5dd69da69
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla38
You need to log in before you can comment on or make changes to this bug.