Closed
Bug 777935
Opened 10 years ago
Closed 10 years ago
Marionette tests can be flaky because previous tests dont clean up
Categories
(Testing :: Marionette, defect)
Tracking
(Not tracked)
RESOLVED
FIXED
mozilla17
People
(Reporter: automatedtester, Assigned: automatedtester)
References
Details
Attachments
(1 file)
2.34 KB,
patch
|
jgriffin
:
review+
|
Details | Diff | Splinter Review |
I have noticed that when running the entire test suite we are getting some flakiness. E.g. Window Management test counts the amount of open windows when asserting and had 1 more than expected which appears to be from the test that ran before We should probably restart the browser between test modules
Comment 1•10 years ago
|
||
Is there actually 1 extra window open, or are we holding a stale reference?
Comment 2•10 years ago
|
||
Yes, I see the same problem; basically Marionette gets confused when switching windows sometimes, and switches to the wrong one. It may be a timing issue. Definitely needs to be fixed.
Blocks: 770769
Comment 3•10 years ago
|
||
On my machine, this problem is easily reproduced if Firefox is in the foreground while the tests run. If it's in the background, obscured by a Terminal window for example, it does not reproduce. I'm guessing this is a timing issue.
Comment 4•10 years ago
|
||
So it looks like this is what's happening: - some tests use window.open in an execute_script to open a new window to run a test, e.g., http://mxr.mozilla.org/mozilla-central/source/testing/marionette/client/marionette/tests/unit/test_findelement.py#131 - the test then continues immediately, without waiting for the window to load - sometimes, this means that the test will be executed before the window is really open - then, when the test teardown happens (e.g., http://mxr.mozilla.org/mozilla-central/source/testing/marionette/client/marionette/tests/unit/test_findelement.py#134), it will close the wrong window
Assignee | ||
Updated•10 years ago
|
Assignee: nobody → dburns
Assignee | ||
Comment 5•10 years ago
|
||
I tracked it down to a test file that opens a window and doesnt clean it up. I have added a tearDown to close all windows but the main one.
Attachment #646576 -
Flags: review?(jgriffin)
Comment 6•10 years ago
|
||
Comment on attachment 646576 [details] [diff] [review] add test cleanup Review of attachment 646576 [details] [diff] [review]: ----------------------------------------------------------------- Thanks!
Attachment #646576 -
Flags: review?(jgriffin) → review+
Assignee | ||
Comment 7•10 years ago
|
||
landed in http://hg.mozilla.org/integration/mozilla-inbound/rev/52f5b60aa1d6
Comment 8•10 years ago
|
||
https://hg.mozilla.org/mozilla-central/rev/52f5b60aa1d6
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla17
You need to log in
before you can comment on or make changes to this bug.
Description
•