Closed Bug 1078237 Opened 6 years ago Closed 3 years ago

Intermittent test_switch_frame.py TestSwitchFrame.test_should_be_able_to_carry_on_working_if_the_frame_is_deleted_from_under_us | AssertionError: 0 != 1

Categories

(Testing :: Marionette, defect)

x86
Windows 7
defect
Not set
normal

Tracking

(firefox34 disabled, firefox35 disabled, firefox36 disabled, firefox54 fixed)

RESOLVED FIXED
mozilla54
Tracking Status
firefox34 --- disabled
firefox35 --- disabled
firefox36 --- disabled
firefox54 --- fixed

People

(Reporter: cbook, Assigned: automatedtester)

References

()

Details

(Keywords: intermittent-failure, pi-marionette-intermittent)

Attachments

(1 file)

Windows 7 32-bit mozilla-inbound pgo test marionette

https://treeherder.mozilla.org/ui/logviewer.html#?job_id=2789071&repo=mozilla-inbound

03:11:11 ERROR - TEST-UNEXPECTED-FAIL | test_switch_frame.py TestSwitchFrame.test_should_be_able_to_carry_on_working_if_the_frame_is_deleted_from_under_us | AssertionError: 0 != 1
03:11:11 INFO - Traceback (most recent call last):
03:11:11 INFO - File "C:\slave\test\build\tests\marionette\marionette\marionette_test.py", line 267, in run
03:11:11 INFO - testMethod()
03:11:11 INFO - File "C:\slave\test\build\tests\marionette\tests\testing\marionette\client\marionette\tests\unit\test_switch_frame.py", line 83, in test_should_be_able_to_carry_on_working_if_the_frame_is_deleted_from_under_us
03:11:11 INFO - self.assertEqual(0, len(self.marionette.find_elements("id", "iframe1")))
03:11:11 INFO - TEST-INFO took 566ms
Duplicate of this bug: 1079480
Disabled on Windows. We'll see if it just moves the failures to another test like bug 1025040 did.
https://hg.mozilla.org/integration/mozilla-inbound/rev/532df34a8482
Whiteboard: [test disabled on Windows][leave open]
Dave, maybe this is of interest to you.
Flags: needinfo?(dave.hunt)
This suggests that the frame wasn't deleted, and perhaps it's a race condition. If so, instead of asserting that the frame is gone we should wait for it to be gone. Something that would help us to understand this failure (and likely many others) would be to enable the HTML logs. This would show us the screenshot and page source immediately following the failure. I've raised bug 1093045 for this.
Flags: needinfo?(dave.hunt)
If you examine the Python stack trace it looks more like this is an issue with findElement returning prematurely: http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/atolfsen@mozilla.com-1c5f2e266a7e/try-linux/try_ubuntu32_vm_test-marionette-bm01-tests1-linux32-build330.txt.gz

15:49:51    ERROR -  TEST-UNEXPECTED-ERROR | test_switch_frame.py TestSwitchFrame.test_should_be_able_to_carry_on_working_if_the_frame_is_deleted_from_under_us | TypeError: 'NoneType' object has no attribute '__getitem__'
15:49:51     INFO -  Traceback (most recent call last):
15:49:51     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/marionette/marionette_test.py", line 296, in run
15:49:51     INFO -      testMethod()
15:49:51     INFO -    File "/builds/slave/test/build/tests/marionette/tests/testing/marionette/client/marionette/tests/unit/test_switch_frame.py", line 93, in test_should_be_able_to_carry_on_working_if_the_frame_is_deleted_from_under_us
15:49:51     INFO -      self.marionette.find_element("id", "checkbox")
15:49:51     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/marionette_driver/marionette.py", line 1520, in find_element
15:49:51     INFO -      element = HTMLElement(self, response['ELEMENT'])

Basically it returns the wrong type.  It should either return an error or an element.
Flags: needinfo?(dave.hunt)
This is a similar report that fails on the same sort of Python stack trace but for modal dialogues: http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/atolfsen@mozilla.com-1c5f2e266a7e/try-linux/try_ubuntu32_vm_test-marionette-e10s-bm03-tests1-linux32-build348.txt.gz

If you examine the calls to find element they all return expected responses:

-> findElement {using: "id", value: "prompt-result"}
<- {value: {ELEMENT: "…"}, status: 0}

Is this a bug in the Python client?
That's interesting, thanks Andreas. I wonder if David has any thoughts on what might be causing this?
Flags: needinfo?(dave.hunt) → needinfo?(dburns)
a race condition you say... my guess its a suboptimal test on slow machines :/ I will leave the n-i and try debug later in the week
It's been a long time, and a lot has changed in Marionette. Is this still relevant?
The test is still disabled on WIndows, we should do a try run and see if its still broken. I agree it might not be relevant anymore
Flags: needinfo?(dburns)
Comment on attachment 8836091 [details]
Bug 1078237: Reenable frame switching test on Windows.

https://reviewboard.mozilla.org/r/111558/#review113322
Attachment #8836091 - Flags: review?(hskupin) → review+
Pushed by dburns@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/5c52a1658c98
Reenable frame switching test on Windows. r=whimboo
David, should we try to also get this uplifted to 52 in the case its passing there too?
Assignee: nobody → dburns
Status: NEW → RESOLVED
Closed: 3 years ago
Flags: needinfo?(dburns)
Resolution: --- → FIXED
Whiteboard: [test disabled on Windows][leave open]
Target Milestone: --- → mozilla54
tbqh... we can try but if it fails happy to have it in nightly and ride the trains
Flags: needinfo?(dburns)
You need to log in before you can comment on or make changes to this bug.