Intermittent test_switch_frame.py TestSwitchFrame.test_should_be_able_to_carry_on_working_if_the_frame_is_deleted_from_under_us | AssertionError: 0 != 1

RESOLVED FIXED in Firefox 54

Status

defect
RESOLVED FIXED
5 years ago
2 years ago

People

(Reporter: cbook, Assigned: automatedtester)

Tracking

({intermittent-failure, pi-marionette-intermittent})

unspecified
mozilla54
x86
Windows 7
Points:
---

Firefox Tracking Flags

(firefox34 disabled, firefox35 disabled, firefox36 disabled, firefox54 fixed)

Details

()

Attachments

(1 attachment)

Reporter

Description

5 years ago
Windows 7 32-bit mozilla-inbound pgo test marionette

https://treeherder.mozilla.org/ui/logviewer.html#?job_id=2789071&repo=mozilla-inbound

03:11:11 ERROR - TEST-UNEXPECTED-FAIL | test_switch_frame.py TestSwitchFrame.test_should_be_able_to_carry_on_working_if_the_frame_is_deleted_from_under_us | AssertionError: 0 != 1
03:11:11 INFO - Traceback (most recent call last):
03:11:11 INFO - File "C:\slave\test\build\tests\marionette\marionette\marionette_test.py", line 267, in run
03:11:11 INFO - testMethod()
03:11:11 INFO - File "C:\slave\test\build\tests\marionette\tests\testing\marionette\client\marionette\tests\unit\test_switch_frame.py", line 83, in test_should_be_able_to_carry_on_working_if_the_frame_is_deleted_from_under_us
03:11:11 INFO - self.assertEqual(0, len(self.marionette.find_elements("id", "iframe1")))
03:11:11 INFO - TEST-INFO took 566ms
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Duplicate of this bug: 1079480
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Disabled on Windows. We'll see if it just moves the failures to another test like bug 1025040 did.
https://hg.mozilla.org/integration/mozilla-inbound/rev/532df34a8482
Whiteboard: [test disabled on Windows][leave open]
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Dave, maybe this is of interest to you.
Flags: needinfo?(dave.hunt)
This suggests that the frame wasn't deleted, and perhaps it's a race condition. If so, instead of asserting that the frame is gone we should wait for it to be gone. Something that would help us to understand this failure (and likely many others) would be to enable the HTML logs. This would show us the screenshot and page source immediately following the failure. I've raised bug 1093045 for this.
Flags: needinfo?(dave.hunt)
Comment hidden (Legacy TBPL/Treeherder Robot)
If you examine the Python stack trace it looks more like this is an issue with findElement returning prematurely: http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/atolfsen@mozilla.com-1c5f2e266a7e/try-linux/try_ubuntu32_vm_test-marionette-bm01-tests1-linux32-build330.txt.gz

15:49:51    ERROR -  TEST-UNEXPECTED-ERROR | test_switch_frame.py TestSwitchFrame.test_should_be_able_to_carry_on_working_if_the_frame_is_deleted_from_under_us | TypeError: 'NoneType' object has no attribute '__getitem__'
15:49:51     INFO -  Traceback (most recent call last):
15:49:51     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/marionette/marionette_test.py", line 296, in run
15:49:51     INFO -      testMethod()
15:49:51     INFO -    File "/builds/slave/test/build/tests/marionette/tests/testing/marionette/client/marionette/tests/unit/test_switch_frame.py", line 93, in test_should_be_able_to_carry_on_working_if_the_frame_is_deleted_from_under_us
15:49:51     INFO -      self.marionette.find_element("id", "checkbox")
15:49:51     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/marionette_driver/marionette.py", line 1520, in find_element
15:49:51     INFO -      element = HTMLElement(self, response['ELEMENT'])

Basically it returns the wrong type.  It should either return an error or an element.
Flags: needinfo?(dave.hunt)
This is a similar report that fails on the same sort of Python stack trace but for modal dialogues: http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/atolfsen@mozilla.com-1c5f2e266a7e/try-linux/try_ubuntu32_vm_test-marionette-e10s-bm03-tests1-linux32-build348.txt.gz

If you examine the calls to find element they all return expected responses:

-> findElement {using: "id", value: "prompt-result"}
<- {value: {ELEMENT: "…"}, status: 0}

Is this a bug in the Python client?
That's interesting, thanks Andreas. I wonder if David has any thoughts on what might be causing this?
Flags: needinfo?(dave.hunt) → needinfo?(dburns)
Assignee

Comment 114

4 years ago
a race condition you say... my guess its a suboptimal test on slow machines :/ I will leave the n-i and try debug later in the week
It's been a long time, and a lot has changed in Marionette. Is this still relevant?
Assignee

Comment 116

2 years ago
The test is still disabled on WIndows, we should do a try run and see if its still broken. I agree it might not be relevant anymore
Flags: needinfo?(dburns)
Comment hidden (mozreview-request)

Comment 119

2 years ago
mozreview-review
Comment on attachment 8836091 [details]
Bug 1078237: Reenable frame switching test on Windows.

https://reviewboard.mozilla.org/r/111558/#review113322
Attachment #8836091 - Flags: review?(hskupin) → review+

Comment 120

2 years ago
Pushed by dburns@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/5c52a1658c98
Reenable frame switching test on Windows. r=whimboo
David, should we try to also get this uplifted to 52 in the case its passing there too?
Assignee: nobody → dburns
Status: NEW → RESOLVED
Closed: 2 years ago
Flags: needinfo?(dburns)
Resolution: --- → FIXED
Whiteboard: [test disabled on Windows][leave open]
Target Milestone: --- → mozilla54
Assignee

Comment 123

2 years ago
tbqh... we can try but if it fails happy to have it in nightly and ride the trains
Flags: needinfo?(dburns)
You need to log in before you can comment on or make changes to this bug.