Intermittent test_switch_frame.py TestSwitchFrame.test_should_be_able_to_carry_on_working_if_the_frame_is_deleted_from_under_us | AssertionError: 0 != 1

RESOLVED FIXED in Firefox 54

Status

Testing
Marionette
RESOLVED FIXED
3 years ago
11 months ago

People

(Reporter: Tomcat, Assigned: automatedtester)

Tracking

({ateam-marionette-intermittent, intermittent-failure})

unspecified
mozilla54
x86
Windows 7
ateam-marionette-intermittent, intermittent-failure
Points:
---

Firefox Tracking Flags

(firefox34 disabled, firefox35 disabled, firefox36 disabled, firefox54 fixed)

Details

(URL)

MozReview Requests

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(1 attachment)

(Reporter)

Description

3 years ago
Windows 7 32-bit mozilla-inbound pgo test marionette

https://treeherder.mozilla.org/ui/logviewer.html#?job_id=2789071&repo=mozilla-inbound

03:11:11 ERROR - TEST-UNEXPECTED-FAIL | test_switch_frame.py TestSwitchFrame.test_should_be_able_to_carry_on_working_if_the_frame_is_deleted_from_under_us | AssertionError: 0 != 1
03:11:11 INFO - Traceback (most recent call last):
03:11:11 INFO - File "C:\slave\test\build\tests\marionette\marionette\marionette_test.py", line 267, in run
03:11:11 INFO - testMethod()
03:11:11 INFO - File "C:\slave\test\build\tests\marionette\tests\testing\marionette\client\marionette\tests\unit\test_switch_frame.py", line 83, in test_should_be_able_to_carry_on_working_if_the_frame_is_deleted_from_under_us
03:11:11 INFO - self.assertEqual(0, len(self.marionette.find_elements("id", "iframe1")))
03:11:11 INFO - TEST-INFO took 566ms
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Duplicate of this bug: 1079480
Keywords: ateam-marionette-intermittent
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Disabled on Windows. We'll see if it just moves the failures to another test like bug 1025040 did.
https://hg.mozilla.org/integration/mozilla-inbound/rev/532df34a8482
Whiteboard: [test disabled on Windows][leave open]
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
https://hg.mozilla.org/releases/mozilla-aurora/rev/18ca813ed527
https://hg.mozilla.org/releases/mozilla-beta/rev/76dcced7d838
status-firefox34: --- → disabled
status-firefox35: --- → disabled
status-firefox36: --- → disabled
Comment hidden (Treeherder Robot)
Dave, maybe this is of interest to you.
Flags: needinfo?(dave.hunt)
This suggests that the frame wasn't deleted, and perhaps it's a race condition. If so, instead of asserting that the frame is gone we should wait for it to be gone. Something that would help us to understand this failure (and likely many others) would be to enable the HTML logs. This would show us the screenshot and page source immediately following the failure. I've raised bug 1093045 for this.
Flags: needinfo?(dave.hunt)
Comment hidden (Treeherder Robot)
If you examine the Python stack trace it looks more like this is an issue with findElement returning prematurely: http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/atolfsen@mozilla.com-1c5f2e266a7e/try-linux/try_ubuntu32_vm_test-marionette-bm01-tests1-linux32-build330.txt.gz

15:49:51    ERROR -  TEST-UNEXPECTED-ERROR | test_switch_frame.py TestSwitchFrame.test_should_be_able_to_carry_on_working_if_the_frame_is_deleted_from_under_us | TypeError: 'NoneType' object has no attribute '__getitem__'
15:49:51     INFO -  Traceback (most recent call last):
15:49:51     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/marionette/marionette_test.py", line 296, in run
15:49:51     INFO -      testMethod()
15:49:51     INFO -    File "/builds/slave/test/build/tests/marionette/tests/testing/marionette/client/marionette/tests/unit/test_switch_frame.py", line 93, in test_should_be_able_to_carry_on_working_if_the_frame_is_deleted_from_under_us
15:49:51     INFO -      self.marionette.find_element("id", "checkbox")
15:49:51     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/marionette_driver/marionette.py", line 1520, in find_element
15:49:51     INFO -      element = HTMLElement(self, response['ELEMENT'])

Basically it returns the wrong type.  It should either return an error or an element.
Flags: needinfo?(dave.hunt)
This is a similar report that fails on the same sort of Python stack trace but for modal dialogues: http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/atolfsen@mozilla.com-1c5f2e266a7e/try-linux/try_ubuntu32_vm_test-marionette-e10s-bm03-tests1-linux32-build348.txt.gz

If you examine the calls to find element they all return expected responses:

-> findElement {using: "id", value: "prompt-result"}
<- {value: {ELEMENT: "…"}, status: 0}

Is this a bug in the Python client?
That's interesting, thanks Andreas. I wonder if David has any thoughts on what might be causing this?
Flags: needinfo?(dave.hunt) → needinfo?(dburns)
(Assignee)

Comment 114

3 years ago
a race condition you say... my guess its a suboptimal test on slow machines :/ I will leave the n-i and try debug later in the week
It's been a long time, and a lot has changed in Marionette. Is this still relevant?
(Assignee)

Comment 116

a year ago
The test is still disabled on WIndows, we should do a try run and see if its still broken. I agree it might not be relevant anymore
Flags: needinfo?(dburns)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)

Comment 119

11 months ago
mozreview-review
Comment on attachment 8836091 [details]
Bug 1078237: Reenable frame switching test on Windows.

https://reviewboard.mozilla.org/r/111558/#review113322
Attachment #8836091 - Flags: review?(hskupin) → review+

Comment 120

11 months ago
Pushed by dburns@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/5c52a1658c98
Reenable frame switching test on Windows. r=whimboo
David, should we try to also get this uplifted to 52 in the case its passing there too?
Assignee: nobody → dburns
Status: NEW → RESOLVED
Last Resolved: 11 months ago
status-firefox54: --- → fixed
Flags: needinfo?(dburns)
Resolution: --- → FIXED
Whiteboard: [test disabled on Windows][leave open]
Target Milestone: --- → mozilla54
(Assignee)

Comment 123

11 months ago
tbqh... we can try but if it fails happy to have it in nightly and ride the trains
Flags: needinfo?(dburns)
You need to log in before you can comment on or make changes to this bug.