936019 - Intermittent test_findelement.py test_findelement.TestElements.test_timeout | AssertionError: NoSuchElementException not raised

Status: RESOLVED FIXED

(Testing :: Marionette Client and Harness, defect)

Description

[Ryan VanderMeulen [:RyanVM]]

https://tbpl.mozilla.org/php/getParsedLog.php?id=30265345&tree=Mozilla-Inbound

Ubuntu VM 12.04 mozilla-inbound debug test marionette on 2013-11-07 03:26:27 PST for push e6e339fbf953
slave: tst-linux32-ec2-111

03:30:51     INFO -  TEST-START test_findelement.py
03:30:52     INFO -  test_by_name (test_findelement.TestElements) ... ok
03:30:53     INFO -  test_child_element (test_findelement.TestElements) ... ok
03:30:53     INFO -  test_child_elements (test_findelement.TestElements) ... ok
03:30:54     INFO -  test_class_name (test_findelement.TestElements) ... ok
03:30:54     INFO -  test_css_selector_scope_doesnt_start_at_rootnode (test_findelement.TestElements) ... ok
03:30:55     INFO -  test_finding_active_element_returns_element (test_findelement.TestElements) ... ok
03:30:56     INFO -  test_id (test_findelement.TestElements) ... ok
03:30:56     INFO -  test_link_text (test_findelement.TestElements) ... ok
03:30:58     INFO -  test_not_found (test_findelement.TestElements) ... ok
03:30:59     INFO -  test_partial_link_text (test_findelement.TestElements) ... ok
03:31:00     INFO -  test_selector (test_findelement.TestElements) ... ok
03:31:01     INFO -  test_tag_name (test_findelement.TestElements) ... ok
03:31:02     INFO -  test_timeout (test_findelement.TestElements) ... FAIL
03:31:04     INFO -  test_xpath (test_findelement.TestElements) ... ok
03:31:04     INFO -  ======================================================================
03:31:04     INFO -  FAIL: None
03:31:04     INFO -  ----------------------------------------------------------------------
03:31:04     INFO -  Traceback (most recent call last):
03:31:04     INFO -    File "/builds/slave/test/build/tests/marionette/marionette/marionette_test.py", line 143, in run
03:31:04     INFO -      testMethod()
03:31:04     INFO -    File "/builds/slave/test/build/tests/marionette/tests/testing/marionette/client/marionette/tests/unit/test_findelement.py", line 128, in test_timeout
03:31:04    ERROR -      self.assertRaises(NoSuchElementException, self.marionette.find_element, By.ID, "newDiv")
03:31:04    ERROR -  TEST-UNEXPECTED-FAIL | test_findelement.py test_findelement.TestElements.test_timeout | AssertionError: NoSuchElementException not raised
03:31:04     INFO -  ----------------------------------------------------------------------
03:31:04     INFO -  Ran 14 tests in 12.645s
03:31:04     INFO -  FAILED (failures=1)

Comment 1

[Legacy TBPL/Treeherder Robot]

RyanVM
https://tbpl.mozilla.org/php/getParsedLog.php?id=30278030&tree=Mozilla-Inbound
Ubuntu VM 12.04 mozilla-inbound debug test marionette on 2013-11-07 09:11:13
revision: 5c4ae2a331c4
slave: tst-linux32-ec2-115

self.assertRaises(NoSuchElementException, self.marionette.find_element, By.ID, "newDiv")
TEST-UNEXPECTED-FAIL | test_findelement.py test_findelement.TestElements.test_timeout | AssertionError: NoSuchElementException not raised
Return code: 2560
Marionette exited with return code 2560: harness failures
# TBPL FAILURE #

Comment 2

[Legacy TBPL/Treeherder Robot]

philor
https://tbpl.mozilla.org/php/getParsedLog.php?id=30278030&tree=Mozilla-Inbound
Ubuntu VM 12.04 mozilla-inbound debug test marionette on 2013-11-07 09:11:13
revision: 5c4ae2a331c4
slave: tst-linux32-ec2-115

self.assertRaises(NoSuchElementException, self.marionette.find_element, By.ID, "newDiv")
TEST-UNEXPECTED-FAIL | test_findelement.py test_findelement.TestElements.test_timeout | AssertionError: NoSuchElementException not raised
Return code: 2560
Marionette exited with return code 2560: harness failures
# TBPL FAILURE #

Comment 3

[Legacy TBPL/Treeherder Robot]

philor
https://tbpl.mozilla.org/php/getParsedLog.php?id=30299168&tree=Fx-Team
Ubuntu VM 12.04 fx-team debug test marionette on 2013-11-07 16:05:19
revision: 4ee49067a5c6
slave: tst-linux32-ec2-130

self.assertRaises(NoSuchElementException, self.marionette.find_element, By.ID, "newDiv")
TEST-UNEXPECTED-FAIL | test_findelement.py test_findelement.TestElements.test_timeout | AssertionError: NoSuchElementException not raised
Return code: 2560
Marionette exited with return code 2560: harness failures
# TBPL FAILURE #

Comment 4

[Malini Das [:mdas] - Away, not checking bugmail]

Attachment: trigger div creation

This intermittent was caused by a slight slow down between navigation time and element retrieval time. The page would use setTimeout to wait 1000ms before creating the div, and the test expects that upon successful navigation, the div isn't there. The problem is that there was some delay somewhere between the navigation return and the element retrieval call, so the div would eventually be created just in time for marionette to find it.

I've changed this so the test page will create the div after a new button is clicked. So now, we navigate to the page and then trigger the eventual creation of the element. I've changed test_timeout.py to reflect the change.

Comment 5

[Jonathan Griffin (:jgriffin)]

Comment on attachment 830378 [details] [diff] [review]
trigger div creation

Review of attachment 830378 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks.  Using 1000ms timing in a TBPL test is definitely risky.  :)

Comment 6

[Malini Das [:mdas] - Away, not checking bugmail]

landed in m-i:
https://hg.mozilla.org/integration/mozilla-inbound/rev/0d8f5ac28c5a

Comment 7

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/0d8f5ac28c5a

Comment 8

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-b2g26_v1_2/rev/bdf6406320c6

Comment 9

[BMO Automation]

Moving bug to Testing::Marionette Client and Harness component per bug 1815831.