Closed Bug 838607 Opened 12 years ago Closed 11 years ago

Intermittent B2G desktop Marionette test_click.py TestClick.testClickingALinkMadeUpOfNumbersIsHandledCorrectly | NoSuchElementException: Unable to locate element: {333333,username}

Categories

(Testing :: Marionette Client and Harness, defect)

Product:
Testing
Component:
Marionette Client and Harness
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME
Milestone:
mozilla24

People

(Reporter: miker, Unassigned)

References

Details

(Keywords: intermittent-failure)

Attachments

(3 files)

Explicit wait for element in tests
2.19 KB, patch
mdas
: review+
Details | Diff | Splinter Review
Disable firstrun page during Marionette tests,
855 bytes, patch
mdas
: review+
Details | Diff | Splinter Review
adding some prefs to try stabilise
1.24 KB, patch
Details | Diff | Splinter Review
05:38:54 INFO - ERROR: testClickingALinkMadeUpOfNumbersIsHandledCorrectly (test_click.TestClick) 05:38:54 INFO - ---------------------------------------------------------------------- 05:38:54 ERROR - Traceback (most recent call last): 05:38:54 INFO - File "/Users/cltbld/talos-slave/test/build/tests/marionette/tests/testing/marionette/client/marionette/tests/unit/test_click.py", line 19, in testClickingALinkMadeUpOfNumbersIsHandledCorrectly 05:38:54 INFO - self.marionette.find_element("link text", "333333").click() 05:38:54 INFO - File "/Users/cltbld/talos-slave/test/build/tests/marionette/marionette/marionette.py", line 511, in find_element 05:38:54 INFO - response = self._send_message('findElement', 'value', **kwargs) 05:38:54 INFO - File "/Users/cltbld/talos-slave/test/build/tests/marionette/marionette/marionette.py", line 249, in _send_message 05:38:54 INFO - self._handle_error(response) 05:38:54 INFO - File "/Users/cltbld/talos-slave/test/build/tests/marionette/marionette/marionette.py", line 270, in _handle_error 05:38:54 ERROR - raise NoSuchElementException(message=message, status=status, stacktrace=stacktrace) 05:38:54 ERROR - TEST-UNEXPECTED-FAIL | test_click.py TestClick.testClickingALinkMadeUpOfNumbersIsHandledCorrectly | NoSuchElementException: Unable to locate element: 333333 https://tbpl.mozilla.org/php/getParsedLog.php?id=19490599&tree=Fx-Team
Product: Core → Boot2Gecko
Summary: TEST-UNEXPECTED-FAIL | test_click.py TestClick.testClickingALinkMadeUpOfNumbersIsHandledCorrectly | NoSuchElementException: Unable to locate element: 333333 → Intermittent B2G desktop Marionette test_click.py TestClick.testClickingALinkMadeUpOfNumbersIsHandledCorrectly | NoSuchElementException: Unable to locate element: {333333,username}
This seems very likely a regression from https://hg.mozilla.org/integration/mozilla-inbound/rev/56fe7e26fa47 (Bug 847655). David, can you take a look?
Component: General → Marionette
Product: Boot2Gecko → Testing
Assignee: nobody → dburns
This showed up on b2g18 now that bug 848489 has landed there.
Blocks: 848489
Comment on attachment 734641 [details] [diff] [review] Explicit wait for element in tests Review of attachment 734641 [details] [diff] [review]: ----------------------------------------------------------------- r+d so as to not change the search timeout for the rest of the test
Attachment #734641 - Flags: review?(mdas) → review+
This must be page load issue after a click()
Seems like it, which is even more puzzling.
Might be worth adding some debugging code to this test, that would output the current page url in case of failure.
(In reply to TinderboxPushlog Robot from comment #91) > RyanVM > https://tbpl.mozilla.org/php/getParsedLog.php?id=21551241&tree=Mozilla- > Inbound > Ubuntu VM 12.04 x64 mozilla-inbound debug test marionette on 2013-04-08 > 04:05:33 > slave: tst-linux64-ec2-391 > > 11:07:53 INFO - TEST-UNEXPECTED-FAIL | test_click.py > TestClick.testClickingALinkMadeUpOfNumbersIsHandledCorrectly | > AssertionError: Element id=username not found after 30 seconds > 11:10:36 ERROR - Return code: 10 This is a post-comment 84 push :(
Whiteboard: [leave open]
This bug is getting out of hand; I'm going to disable this test until we have a chance to find a solution for this problem.
Disabled in https://hg.mozilla.org/integration/mozilla-inbound/rev/1625ba632d1b. RyanVM noted that this failure only seems to occur on Ubuntu64 debug, which makes me think it's likely to be a timing issue.
(In reply to Jonathan Griffin (:jgriffin) from comment #147) > Disabled in > https://hg.mozilla.org/integration/mozilla-inbound/rev/1625ba632d1b. > > RyanVM noted that this failure only seems to occur on Ubuntu64 debug, which > makes me think it's likely to be a timing issue. I have been running it in my Ubuntu64 machine today, all day, and I can not replicate the issue. I dont think its a timing issue because we are waiting 30 seconds for it to happen. My only guess is that its a scrolling issue and the element is on the cusp of the viewport meaning the click on 333333 isnt actually happening But to replicate the issue is proving nearly impossible.
So there are basically two ways to handle this...add a lot more debugging output to the test so we can see if we can figure out what's going on, and/or ask rel-eng to loan you one of the problematic slaves we've been seeing the issue with.
I will try run this over the entire weekend and see what I can find. If not, will raise a bug for the relevant machine and try replicate it there
Whiteboard: [leave open] → [test disabled][leave open]
Disable another test_click test for the same problem: https://hg.mozilla.org/integration/mozilla-inbound/rev/82f8115af316
Depends on: 861810
I can't reproduce this locally either, on my ubuntu64 vm, using either a local build or the TBPL build.
AutomatedTester: I have an ubuntu64 slave checked out for a different purpose, and i tried about 20 times to reproduce this problem, both by invoking Marionette directly, and by running it through the mozharness script that is used in TBPL. I was unable to reproduce it. Looking at TBPL, it seems that this failure occurs more than 10% of the time, so I doubt that more attempts will be successful. I think we may want to look at the original click atom and figure out what it's doing that we're not.
Any more news with this? Bug 861810 which is dependent on this, is appearing in our top oranges list at the moment.
Assignee: dburns → jgriffin
The proximate cause of this is an apparent bug in getMostRecentWindow() such that it sometimes returns the window of the tab loading the first run content, even when the "about:blank" tab that Marionette has created is visually on top. The easiest "fix" to this is to disable the firstrun content, which is what this try run does: https://tbpl.mozilla.org/?tree=Try&rev=23ca8b06fb0d But, it shows that always relying on getMostRecentWindow() is risky; it might be better to pin Marionette to the tab we want (the one we open for a session), but then tests that deliberately open a new tab and want to operate in the new tab would need something like a switch_to_tab() command to explicitly switch to the new tab, and I don't think that jives well with the way WebDriver currently works. Meanwhile, I'll try to create a reproducible test for the getMostRecentWindow() problem and file it as a separate bug.
Try was green. I'm pretty sure this will fix the problem, even though there is a more fundamental issue we'll still need to fix one way or another. This should make us greener, though, and seems like something we should do regardless.
Attachment #746052 - Flags: review?(mdas)
(In reply to Jonathan Griffin (:jgriffin) from comment #161) > But, it shows that always relying on getMostRecentWindow() is risky; it > might be better to pin Marionette to the tab we want (the one we open for a > session), but then tests that deliberately open a new tab and want to > operate in the new tab would need something like a switch_to_tab() command > to explicitly switch to the new tab, and I don't think that jives well with > the way WebDriver currently works. > If I understand what you're saying correctly, then that's fine. That's what switchToWindow is supposed to do, and if you want to operate in a new tab, you have to call that function. We should pin marionette to the new tab since that's what we're implicitly trying to do, since we use 'getMostRecentWindow' on the assumption that it will point to the newly opened tab.
Attachment #746052 - Flags: review?(mdas) → review+
Ah, cool. Then I'll land this fix to clean up the oranges, and file a separate bug to force Marionette into its own tab.
If we stop seeing failures in bug 861810, I'll re-enable the test_click tests that were failing and close this bug.
Oddly it doesn't look like this helped. There have been a couple of instances of bug 861810 after this landed. :(
I have added a few more that, at least in the Selenium world, have stabilized things. Try Push - https://tbpl.mozilla.org/?tree=Try&rev=c87103085158
I have a patch which might fix this, and if not, it spews a bunch of debug things that will help me understand what's going on. Running on try: https://tbpl.mozilla.org/?tree=Try&rev=e6eb70a4705f
Re-enabled test_click.py since the underlying problem in bug 869536 has been fixed: https://hg.mozilla.org/integration/mozilla-inbound/rev/905e081060c5
Whiteboard: [test disabled][leave open]
https://hg.mozilla.org/mozilla-central/rev/905e081060c5
Status: NEW → RESOLVED
Closed: 12 years ago
Flags: in-testsuite+
Resolution: --- → FIXED
Target Milestone: --- → mozilla24
Depends on: 876199
ruh roh
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Assignee: jgriffin → nobody
This seems to be fixed, with no occurrences in the last 2 months.
Status: REOPENED → RESOLVED
Closed: 12 years ago → 11 years ago
Resolution: --- → WORKSFORME
Product: Testing → Remote Protocol
Moving bug to Testing::Marionette Client and Harness component per bug 1815831.
Component: Marionette → Marionette Client and Harness
Product: Remote Protocol → Testing
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: