Closed Bug 1064299 Opened 9 years ago Closed 9 years ago

Investigate intermittent keyboard failure when typing text in search bar

Categories

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

ARM
Gonk (Firefox OS)
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED WORKSFORME

People

(Reporter: viorela, Assigned: martijn.martijn)

References

()

Details

Attachments

(1 file)

Test test_cost_control_data_alert_mobile.py has failed in the latest v2.1 build. 
The failure is really intermittent, I was able to reproduce it 1 out of 20 times while running the automated test locally. 
The issue is not reproducible manually. 
There is no screenshot available, but from the stacktrace we can notice that the test failed to type the url in the input field. 
We should probably add a wait before typing in the input field: https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/apps/homescreen/regions/search_panel.py#L28.

Jenkins report: http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame.mozilla-aurora.ui.functional.smoke/36/HTML_Report/

Traceback (most recent call last):
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.3-py2.7.egg/marionette/marionette_test.py", line 171, in run
testMethod()
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/tests/functional/cost_control/test_cost_control_data_alert_mobile.py", line 45, in test_cost_control_data_alert_mobile
browser = search.go_to_url('http://www.mozilla.org/')
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/apps/search/app.py", line 26, in go_to_url
return search_panel.go_to_url(url)
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/apps/homescreen/regions/search_panel.py", line 28, in go_to_url
self.keyboard.send(url)
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/apps/keyboard/app.py", line 237, in send
self._switch_to_correct_layout(val)
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/apps/keyboard/app.py", line 125, in _switch_to_correct_layout
self.wait_for_element_displayed(*self._key_locator(self._alpha_key))
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 44, in wait_for_element_displayed
lambda m: m.find_element(by, locator).is_displayed())
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.3-py2.7.egg/marionette/wait.py", line 143, in until
cause=last_exc)
TimeoutException: Traceback (most recent call last):
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.3-py2.7.egg/marionette/wait.py", line 122, in until
rv = condition(self.marionette)
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 44, in <lambda>
lambda m: m.find_element(by, locator).is_displayed())
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.3-py2.7.egg/marionette/marionette.py", line 1278, in find_element
response = self._send_message('findElement', 'value', **kwargs)
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.3-py2.7.egg/marionette/decorators.py", line 35, in _
return func(*args, **kwargs)
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.3-py2.7.egg/marionette/marionette.py", line 638, in _send_message
self._handle_error(response)
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.3-py2.7.egg/marionette/marionette.py", line 672, in _handle_error
raise errors.NoSuchElementException(message=message, status=status, stacktrace=stacktrace)
TimeoutException: Timed out after 20.2 seconds, caused by <class 'marionette.errors.NoSuchElementException'>


Device firmware (date) 	31 Aug 2014 13:31:23
Device firmware (incremental) 	eng.cltbld.20140831.163113
Device firmware (release) 	4.3
Device identifier 	flame
Gaia date 	06 Sep 2014 11:06:20
Gaia revision 	a8e4d26555e5
Gecko build 	20140908000204
Gecko revision 	746f24f9d21d
Gecko version 	34.0a2
This issue is still reproduced on latest aurora:
Gaia      944e5b4582c4efa1b67cd33245dbb8f6aa25d09f
Gecko     https://hg.mozilla.org/releases/mozilla-aurora/rev/7546fedad918
BuildID   20140915000203
Version   34.0a2
ro.build.date   Fri Jun 27 15:57:58 CST 2014
ro.bootloader   L1TC00011230
ro.build.version.incremental   110

The test is failing with different errors.
http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-319.mozilla-aurora.ui.functional.smoke/57/HTML_Report/
http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame.ui.adhoc/140/console
Blocks: 1068016
This test has failed in latest master build with a similar issue: http://jenkins1.qa.scl3.mozilla.com/job/flame-319.mozilla-central.ui.functional.smoke/153/HTML_Report/

Device firmware (date) 	31 Aug 2014 13:31:23
Device firmware (incremental) 	eng.cltbld.20140831.163113
Device firmware (release) 	4.3
Device identifier 	flame
Gaia date 	15 Sep 2014 10:39:43
Gaia revision 	e2d70bee03b5
Gecko build 	20140916040203
Gecko revision 	3b7921328fc1
Gecko version 	35.0a1

It might be a keyboard issue, as the test is failing when typing the url in the browser search bar. 
This requires further investigation.
Summary: [v2.1] Investigate intermittent failure in test_cost_control_data_alert_mobile.py → Investigate intermittent failure in test_cost_control_data_alert_mobile.py
I ran this automated test locally several times on master and it failed intermittently. 
Repro rate would be 2 out of 10.
The test fails while typing the url in the rocketbar input. It types 'htt', then the input field loses focus and the keyboard disappears. I'm not sure what causes the input field to loose focus.

The test is failing when executing https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/apps/homescreen/regions/search_panel.py#L28. We might need to investigate the send method, even though modifying it is risky, as it is used in several other places. 

I was able to take a video of the test failure. 

Device firmware (date) 	31 Aug 2014 13:31:23
Device firmware (incremental) 	eng.cltbld.20140831.163113
Device firmware (release) 	4.3
Device identifier 	flame
Gaia date 	15 Sep 2014 10:39:43
Gaia revision 	e2d70bee03b5
Gecko build 	20140916160204
Gecko revision 	543f57ab6a44
Gecko version 	35.0a1
Attached video cost_control.webm
We have managed to reproduce something similar in test_everythingme_launch_link.TestEverythingMeLaunchLink

In this test the send_keys is not writing the full name of the search term. We want to type "facebook" and the resulted text is "faceboo" because of this the test is failing.


There are multiple issues lately with our keyboard atom and the "search panel". From what it looks like for every typed letter there are multiple search events triggered. I think because of this we see the keyboard app looses focus or types incorrect terms.

See also:
Bug 1068016 - [v2.2] go_to_url method times out on longer text

http://jenkins1.qa.scl3.mozilla.com/job/flame-319.mozilla-central.ui.functional.smoke/158/HTML_Report/

Traceback (most recent call last):
File "/var/jenkins/1/workspace/flame-319.mozilla-central.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.4-py2.7.egg/marionette/marionette_test.py", line 264, in run
testMethod()
File "/var/jenkins/1/workspace/flame-319.mozilla-central.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/tests/functional/everythingme/test_everythingme_launch_link.py", line 29, in test_launch_everything_me_link
self.assertIn(search_string.lower(), self.marionette.title.lower())
AssertionError: 'facebook' not found in u'faceboo - google search'
Summary: Investigate intermittent failure in test_cost_control_data_alert_mobile.py → Investigate intermittent keyboard failure when typing text in search bar
I took a look over the logcat of this fail and we do emit the tap command for the last "k" element but we don't see it in the UI. 

I can't reproduce this issue locally
QA Whiteboard: [fxosqa-auto-backlog+]
Is this still an issue?
Assignee: nobody → martijn.martijn
QA Whiteboard: [fxosqa-auto-backlog+] → fxosqa-auto-s5
test_cost_control_data_alert_mobile.py seems to have been disabled in the latests 2.1 runs: http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.b2g-inbound.ui.functional.smoke/HTML_Report/
Because of bug 1061698, apparently.

test_launch_everything_me_link seems to be stable for a long time already: http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.b2g-inbound.ui.functional.smoke/236/testReport/junit/test_everythingme_launch_link/TestEverythingMeLaunchLink/test_launch_everything_me_link/history/?start=200
Although, I'm not sure that link reflects the runs in 2.1.
At this point, we should probably be considering whether it's working for 2.2 more than anything. We should probably be discussing soon how much more we want to maintain the 2.1 suite.
Florin, do you think we can mark this bug resolved->worksforme? See also comment 9.
Flags: needinfo?(florin.strugariu)
Yeah seems OK

These tests are peaty stable now
Status: NEW → RESOLVED
Closed: 9 years ago
Flags: needinfo?(florin.strugariu)
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.