1064299 - Investigate intermittent keyboard failure when typing text in search bar

Status: RESOLVED WORKSFORME

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Description

[Viorela Ioia [:viorela]]

Test test_cost_control_data_alert_mobile.py has failed in the latest v2.1 build. 
The failure is really intermittent, I was able to reproduce it 1 out of 20 times while running the automated test locally. 
The issue is not reproducible manually. 
There is no screenshot available, but from the stacktrace we can notice that the test failed to type the url in the input field. 
We should probably add a wait before typing in the input field: https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/apps/homescreen/regions/search_panel.py#L28.

Jenkins report: http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame.mozilla-aurora.ui.functional.smoke/36/HTML_Report/

Traceback (most recent call last):
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.3-py2.7.egg/marionette/marionette_test.py", line 171, in run
testMethod()
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/tests/functional/cost_control/test_cost_control_data_alert_mobile.py", line 45, in test_cost_control_data_alert_mobile
browser = search.go_to_url('http://www.mozilla.org/')
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/apps/search/app.py", line 26, in go_to_url
return search_panel.go_to_url(url)
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/apps/homescreen/regions/search_panel.py", line 28, in go_to_url
self.keyboard.send(url)
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/apps/keyboard/app.py", line 237, in send
self._switch_to_correct_layout(val)
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/apps/keyboard/app.py", line 125, in _switch_to_correct_layout
self.wait_for_element_displayed(*self._key_locator(self._alpha_key))
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 44, in wait_for_element_displayed
lambda m: m.find_element(by, locator).is_displayed())
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.3-py2.7.egg/marionette/wait.py", line 143, in until
cause=last_exc)
TimeoutException: Traceback (most recent call last):
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.3-py2.7.egg/marionette/wait.py", line 122, in until
rv = condition(self.marionette)
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 44, in <lambda>
lambda m: m.find_element(by, locator).is_displayed())
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.3-py2.7.egg/marionette/marionette.py", line 1278, in find_element
response = self._send_message('findElement', 'value', **kwargs)
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.3-py2.7.egg/marionette/decorators.py", line 35, in _
return func(*args, **kwargs)
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.3-py2.7.egg/marionette/marionette.py", line 638, in _send_message
self._handle_error(response)
File "/var/jenkins/2/workspace/flame.mozilla-aurora.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.3-py2.7.egg/marionette/marionette.py", line 672, in _handle_error
raise errors.NoSuchElementException(message=message, status=status, stacktrace=stacktrace)
TimeoutException: Timed out after 20.2 seconds, caused by <class 'marionette.errors.NoSuchElementException'>


Device firmware (date) 	31 Aug 2014 13:31:23
Device firmware (incremental) 	eng.cltbld.20140831.163113
Device firmware (release) 	4.3
Device identifier 	flame
Gaia date 	06 Sep 2014 11:06:20
Gaia revision 	a8e4d26555e5
Gecko build 	20140908000204
Gecko revision 	746f24f9d21d
Gecko version 	34.0a2

Comment 1

[Robert Chira [:RobertC]]

This issue is still reproduced on latest aurora:
Gaia      944e5b4582c4efa1b67cd33245dbb8f6aa25d09f
Gecko     https://hg.mozilla.org/releases/mozilla-aurora/rev/7546fedad918
BuildID   20140915000203
Version   34.0a2
ro.build.date   Fri Jun 27 15:57:58 CST 2014
ro.bootloader   L1TC00011230
ro.build.version.incremental   110

The test is failing with different errors.
http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-319.mozilla-aurora.ui.functional.smoke/57/HTML_Report/
http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame.ui.adhoc/140/console

Comment 2

[Viorela Ioia [:viorela]]

This test has failed in latest master build with a similar issue: http://jenkins1.qa.scl3.mozilla.com/job/flame-319.mozilla-central.ui.functional.smoke/153/HTML_Report/

Device firmware (date) 	31 Aug 2014 13:31:23
Device firmware (incremental) 	eng.cltbld.20140831.163113
Device firmware (release) 	4.3
Device identifier 	flame
Gaia date 	15 Sep 2014 10:39:43
Gaia revision 	e2d70bee03b5
Gecko build 	20140916040203
Gecko revision 	3b7921328fc1
Gecko version 	35.0a1

It might be a keyboard issue, as the test is failing when typing the url in the browser search bar. 
This requires further investigation.

Comment 3

[Viorela Ioia [:viorela]]

I ran this automated test locally several times on master and it failed intermittently. 
Repro rate would be 2 out of 10.
The test fails while typing the url in the rocketbar input. It types 'htt', then the input field loses focus and the keyboard disappears. I'm not sure what causes the input field to loose focus.

The test is failing when executing https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/apps/homescreen/regions/search_panel.py#L28. We might need to investigate the send method, even though modifying it is risky, as it is used in several other places. 

I was able to take a video of the test failure. 

Device firmware (date) 	31 Aug 2014 13:31:23
Device firmware (incremental) 	eng.cltbld.20140831.163113
Device firmware (release) 	4.3
Device identifier 	flame
Gaia date 	15 Sep 2014 10:39:43
Gaia revision 	e2d70bee03b5
Gecko build 	20140916160204
Gecko revision 	543f57ab6a44
Gecko version 	35.0a1

Comment 4

[Viorela Ioia [:viorela]]

Attachment: cost_control.webm

Comment 5

[Florin Strugariu [:Bebe]]

We have managed to reproduce something similar in test_everythingme_launch_link.TestEverythingMeLaunchLink

In this test the send_keys is not writing the full name of the search term. We want to type "facebook" and the resulted text is "faceboo" because of this the test is failing.


There are multiple issues lately with our keyboard atom and the "search panel". From what it looks like for every typed letter there are multiple search events triggered. I think because of this we see the keyboard app looses focus or types incorrect terms.

See also:
Bug 1068016 - [v2.2] go_to_url method times out on longer text

http://jenkins1.qa.scl3.mozilla.com/job/flame-319.mozilla-central.ui.functional.smoke/158/HTML_Report/

Traceback (most recent call last):
File "/var/jenkins/1/workspace/flame-319.mozilla-central.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.4-py2.7.egg/marionette/marionette_test.py", line 264, in run
testMethod()
File "/var/jenkins/1/workspace/flame-319.mozilla-central.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/tests/functional/everythingme/test_everythingme_launch_link.py", line 29, in test_launch_everything_me_link
self.assertIn(search_string.lower(), self.marionette.title.lower())
AssertionError: 'facebook' not found in u'faceboo - google search'

Comment 6

[Florin Strugariu [:Bebe]]

I took a look over the logcat of this fail and we do emit the tap command for the last "k" element but we don't see it in the UI. 

I can't reproduce this issue locally

Comment 7

[Florin Strugariu [:Bebe]]

This is failing on b2g-inbound smoke tests with the same issue in comment 5 

http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.b2g-inbound.ui.functional.smoke/236/testReport/junit/test_everythingme_launch_link/TestEverythingMeLaunchLink/test_launch_everything_me_link/history/

Comment 8

[Martijn Wargers (dead)]

Is this still an issue?

Comment 9

[Martijn Wargers (dead)]

test_cost_control_data_alert_mobile.py seems to have been disabled in the latests 2.1 runs: http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.b2g-inbound.ui.functional.smoke/HTML_Report/
Because of bug 1061698, apparently.

test_launch_everything_me_link seems to be stable for a long time already: http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.b2g-inbound.ui.functional.smoke/236/testReport/junit/test_everythingme_launch_link/TestEverythingMeLaunchLink/test_launch_everything_me_link/history/?start=200
Although, I'm not sure that link reflects the runs in 2.1.

Comment 10

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

At this point, we should probably be considering whether it's working for 2.2 more than anything. We should probably be discussing soon how much more we want to maintain the 2.1 suite.

Comment 11

[Martijn Wargers (dead)]

Florin, do you think we can mark this bug resolved->worksforme? See also comment 9.

Comment 12

[Florin Strugariu [:Bebe]]

Yeah seems OK

These tests are peaty stable now