1116283 - [v2.2] Random Failure on test_launch_rocketbar_link automated case

Status: RESOLVED WORKSFORME

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Description

[Oliver Nelson [:oliverthor]]

test_launch_rocketbar_link failed erroneously on the new base when performing automated smoketests this morning.
New Base Image: v18D

Traceback (most recent call last):
  File "/home/flash/Desktop/oliverthor/git/gaia/tests/python/gaia-ui-tests/.env/local/lib/python2.7/site-packages/marionette_client-0.8.6-py2.7.egg/marionette/marionette_test.py", line 268, in run
    testMethod()
  File "/home/flash/Desktop/oliverthor/git/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/rocketbar/test_rocketbar_launch_link.py", line 29, in test_launch_rocketbar_link
    self.assertIn(search_string.lower(), self.marionette.title.lower())
AssertionError: 'facebook' not found in u'youtube'

Error was not reproducible manually, and was not observed on the 2.2 report by Robert Chira on the previous base: v188-1


Environmental Variables:
----------------------------------------------
Device: Flame KK
BuildID: 20141229010215
Gaia: bdedbaf9f18a43c091ede770407d68d38582fe29
Gecko: 8850aa0f5332
Gonk: a814b2e2dfdda7140cb3a357617dc4fbb1435e76
Version: 37.0a1 (KK)
Firmware: V18D
User Agent: Mozilla/5.0 (Mobile; rv:37.0) Gecko/37.0 Firefox/37.0
----------------------------------------------
Repro Rate: 1/15

Comment 1

[Oliver Nelson [:oliverthor]]

Attachment: dec29_results.html

HTML Reporting from Automation performed with the failed case

Comment 2

[Oliver Nelson [:oliverthor]]

Attachment: rocketbar_results.html

HTML Results of failed case performing 11 times (all passes)

Comment 3

[Peter Bylenga [:PBylenga]]

Robert, Bebe, have you seen this error before?

Comment 4

[Robert Chira [:RobertC]]

I tried reproducing the issue locally, but with no success. I also started an adhoc run with the test on 20 repeats and they all passed.

I am not sure what happened there. When I run the steps locally after I type 'faceboo' (without the 'k'), youtube is one of the links in the search result. In the test after we type the search term we wait for at least 1 result and then tap on the 1st link. It might be that the result list did not filter fast enough after typing 'k' and the 'youtube' link was the first in the list and it tapped on that one.

Oliver, the 1/15 repro rate was achieved locally or on Jenkins (if on Jenkins then on what node)? Where you able to reproduce the issue on 188-1?

This could also have been caused by a weak wifi signal or a slow response from the device. I could try a blind patch where I wait for the link to contain the full search term, but I cannot verify it since I can't reproduce the error.

Comment 5

[Oliver Nelson [:oliverthor]]

This was performed locally as I don't have Jenkins access. My thought was a timing related issue as well, and the only occurrence of the issue was when it was performed in tandem during the other smoketests cases [attachment from comment 1]. It seems doubtful that the other tests performed beforehand would have caused an issue since the phone restarts between every test; a weak network signal seems very probable.
I'll be sure to reach you if the failure presents itself again.

Comment 6

[Peter Bylenga [:PBylenga]]

I'll resolve wfm this then.  Re-open if it occurs again.