1011469 - Intermittent in cost control tests, tapping Next button has no effect

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::UI Tests, defect, P1)

Description

[[:AndreiH]]

Attachment: download.png

Test test_cost_control_reset_wifi.TestCostControlReset failed on todays run
Let's add a better wait, master can also benefit from this.

http://selenium.qa.mtv2.mozilla.com:8080/view/B2G%20Hamachi/job/b2g.hamachi.mozilla-b2g30_v1_4.ui.non-smoketest/35/HTML_Report/

Traceback (most recent call last):
  File "/var/jenkins/workspace/b2g.hamachi.mozilla-b2g30_v1_4.ui.non-smoketest/.env/local/lib/python2.7/site-packages/marionette_client-0.7.5-py2.7.egg/marionette/marionette_test.py", line 163, in run
    testMethod()
  File "/var/jenkins/workspace/b2g.hamachi.mozilla-b2g30_v1_4.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/tests/functional/cost_control/test_cost_control_reset_wifi.py", line 24, in test_cost_control_reset_wifi
    cost_control.run_ftu_accepting_defaults()
  File "/var/jenkins/workspace/b2g.hamachi.mozilla-b2g30_v1_4.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/apps/cost_control/app.py", line 60, in run_ftu_accepting_defaults
    ftu_step2 = ftu_step1.tap_next()
  File "/var/jenkins/workspace/b2g.hamachi.mozilla-b2g30_v1_4.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/apps/cost_control/regions/ftu_step1.py", line 22, in tap_next
    return FTUStep2(self.marionette)
  File "/var/jenkins/workspace/b2g.hamachi.mozilla-b2g30_v1_4.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/apps/cost_control/regions/ftu_step2.py", line 19, in __init__
    self.wait_for_condition(lambda m: view.location['x'] == 0)
  File "/var/jenkins/workspace/b2g.hamachi.mozilla-b2g30_v1_4.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 54, in wait_for_condition
    Wait(self.marionette, timeout).until(method, message=message)
  File "/var/jenkins/workspace/b2g.hamachi.mozilla-b2g30_v1_4.ui.non-smoketest/.env/local/lib/python2.7/site-packages/marionette_client-0.7.5-py2.7.egg/marionette/wait.py", line 143, in until
    cause=last_exc)
TimeoutException: TimeoutException: Timed out after 30.1255269051 seconds

Comment 1

[[:AndreiH]]

Attachment: Master PR https://github.com/mozilla-b2g/gaia/pull/19324

Comment 2

[[:AndreiH]]

Attachment: v1.4 PR https://github.com/mozilla-b2g/gaia/pull/19325

Comment 3

[Viorela Ioia [:viorela]]

Comment on attachment 8423833 [details] [review]
Master PR https://github.com/mozilla-b2g/gaia/pull/19324

Lgtm

Comment 4

[Florin Strugariu [:Bebe]]

https://github.com/mozilla-b2g/gaia/commit/17cc405ab6163528efe321e831bf0e72d1166a28

Comment 5

[Viorela Ioia [:viorela]]

Comment on attachment 8423842 [details] [review]
v1.4 PR https://github.com/mozilla-b2g/gaia/pull/19325

Test still failing

Comment 6

[[:AndreiH]]

I know it failed with this new wait, but it is better that the last one on v1.4.
And I want this to be merged instead of using a time.sleep(1)
This definitely improves the test on that step. If it fails badly on CI we will investigate and add another wait. We currently don't have for something else to be more simple to wait for.

Comment 7

[Viorela Ioia [:viorela]]

Comment on attachment 8423842 [details] [review]
v1.4 PR https://github.com/mozilla-b2g/gaia/pull/19325

I agree, this wait is better than the one we already have.

Comment 8

[Florin Strugariu [:Bebe]]

https://github.com/mozilla-b2g/gaia/commit/117a72f740b941fafb0babdd07ac3eed4db57f33

Comment 9

[Viorela Ioia [:viorela]]

The failure described in this bug was not fixed, we just replaced a wait. We still need to work on fixing it, as it was reproducible in the latest v1.4 build: http://selenium.qa.mtv2.mozilla.com:8080/job/b2g.hamachi.mozilla-b2g30_v1_4.ui.smoketest/50/HTML_Report/

Gaia      16f7ef17feb08562aa6a3cfd6d561b2fcca652c0
Gecko     https://hg.mozilla.org/releases/mozilla-b2g30_v1_4/rev/9cc96d59cc30
BuildID   20140523000202
Version   30.0
ro.build.version.incremental=eng.tclxa.20131223.163538
ro.build.date=Mon Dec 23 16:36:04 CST 2013

Comment 10

[Viorela Ioia [:viorela]]

I've been able to reproduce this failure manually, easily. 
 
#STR:
1. Flash your phone with the latest v1.4 build.
2. Launch the Usage app
3. Tap next button right when the Welcome page is displayed

#Expected results:
Data report page is displayed

#Actual report:
Tapping Next button has no effect

Gaia      16f7ef17feb08562aa6a3cfd6d561b2fcca652c0
Gecko     https://hg.mozilla.org/releases/mozilla-b2g30_v1_4/rev/9cc96d59cc30
BuildID   20140523000202
Version   30.0
ro.build.version.incremental=eng.tclxa.20131223.163538
ro.build.date=Mon Dec 23 16:36:04 CST 2013

In our automated tests, we can avoid this failure by adding a wait of 1-2 seconds after we get on the Welcome page, then tap the Next button. 
I see this as a bug, the Next button being tapped should have effect even if we tap it quickly after it is displayed.

Comment 11

[Jason Smith [:jsmith]]

Can you get a bug on file for comment 10 then?

Comment 12

[Viorela Ioia [:viorela]]

I opened Bug 1015840

Comment 13

[[:AndreiH]]

I saw this reproducing on master also,

http://selenium.qa.mtv2.mozilla.com:8080/view/B2G%20Master/job/b2g.hamachi.mozilla-central.ui.smoketest/48/HTML_Report/

if we add a workaround we should add it on master also.

Comment 14

[Viorela Ioia [:viorela]]

Attachment: PR for v2.0: https://github.com/mozilla-b2g/gaia/pull/19633

Comment 15

[Viorela Ioia [:viorela]]

Attachment: PR for v1.4: https://github.com/mozilla-b2g/gaia/pull/19595

Comment 16

[[:AndreiH]]

Comment on attachment 8428569 [details] [review]
PR for v1.4: https://github.com/mozilla-b2g/gaia/pull/19595

Merged on v1.4:
https://github.com/mozilla-b2g/gaia/commit/a209b03c3fc64beda3580be39a79e5c7495b3acf

Comment 17

[[:AndreiH]]

Comment on attachment 8428568 [details] [review]
PR for v2.0: https://github.com/mozilla-b2g/gaia/pull/19633

Merged on v2.0:

https://github.com/mozilla-b2g/gaia/commit/4c5f4020adf6b774331a08bf8ca72fb3f95331ac