[v2.1] Fix intermittent failure in test_cost_control_reset_wifi.py

RESOLVED FIXED

Status

defect
RESOLVED FIXED
5 years ago
5 years ago

People

(Reporter: viorela, Assigned: viorela)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(4 attachments)

Assignee

Description

5 years ago
Test test_cost_control_reset_wifi.py is failing intermittently on v2.1: http://jenkins1.qa.scl3.mozilla.com/job/flame-319.mozilla-aurora.ui.functional.non-smoke/59/HTML_Report/
The test is failing because Homescreen is returned when calling self.apps.displayed_app.name, after the Browser app is launched, instead of Browser app. We need to add a better wait after launching Browser app. 

I didn't see this failing on master so far. I was able to replicate the failure locally, by running the automated test, but not manually.
Repro rate: 2 out of 10 times. 

Traceback (most recent call last):
  File "/home/viorelaioia/.virtualenvs/gaia_tests/local/lib/python2.7/site-packages/marionette_client-0.8.3-py2.7.egg/marionette/marionette_test.py", line 171, in run
    testMethod()
  File "/home/viorelaioia/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/cost_control/test_cost_control_reset_wifi.py", line 36, in test_cost_control_reset_wifi
    search.launch()
  File "/home/viorelaioia/gaia/tests/python/gaia-ui-tests/gaiatest/apps/search/app.py", line 19, in launch
    self.wait_for_condition(lambda m: self.apps.displayed_app.name == self.name)
  File "/home/viorelaioia/gaia/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 56, in wait_for_condition
    Wait(self.marionette, timeout).until(method, message=message)
  File "/home/viorelaioia/.virtualenvs/gaia_tests/local/lib/python2.7/site-packages/marionette_client-0.8.3-py2.7.egg/marionette/wait.py", line 143, in until
    cause=last_exc)

Device firmware (date) 	22 Sep 2014 16:16:10
Device firmware (incremental) 	eng.cltbld.20140922.191600
Device firmware (release) 	4.3
Device identifier 	flame
Gaia date 	22 Sep 2014 15:04:45
Gaia revision 	3742913e11f6
Gecko build 	20140922160204
Gecko revision 	b24d4d412645
Gecko version 	34.0a2
Assignee

Updated

5 years ago
Component: Gaia::Video → Gaia::UI Tests
Assignee: nobody → robert.chira
Posted image browse.png
The test still fails intermittently on v2.1. When the test fails the screenshot shows the browser app not fully loaded (screenshot attached).
In the logcat we can see that the displayed app is the vertical homescreen, but I didn't notice any errors.

Build info v2.1 flame:
Gaia-Rev        94dcc25f2e34a4900ea58310c26be52bcb089161
Gecko-Rev       https://hg.mozilla.org/releases/mozilla-aurora/rev/baaa0c3ab8fd
Build-ID        20141002000202
Version         34.0a2
Device-Name     flame
FW-Release      4.4.2
FW-Incremental  eng.cltbld.20141002.031332
FW-Date         Thu Oct  2 03:13:43 EDT 2014
Bootloader      L1TC10011800
Posted file logcat.txt
Assignee

Updated

5 years ago
Assignee: robert.chira → viorela.ioia
Assignee

Comment 3

5 years ago
I was able to get a failure of test_cost_control_reset_wifi.py on v2.2, because of the issue described here.
I started an adhoc run of this test on v2.2 and I got 1 failure out of 30 runs: http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk.ui.adhoc/187/HTML_Report/. 
The repro rate on v2.2 is very low, compared to repro rate on v2.1. 

Device firmware (date) 	07 Oct 2014 16:16:20
Device firmware (incremental) 	eng.cltbld.20141007.191609
Device firmware (release) 	4.4.2
Device identifier 	flame
Gaia date 	07 Oct 2014 06:32:28
Gaia revision 	9050edcda308
Gecko build 	20141007160202
Gecko revision 	0c8ae792f1c0
Gecko version 	35.0a1

We are still working on investigating this issue

Comment 4

5 years ago
Couldn't replicate this myself locally so I've thrown up a test PR in an adhoc job and will check the results on Monday.

Comment 5

5 years ago
Posted file github pr
This removes a hack that viorela put in (and subsequently removed the need for with the atom change) and also uses local content which gets around the slow loading issue that RobertC identified.

Adhoc results here:
http://jenkins1.qa.scl3.mozilla.com/job/flame-kk.ui.adhoc/191/
Attachment #8503990 - Flags: review?(viorela.ioia)
Attachment #8503990 - Flags: review?(robert.chira)
Assignee

Comment 6

5 years ago
Comment on attachment 8503990 [details] [review]
github pr

r+, the adhoc looks good, one failure unrelated to the fix in this PR.
Attachment #8503990 - Flags: review?(viorela.ioia) → review+
Assignee

Comment 9

5 years ago
Comment on attachment 8504047 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/25088

r+, adhoc looks good.
Attachment #8504047 - Flags: review?(viorela.ioia) → review+
Assignee

Updated

5 years ago
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
QA Whiteboard: [fxosqa-auto-s1]
You need to log in before you can comment on or make changes to this bug.