If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Investigate intermittent failure in test_cost_control_reset_wifi.py

RESOLVED WONTFIX

Status

Firefox OS
Gaia::UI Tests
RESOLVED WONTFIX
3 years ago
3 years ago

People

(Reporter: viorela, Assigned: RobertC)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

3 years ago
Test  test_cost_control_reset_wifi.py has failed in latest master build: http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.mozilla-central.ui.functional.non-smoke/90/HTML_Report/
I was able to reproduce the failure locally, by running the automated test, but not manually.
The test is failing intermittently, repro rate: 1/5.

Traceback (most recent call last):
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.non-smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.4-py2.7.egg/marionette/marionette_test.py", line 264, in run
testMethod()
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.non-smoke/tests/python/gaia-ui-tests/gaiatest/tests/functional/cost_control/test_cost_control_reset_wifi.py", line 40, in test_cost_control_reset_wifi
cost_control.launch()
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.non-smoke/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 27, in launch
self.app = self.apps.launch(self.name, self.manifest_url, self.entry_point, launch_timeout=launch_timeout)
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.non-smoke/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 65, in launch
result = self.marionette.execute_async_script("GaiaApps.launchWithName('%s')" % name, script_timeout=launch_timeout)
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.non-smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.4-py2.7.egg/marionette/marionette.py", line 1252, in execute_async_script
filename=os.path.basename(frame[0]))
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.non-smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.4-py2.7.egg/marionette/decorators.py", line 35, in _
return func(*args, **kwargs)
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.non-smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.4-py2.7.egg/marionette/marionette.py", line 638, in _send_message
self._handle_error(response)
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.non-smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.4-py2.7.egg/marionette/marionette.py", line 700, in _handle_error
raise errors.ScriptTimeoutException(message=message, status=status, stacktrace=stacktrace)
ScriptTimeoutException: ScriptTimeoutException: timed out

The test has also failed on v2.1
This issue needs further investigation.
QA Whiteboard: [fxosqa-auto-backlog+]
(Assignee)

Updated

3 years ago
Assignee: nobody → robert.chira
QA Whiteboard: [fxosqa-auto-backlog+] → [fxosqa-auto-s3]
(Assignee)

Comment 1

3 years ago
I was able to reproduce the issue on latest master with automated tests.
Reproduction rate: 6 out of 10

The test fails when we launch the cost control app for the second time because the src of the app contains a hash at the end (change introduced in Bug 1064507).

The problem is that this happens intermittently. Is there a reason why the hash would only be added some of the time while running the same test?

Test link:
https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/tests/functional/cost_control/test_cost_control_reset_wifi.py

Build info v2.2 full flash 319MB:
Gaia-Rev        73eeaa23172c26e732972c318ea6aab20e0e1cae
Gecko-Rev       https://hg.mozilla.org/mozilla-central/rev/a458860efad7
Build-ID        20141103160202
Version         36.0a1
Device-Name     flame
FW-Release      4.4.2
FW-Incremental  eng.cltbld.20141103.192327
FW-Date         Mon Nov  3 19:23:36 EST 2014
Bootloader      L1TC00011880
Flags: needinfo?(marina.rodrigueziglesias)
Hi, 
sorry for delay. If the App maintains the hash the second time that the App is launched, it means the App was not closed normally, (e.g. an app is killed by OOM at background). This is a new feature launched this year, "Resume background-killed app by default". 
On the patch of the bug 1064507 we use this new feature to resume the App of its previous state.

I don't know how your tests are launched (the environment of the device), but WDYT about clearing the hash of costcontrol when your previous test has ended or maybe, I don't know if it's possible, try to clear the previous state of the app before starting the new test.

Regards.
Flags: needinfo?(marina.rodrigueziglesias)
(Assignee)

Updated

3 years ago
QA Whiteboard: [fxosqa-auto-s3] → [fxosqa-auto-from-s2] [fxosqa-auto-from-s3] [fxosqa-auto-s4]
(Assignee)

Comment 3

3 years ago
Created attachment 8524532 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/26235

It looks like the src of the cost control app contains a src when we start the app the second time. This happens intermittently because the oom killer sometimes stops the app when it's in the background (while we are using the browser).
Attachment #8524532 - Flags: review?(viorela.ioia)
Attachment #8524532 - Flags: review?(florin.strugariu)
Comment on attachment 8524532 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/26235

This is not the best option but it fixes the test

Add a comment in the test to explain why we close the app
Attachment #8524532 - Flags: review?(florin.strugariu) → review+
Comment on attachment 8524532 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/26235

merged in https://github.com/mozilla-b2g/gaia/commit/2402197700c9f1c3b24ce81f5bef3aec85d7bb7b
Attachment #8524532 - Flags: review?(viorela.ioia)
Leaving this open as we need to find a better solution for this issue
(Assignee)

Comment 7

3 years ago
Did you manage to look over the issue described here in more detail?
Flags: needinfo?(marina.rodriguez.iglesias)
(Assignee)

Comment 8

3 years ago
I've talked to Marina on IRC and she told me that the app will get massive UI changes in v2.2. Because of this I will close the bug and will revisit the issue after the app refactoring.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Flags: needinfo?(marina.rodriguez.iglesias)
Resolution: --- → WONTFIX
Can this workaround now be removed for v2.2?
Flags: needinfo?(robert.chira)
(Assignee)

Comment 10

3 years ago
There haven't been any changes done to the app yet. This workaround is still needed for now.
Flags: needinfo?(robert.chira)
I guess so. On the other hand, it might be that the memory usage has become better now and this workaround is not needed anymore for 2.2.
You need to log in before you can comment on or make changes to this bug.