Test test_cost_control_reset_wifi.py has failed in latest master build: http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.mozilla-central.ui.functional.non-smoke/90/HTML_Report/ I was able to reproduce the failure locally, by running the automated test, but not manually. The test is failing intermittently, repro rate: 1/5. Traceback (most recent call last): File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.non-smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.4-py2.7.egg/marionette/marionette_test.py", line 264, in run testMethod() File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.non-smoke/tests/python/gaia-ui-tests/gaiatest/tests/functional/cost_control/test_cost_control_reset_wifi.py", line 40, in test_cost_control_reset_wifi cost_control.launch() File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.non-smoke/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 27, in launch self.app = self.apps.launch(self.name, self.manifest_url, self.entry_point, launch_timeout=launch_timeout) File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.non-smoke/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 65, in launch result = self.marionette.execute_async_script("GaiaApps.launchWithName('%s')" % name, script_timeout=launch_timeout) File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.non-smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.4-py2.7.egg/marionette/marionette.py", line 1252, in execute_async_script filename=os.path.basename(frame)) File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.non-smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.4-py2.7.egg/marionette/decorators.py", line 35, in _ return func(*args, **kwargs) File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.non-smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.4-py2.7.egg/marionette/marionette.py", line 638, in _send_message self._handle_error(response) File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.non-smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.4-py2.7.egg/marionette/marionette.py", line 700, in _handle_error raise errors.ScriptTimeoutException(message=message, status=status, stacktrace=stacktrace) ScriptTimeoutException: ScriptTimeoutException: timed out The test has also failed on v2.1 This issue needs further investigation.
I was able to reproduce the issue on latest master with automated tests. Reproduction rate: 6 out of 10 The test fails when we launch the cost control app for the second time because the src of the app contains a hash at the end (change introduced in Bug 1064507). The problem is that this happens intermittently. Is there a reason why the hash would only be added some of the time while running the same test? Test link: https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/tests/functional/cost_control/test_cost_control_reset_wifi.py Build info v2.2 full flash 319MB: Gaia-Rev 73eeaa23172c26e732972c318ea6aab20e0e1cae Gecko-Rev https://hg.mozilla.org/mozilla-central/rev/a458860efad7 Build-ID 20141103160202 Version 36.0a1 Device-Name flame FW-Release 4.4.2 FW-Incremental eng.cltbld.20141103.192327 FW-Date Mon Nov 3 19:23:36 EST 2014 Bootloader L1TC00011880
Hi, sorry for delay. If the App maintains the hash the second time that the App is launched, it means the App was not closed normally, (e.g. an app is killed by OOM at background). This is a new feature launched this year, "Resume background-killed app by default". On the patch of the bug 1064507 we use this new feature to resume the App of its previous state. I don't know how your tests are launched (the environment of the device), but WDYT about clearing the hash of costcontrol when your previous test has ended or maybe, I don't know if it's possible, try to clear the previous state of the app before starting the new test. Regards.
Created attachment 8524532 [details] [review] Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/26235 It looks like the src of the cost control app contains a src when we start the app the second time. This happens intermittently because the oom killer sometimes stops the app when it's in the background (while we are using the browser).
Comment on attachment 8524532 [details] [review] Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/26235 This is not the best option but it fixes the test Add a comment in the test to explain why we close the app
Comment on attachment 8524532 [details] [review] Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/26235 merged in https://github.com/mozilla-b2g/gaia/commit/2402197700c9f1c3b24ce81f5bef3aec85d7bb7b
Leaving this open as we need to find a better solution for this issue
Did you manage to look over the issue described here in more detail?
I've talked to Marina on IRC and she told me that the app will get massive UI changes in v2.2. Because of this I will close the bug and will revisit the issue after the app refactoring.
Can this workaround now be removed for v2.2?
There haven't been any changes done to the app yet. This workaround is still needed for now.
I guess so. On the other hand, it might be that the memory usage has become better now and this workaround is not needed anymore for 2.2.