1201268 - test_browser_clear_browsing_history.py: ScriptTimeoutException: ScriptTimeoutException: timed out

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Description

[Nicholas Nelson [:NicholasN]]

Description:
The test hangs after navigation to a web page, and eventually times out. The settings app is never opened.

Steps:
1. Update a flame device to 20150902030203.
2. Open the browser and navigate to http://192.168.2.12:57046/mozilla.html.
3. Open Settings and go to browsing privacy.
4. Tap "Clear browsing history".

Actual Result:
Test fails after navigating to web page.

Expected Result:
Settings app is opened and browsing history is cleared.

Repro Rate: 5/5 in Jenkins adhoc, not reproduced manually.

Traceback (most recent call last):
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace@3/.env/lib/python2.7/site-packages/marionette_client-0.17-py2.7.egg/marionette/marionette_test.py", line 296, in run
testMethod()
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace@3/tests/python/gaia-ui-tests/gaiatest/tests/functional/browser/test_browser_clear_browsing_history.py", line 32, in test_browser_clear_history
search.launch()
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace@3/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 25, in launch
self.app = self.apps.launch(self.name, self.manifest_url, self.entry_point, launch_timeout=launch_timeout)
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace@3/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 93, in launch
% (manifest_url, json.dumps(entry_point)), script_timeout=launch_timeout)
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace@3/.env/lib/python2.7/site-packages/marionette_driver-0.13-py2.7.egg/marionette_driver/marionette.py", line 1551, in execute_async_script
rv = self._send_message("executeAsyncScript", body, key="value")
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace@3/.env/lib/python2.7/site-packages/marionette_driver-0.13-py2.7.egg/marionette_driver/decorators.py", line 36, in _
return func(*args, **kwargs)
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace@3/.env/lib/python2.7/site-packages/marionette_driver-0.13-py2.7.egg/marionette_driver/marionette.py", line 711, in _send_message
    self._handle_error(resp)
  File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace@3/.env/lib/python2.7/site-packages/marionette_driver-0.13-py2.7.egg/marionette_driver/marionette.py", line 752, in _handle_error
    raise errors.lookup(error)(message, stacktrace=stacktrace)
ScriptTimeoutException: ScriptTimeoutException: timed out

Comment 1

[Martijn Wargers (dead)]

Hmm, I wonder when this started to regress. A simple self.apps.kill(search.app) fixes it, but it shouldn't be necessary.

Comment 2

[Martijn Wargers (dead)]

It looks like the failure comes from here: http://mxr.mozilla.org/gaia/source/tests/atoms/gaia_apps.js#312
312               // wait for the displayed app to have the expected source URL
313               return GaiaApps.getDisplayedApp().src == (origin + launchPath);

console.log('\n********-* ' + origin + launchPath); gives:
********-* app://search.gaiamobile.org/newtab.html 
console.log('\n ********* ' + GaiaApps.getDisplayedApp().src); gives: 
I/GeckoConsole(14360):  ********* http://192.168.1.67:60678/mozilla.html

Comment 3

[Martijn Wargers (dead)]

No idea when this regressed, I guess this could be a regression from bug 1169010 (which makes an intentional behavior change), but it seems difficult fo find out with all the Jenkins failures.
Nowadays, when you tap on the Browser icon, it opens an existing browsers instance. I noticed with an older build that this used to open a new browser instance.
I think it would be good to get a dogfood, sanity or smoketest around this behavior.

In the meantime, I think the workaround from comment 1 might actually be a good solution, because this test actually expects a new browser instance to work on to get the history items from the start page.

Comment 4

[Martijn Wargers (dead)]

But I also think there is an issue with the launch code in gaia_apps.js here.
http://mxr.mozilla.org/gaia/source/tests/atoms/gaia_apps.js#313
313               return GaiaApps.getDisplayedApp().src == (origin + launchPath);

GaiaApps.getDisplayedApp().src changes src to what the current url of that browser instance is.
There is no way of keeping track of that, because normally this is tracked by the app.src (or origin).
So I think here we would want to just get a handle on a browser instance, it doesn't matter which one. Then we could use .origin here, so do something like this:               return GaiaApps.getDisplayedApp().origin == origin;
But if we do that, then we choke at: http://mxr.mozilla.org/gaia/source/tests/atoms/gaia_apps.js#286
286         let appWindow = GaiaApps.getAppByURL(app.origin + launchPath);
Because app.origin + launchPath doesn't exist anymore and was replaced by whatever browser page was loading.
So I guess you would need a GaiaApps.getAppByOrigin or something (or getAppByManifestURL).

Comment 5

[GitHub Autolander Bot]

Attachment: [gaia] mwargers:1201268 > mozilla-b2g:master

Comment 6

[Martijn Wargers (dead)]

Comment on attachment 8656541 [details] [review]
[gaia] mwargers:1201268 > mozilla-b2g:master

If I remove the self.device.touch_home_button() call, then I get intermittent failures in launch(). I guess killing an active app and then immediately restarting it, is not stable, currently.

The app.py are not really necessary for this test to get working again, but it makes this test run 10s faster, because it doesn't wait a 2nd time for if there are history items.

Comment 7

[Martijn Wargers (dead)]

Btw, I didn't see any automated test for opening a new browser window, I think we need one. I don't see it mentioned in the dogfood spreadsheet either: https://docs.google.com/spreadsheets/d/1RNVZbWiqc4n4h0ftlzzdYJ-a7Iu0YZEQZUfMEOJj3Co/edit#gid=0

Comment 8

[No-Jun Park [:njpark]]

Comment on attachment 8656541 [details] [review]
[gaia] mwargers:1201268 > mozilla-b2g:master

Noticed two intermittent failures when running on flame - since 319MB + Browser combo almost guarantees that it'll run really slowly, we need to accommodate for the timeouts.

Comment 9

[Martijn Wargers (dead)]

Comment on attachment 8656541 [details] [review]
[gaia] mwargers:1201268 > mozilla-b2g:master

Ok, I tested this on Aries and Flame with 10 repeats and I didn't see intermittent failures related to this pull request.

I guess on the Flame with 319MB, the browser was sometimes killed after the home button was killed.
Which then causes the assert to fail in that function:
http://mxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/gaia_test.py#127
Perhaps we should remove that assert? Because I think in most use cases we just want to get rid of that particular app when it's there. When it's not there anymore, it's also fine.

Comment 10

[No-Jun Park [:njpark]]

Comment on attachment 8656541 [details] [review]
[gaia] mwargers:1201268 > mozilla-b2g:master

LGTM

Comment 11

[Johan Lorenzo [:jlorenzo]]

Comment on attachment 8656541 [details] [review]
[gaia] mwargers:1201268 > mozilla-b2g:master

Like discussed offline,
> Wait(self.marionette).until(expected.element_not_displayed)
actually waits 10 seconds before timing out.

Hence, I don't see any issue in the PR. Thanks for the investigation, Martijn!

Comment 12

[Martijn Wargers (dead)]

(In reply to Johan Lorenzo [:jlorenzo] (QA) from comment #11)
> Like discussed offline,
> > Wait(self.marionette).until(expected.element_not_displayed)
> actually waits 10 seconds before timing out.

Yes, see bug 1201870, comment 1. I guess that's the reason why we have is_element_displayed/is_element_present left, in case we don't want to wait 10 seconds.

Comment 13

[Martijn Wargers (dead)]

Merged: https://github.com/mozilla-b2g/gaia/commit/aebecea113503792e0ff8009d53b76728f6ca131

Also note that apparently, you can't kill and immediately reopen that killed app from script without failures.