974935 - Intermittent test_homescreen_delete_app | test_homescreen_launch_app

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::UI Tests, defect, P1)

Description

[Robert Chira [:RobertC]]

It seems that the issue is caused be the fact that the notification banner when an app is installed does not appear after confirming the installation.
We ran the tests on local devices (Hamachi) but could not reproduce the issue. We couldn't reproduce the issue manually either.

Stacktrace from the last Jenkins build:
Traceback (most recent call last):
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.2/.env/local/lib/python2.7/site-packages/marionette_client-0.7.3-py2.7.egg/marionette/marionette_test.py", line 127, in run
self.setUp()
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.2/tests/python/gaia-ui-tests/gaiatest/tests/functional/homescreen/test_homescreen_launch_app.py", line 31, in setUp
confirm_install.tap_confirm()
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.2/tests/python/gaia-ui-tests/gaiatest/apps/homescreen/regions/confirm_install.py", line 17, in tap_confirm
System(self.marionette).wait_for_banner()
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.2/tests/python/gaia-ui-tests/gaiatest/apps/system/app.py", line 63, in wait_for_banner
self.wait_for_element_displayed(*self._banner_locator)
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.2/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 40, in wait_for_element_displayed
lambda m: m.find_element(by, locator).is_displayed())
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.2/.env/local/lib/python2.7/site-packages/marionette_client-0.7.3-py2.7.egg/marionette/wait.py", line 143, in until
cause=last_exc)
TimeoutException: Traceback (most recent call last):
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.2/.env/local/lib/python2.7/site-packages/marionette_client-0.7.3-py2.7.egg/marionette/wait.py", line 122, in until
rv = condition(self.marionette)
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.2/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 40, in <lambda>
lambda m: m.find_element(by, locator).is_displayed())
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.2/.env/local/lib/python2.7/site-packages/marionette_client-0.7.3-py2.7.egg/marionette/marionette.py", line 1176, in find_element
response = self._send_message('findElement', 'value', **kwargs)
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.2/.env/local/lib/python2.7/site-packages/marionette_client-0.7.3-py2.7.egg/marionette/marionette.py", line 605, in _send_message
self._handle_error(response)
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.2/.env/local/lib/python2.7/site-packages/marionette_client-0.7.3-py2.7.egg/marionette/marionette.py", line 626, in _handle_error
raise NoSuchElementException(message=message, status=status, stacktrace=stacktrace)
TimeoutException: Timed out after 30.2175428867 seconds, caused by <class 'marionette.errors.NoSuchElementException'>


Latest build on which we ran the tests locally:
Gaia      6e71ab4da1b08586ea0c758edb7aa199ee34cd2f
Gecko     https://hg.mozilla.org/mozilla-central/rev/660b62608951
BuildID   20140220040203
Version   30.0a1
ro.build.version.incremental=eng.zxliu.20130911.142924
ro.build.date=Wed Sep 11 14:29:37 CST 2013

Comment 1

[Zac C (:zac)]

Attachment: github pr

Just neatened these up overall. I hope it makes them a bit more reliable too.

Comment 2

[Zac C (:zac)]

adhoc:
http://qa-selenium.mv.mozilla.com:8080/job/b2g.hamachi.mozilla-central.ui.adhoc/234/

Comment 3

[[:AndreiH]]

I restarted the adhoc job, because yours failed to run:
http://qa-selenium.mv.mozilla.com:8080/view/B2G%20Hamachi/job/b2g.hamachi.mozilla-central.ui.adhoc/235/console

Comment 4

[[:AndreiH]]

Comment on attachment 8379946 [details] [review]
github pr

I agree with the changes
and adhoc job too
http://qa-selenium.mv.mozilla.com:8080/view/B2G%20Hamachi/job/b2g.hamachi.mozilla-central.ui.adhoc/237/consoleFull

Comment 5

[[:AndreiH]]

Merged:
https://github.com/mozilla-b2g/gaia/commit/f8a5809f3297dbf5c08880f7e74f9910eaa1bc9c

Comment 6

[[:AndreiH]]

I've reopened this bug because we need to fix the current failure:

Traceback (most recent call last):
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.2/.env/local/lib/python2.7/site-packages/marionette_client-0.7.3-py2.7.egg/marionette/marionette_test.py", line 127, in run
self.setUp()
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.2/tests/python/gaia-ui-tests/gaiatest/tests/functional/homescreen/test_homescreen_delete_app.py", line 33, in setUp
confirm_install.tap_confirm()
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.2/tests/python/gaia-ui-tests/gaiatest/apps/homescreen/regions/confirm_install.py", line 17, in tap_confirm
System(self.marionette).wait_for_banner()
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.2/tests/python/gaia-ui-tests/gaiatest/apps/system/app.py", line 63, in wait_for_banner
self.wait_for_element_displayed(*self._banner_locator)
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.2/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 40, in wait_for_element_displayed
lambda m: m.find_element(by, locator).is_displayed())
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.2/.env/local/lib/python2.7/site-packages/marionette_client-0.7.3-py2.7.egg/marionette/wait.py", line 143, in until
cause=last_exc)

Comment 7

[[:AndreiH]]

Attachment: PR https://github.com/mozilla-b2g/gaia/pull/16612

Comment 8

[Zac C (:zac)]

Comment on attachment 8381343 [details] [review]
PR https://github.com/mozilla-b2g/gaia/pull/16612

r+

Comment 9

[Zac C (:zac)]

Merged:
https://github.com/mozilla-b2g/gaia/commit/8e21d0a5cdac94f05e9c3623fa3b9ad579ba2967

Comment 10

[[:AndreiH]]

Sadly reopening this, the tests are still failing at "wait_for_app_icon_present" now

Comment 11

[[:AndreiH]]

Making this P1 as it is ruining our perfectly green runs

Comment 12

[[:AndreiH]]

I barely reproduced this locally, 2 times of 100 runs... and when I saw it fail the test failed after tapping on Install button, then nothing happened. I don't really know what is the cause of this.

Thought that the test is moving to fast, so I added a time.sleep() here:
https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/apps/homescreen/regions/confirm_install.py#L15
but then it reproduced again...

Can it be an environmental issue? Zac should we backout the changes?

Also managed to get the logcat that I will post in an attachment.

Comment 13

[[:AndreiH]]

Attachment: logcat2.txt

Comment 14

[Zac C (:zac)]

This failed before and after my patch so I'm not convinced my patch caused it.

I tried again locally couldn't replicate it but I feel like it's something to do with the internet connection. Just a hunch.

The other suspicious thing is that this test is super reliable on Travis.

Comment 15

[[:AndreiH]]

Another thing, when I see this failing, it is always failing on b2g-11.
When it run on b2g-7 it was a green build, and the current green build was run on b2g-9:
http://qa-selenium.mv.mozilla.com:8080/view/B2G%20Hamachi/job/b2g.hamachi.mozilla-central.ui.2/194/consoleFull

Comment 16

[Zac C (:zac)]

We have resolved this by disabling b2g-11.
I will investigate it when I get to MV.

Comment 17

[[:AndreiH]]

Zac, I know the office is moving. Do you have a change in investigating this?
Should we still keep this open or it's good to be closed?

Comment 18

[[:AndreiH]]

I think you got b2g-11 removed, and changed with another phone.
That Hamachi was causing this to fail. I will close this.