995019 - Intermittent failing test, test_homescreen_delete_app.py test_homescreen_delete_app.TestDeleteApp.test_delete_app

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::UI Tests, defect, P1)

Description

[Kevin Grandon :kgrandon]

This test seems to be failing more and more on travis, and today it's gotten up to 50% or more of all builds. We need to disable this while we investigate what's going on.

TEST-START test_homescreen_delete_app.py

test_delete_app (test_homescreen_delete_app.TestDeleteApp) ... ERROR

======================================================================

ERROR: None

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/mozilla-b2g/gaia/travis_venv/local/lib/python2.7/site-packages/marionette_client-0.7.5-py2.7.egg/marionette/marionette_test.py", line 163, in run

testMethod()

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/homescreen/test_homescreen_delete_app.py", line 52, in test_delete_app

self.homescreen.wait_for_app_icon_not_present(self.APP_NAME)

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/apps/homescreen/app.py", line 44, in wait_for_app_icon_not_present

self.wait_for_element_not_present(self._homescreen_icon_locator[0], self._homescreen_icon_locator[1] % app_name)

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 35, in wait_for_element_not_present

lambda m: not m.find_element(by, locator))

File "/home/travis/build/mozilla-b2g/gaia/travis_venv/local/lib/python2.7/site-packages/marionette_client-0.7.5-py2.7.egg/marionette/wait.py", line 143, in until

cause=last_exc)

TimeoutException: TimeoutException: Timed out after 10.0153160095 seconds

TEST-UNEXPECTED-FAIL | test_homescreen_delete_app.py test_homescreen_delete_app.TestDeleteApp.test_delete_app |

Comment 1

[Kevin Grandon :kgrandon]

This test has been disabled in: https://github.com/mozilla-b2g/gaia/commit/866c05655407097b4d56d72b418920100e029e2c

Zac - can you get someone to investigate this? Thanks!

Comment 2

[[:AndreiH]]

I will take a look at this. Thanks!

Comment 3

[[:AndreiH]]

This is a case of the button being visible but not usable yet:
"ElementNotVisibleException: Element is not currently visible and may not be manipulated".
I tried waiting for:

confirm_button = self.marionette.find_element(*self._confirm_button_locator)
confirm_dialog_menu = self.marionette.find_element(*self._confirm_dialog_menu)
self.wait_for_condition(lambda m: confirm_dialog_menu.location['x'] + confirm_button.location['x'] == confirm_button.location['x'])

But this does not help either.

Comment 4

[[:AndreiH]]

confirm_button = self.marionette.find_element(*self._confirm_button_locator)
self.wait_for_condition(lambda m: confirm_button.is_displayed())

Does not seem to help :(

Comment 5

[[:AndreiH]]

Attachment: PR https://github.com/mozilla-b2g/gaia/pull/18228

Changed Travis script also, for tests.

Comment 6

[Zac C (:zac)]

(In reply to [:AndreiH] from comment #5)
> Created attachment 8405344 [details] [review]
> PR https://github.com/mozilla-b2g/gaia/pull/18228
> 
> Changed Travis script also, for tests.

I've seen this on v1.4 too. 
If we do go with a 'hack' like this then we should file a bug against the app requesting a 'loaded' class and then refer to that in the code comment.

Comment 7

[[:AndreiH]]

(In reply to Zac C (:zac) from comment #6)
> (In reply to [:AndreiH] from comment #5)
> > Created attachment 8405344 [details] [review]
> > PR https://github.com/mozilla-b2g/gaia/pull/18228
> > 
> > Changed Travis script also, for tests.
> 
> I've seen this on v1.4 too. 
> If we do go with a 'hack' like this then we should file a bug against the
> app requesting a 'loaded' class and then refer to that in the code comment.


Sounds good to me. I will file a bug requesting a 'loaded' class. For what gaia component should this bug be filed?

Comment 8

[[:AndreiH]]

zac but this cannot be reproducible manually.
If this is the case should we open a bug to request a "loaded: class just because our automated tests requires this?

Comment 9

[Zac C (:zac)]

(In reply to [:AndreiH] from comment #8)
> zac but this cannot be reproducible manually.
> If this is the case should we open a bug to request a "loaded: class just
> because our automated tests requires this?

yeah let's do it, we need stability on this.

Comment 10

[[:AndreiH]]

I added another wait : 'self.wait_for_condition(lambda m: m.find_element(*self._confirm_dialog_locator).get_attribute('class')=='visible show')'

Let's see how Travis likes it this way.

Comment 11

[Zac C (:zac)]

Comment on attachment 8405344 [details] [review]
PR https://github.com/mozilla-b2g/gaia/pull/18228

r+

Comment 12

[Zac C (:zac)]

Merged:
https://github.com/mozilla-b2g/gaia/commit/45f8ee7409d638a84635647e2e5d375f3e131ef0

Comment 13

[Rudy Lu [:rudyl] (inactive after 2015/7/3)]

I can still this intermittent failure on master or v2.0,

master,
https://bugzilla.mozilla.org/show_bug.cgi?id=1016850#c8

v2.0,
https://travis-ci.org/mozilla-b2g/gaia/jobs/28486780

--
Zac,

Do we need to re-open this bug or disable this test?
Thanks.

Comment 14

[Zac C (:zac)]

really annoying, the output report has not uploaded to AWS on that job so I can't debug it :(

Comment 15

[Zac C (:zac)]

I will try locally.

Comment 16

[Zac C (:zac)]

The failure in comment#13 is a dupe of bug 1016850. 

I'll reclose this as per its original resolution and move discussion over to that bug.

Comment 17

[Rudy Lu [:rudyl] (inactive after 2015/7/3)]

(In reply to Zac C (:zac) from comment #16)
> The failure in comment#13 is a dupe of bug 1016850. 
> 
> I'll reclose this as per its original resolution and move discussion over to
> that bug.

Sorry, if this caused any confusion, but I was thinking to move the discussion here, since I thought bug 1016850 is just about moving the patch here to v1.4.
Actually, I thought we should have set bug 1016850 a dupe of this bug, since these 2 bugs talk about the same issue, right?

Comment 18

[Firefox Bug Husbandry Bot]

Removing leave-open keyword from resolved bugs, per :sylvestre.