985498 - Investigate and fix test_delete_app fail on Travis

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Description

[Florin Strugariu [:Bebe]]

https://travis-ci.org/mozilla-b2g/gaia/jobs/20567329


TEST-START test_delete_contact.py

test_delete_contact (test_delete_contact.TestContacts) ... ERROR

======================================================================

ERROR: None

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/mozilla-b2g/gaia/travis_venv/local/lib/python2.7/site-packages/marionette_client-0.7.5-py2.7.egg/marionette/marionette_test.py", line 163, in run

testMethod()

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/contacts/test_delete_contact.py", line 29, in test_delete_contact

contact_item_detail = contact_item.tap()

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/apps/contacts/app.py", line 118, in tap

return ContactDetails(self.marionette)

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/apps/contacts/regions/contact_details.py", line 22, in __init__

self.wait_for_condition(lambda m: m.find_element(*self._contact_name_title_locator).location['x'] == 0)

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 54, in wait_for_condition

Wait(self.marionette, timeout).until(method, message=message)

File "/home/travis/build/mozilla-b2g/gaia/travis_venv/local/lib/python2.7/site-packages/marionette_client-0.7.5-py2.7.egg/marionette/wait.py", line 143, in until

cause=last_exc)

TimeoutException: TimeoutException: Timed out after 10.0300328732 seconds

TEST-UNEXPECTED-FAIL | test_delete_contact.py test_delete_contact.TestContacts.test_delete_contact |

----------------------------------------------------------------------

Ran 1 test in 20.888s

Comment 1

[Florin Strugariu [:Bebe]]

Sorry wrong stack trace


The right one is:
est_delete_app (test_homescreen_delete_app.TestDeleteApp) ... ERROR

======================================================================

ERROR: None

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/mozilla-b2g/gaia/travis_venv/local/lib/python2.7/site-packages/marionette_client-0.7.5-py2.7.egg/marionette/marionette_test.py", line 163, in run

testMethod()

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/homescreen/test_homescreen_delete_app.py", line 52, in test_delete_app

self.homescreen.wait_for_app_icon_not_present(self.APP_NAME)

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/apps/homescreen/app.py", line 44, in wait_for_app_icon_not_present

self.wait_for_element_not_present(self._homescreen_icon_locator[0], self._homescreen_icon_locator[1] % app_name)

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 35, in wait_for_element_not_present

lambda m: not m.find_element(by, locator))

File "/home/travis/build/mozilla-b2g/gaia/travis_venv/local/lib/python2.7/site-packages/marionette_client-0.7.5-py2.7.egg/marionette/wait.py", line 143, in until

cause=last_exc)

TimeoutException: TimeoutException: Timed out after 10.0454580784 seconds

TEST-UNEXPECTED-FAIL | test_homescreen_delete_app.py test_homescreen_delete_app.TestDeleteApp.test_delete_app |

----------------------------------------------------------------------

Ran 1 test in 28.563s

Comment 2

[Florin Strugariu [:Bebe]]

https://travis-ci.org/mozilla-b2g/gaia/jobs/21034659

Comment 3

[Kevin Grandon :kgrandon]

Disabled for being intermittent: https://github.com/mozilla-b2g/gaia/commit/d2eaf26da6a1806d5a8fd73e7060273e8dbe1015

Comment 4

[Zac C (:zac)]

This seems to be legitimate, sometimes the app just doesn't delete.

Comment 5

[Zac C (:zac)]

I know how to fix this but I don't know what's causing it. As ith most things in this suite it seems to be a race between marionette and the mozApps.install. 
When the test fails it leaves the installed app in state in which it can neither be launched or deleted. It's hard to debug it as as soon as you measure it it starts to pass.
Needs some more work..

Comment 6

[[:AndreiH]]

This still fails on Travis: https://travis-ci.org/mozilla-b2g/gaia/jobs/22090175

I just wanted to know why is this disabled on device also?
http://selenium.qa.mtv2.mozilla.com:8080/view/B2G%20Hamachi/job/b2g.hamachi.mozilla-central.ui.adhoc/330/console

Comment 7

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to [:AndreiH] from comment #6)
> This still fails on Travis:
> https://travis-ci.org/mozilla-b2g/gaia/jobs/22090175
> 
> I just wanted to know why is this disabled on device also?
> http://selenium.qa.mtv2.mozilla.com:8080/view/B2G%20Hamachi/job/b2g.hamachi.
> mozilla-central.ui.adhoc/330/console

Because it fails, on mozilla-central, on device, too: bug 991262.

Comment 9

[Michael Henretty [:mikehenrty][:mhenretty]]

For informational purposes, we didn't start seeing this test fail on 1.4 travis until the camera uplifts [1]. I'm going to disable this on 1.4 as well.

1.) https://github.com/mozilla-b2g/gaia/pull/17935

Comment 10

[Michael Henretty [:mikehenrty][:mhenretty]]

Disabled on v1.4: https://github.com/mozilla-b2g/gaia/commit/288f85c5d90f2e40d32ac035f2b04d43eb10360c

Comment 11

[Jason Smith [:jsmith]]

Have we determined here if this is a bug in the test or a real gaia/gecko bug?

Comment 12

[Zac C (:zac)]

This is a test issue, the test is tapping on the button before it's ready. This happens often when the event handler is added to the element after it is shown. Marionette trusts heavily on shown.

Nothing has changed in Gaia except perhaps a performance improvement or re-ordering of the initialisation of the confirm-dialog.

It seems fine on master but is failing on v1.4.

Comment 13

[Stephen Donner [:stephend] Not actively reading bugmail]

Looks like the manual testers found an interesting/bad bug over in bug 993682; can we work on getting this re-enabled on master?  Thx.

Comment 14

[[:AndreiH]]

Attachment: PR https://github.com/mozilla-b2g/gaia/pull/18111

I tested this on the latest Hamachi build and test works fine:

Gaia      650e8c2c611ed07495d3bf3769f44a0efd88a492                         
Gecko     https://hg.mozilla.org/mozilla-central/rev/5811efc11011          
BuildID   20140408160203                                                   
Version   31.0a1

Comment 15

[Florin Strugariu [:Bebe]]

https://github.com/mozilla-b2g/gaia/commit/95f0a722620bc9cd0e6921d248753cb4a1320c07

Comment 16

[Firefox Bug Husbandry Bot]

Removing leave-open keyword from resolved bugs, per :sylvestre.