939062 - [UI Tests] Intermittent Travis UI Test Failure: TEST-UNEXPECTED-FAIL | test_add_contact_to_favorites.py test_add_contact_to_favorites.TestAddContactToFavorite.test_add_contact_to_favorite | AssertionError: u'Add as Favorite' != 'Remove as Favorite'

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::UI Tests, defect, P1)

Description

[Julien Wajsberg [:julienw]]

Traceback (most recent call last):

File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.6.2-py2.7.egg/marionette/marionette_test.py", line 143, in run

testMethod()

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/contacts/test_add_contact_to_favorites.py", line 25, in test_add_contact_to_favorite

self.assertEqual(contact_details.add_remove_text, 'Remove as Favorite')

TEST-UNEXPECTED-FAIL | test_add_contact_to_favorites.py test_add_contact_to_favorites.TestAddContactToFavorite.test_add_contact_to_favorite | AssertionError: u'Add as Favorite' != 'Remove as Favorite'

Comment 1

[Zac C (:zac)]

OK the wait here on line #66 is terrible, it's doing almost nothing:
https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/apps/contacts/regions/contact_details.py#L66

After the tap we should wait for something in/on the page to change state properly.

Comment 2

[Bob Silverberg [:bsilverberg]]

Attachment: Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/13746

Pointer to Github pull-request

Comment 3

[Bob Silverberg [:bsilverberg]]

Attachment: Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/13747

Pointer to Github pull-request

Comment 4

[Bob Silverberg [:bsilverberg]]

Comment on attachment 832917 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/13747

Same change for v1.2

Comment 5

[Zac C (:zac)]

Comment on attachment 832913 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/13746

I haven't been able to run it but the logic looks sound.

Comment 6

[[:AndreiH]]

Can someone else please test that on desktopb2g? Because this fails for me intermittently

Comment 7

[Zac C (:zac)]

I can test on desktopb2g.

Comment 8

[Zac C (:zac)]

What is the error you're getting Andrei?

Comment 9

[Zac C (:zac)]

Merged master:
https://github.com/mozilla-b2g/gaia/commit/67498c7024a28091a658626d4e12d92131a2f3e0

Comment 10

[Zac C (:zac)]

Comment on attachment 832917 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/13747

r+

Comment 11

[Zac C (:zac)]

Merged v1.2:
https://github.com/mozilla-b2g/gaia/commit/1155902e8855eeb8206250910c0c719f979091b1

Comment 12

[Zac C (:zac)]

Reverted masteR:
https://github.com/mozilla-b2g/gaia/commit/52978d943425c086482a946e63842e83d5d2b2de

Reverted v1.2:
https://github.com/mozilla-b2g/gaia/commit/3d68f43721911869be14b0876e9d58e817f90e20

This has made stability worse.

It is waiting for visible text which may be off-screen, hence the wait is failing.

Comment 13

[Zac C (:zac)]

Having a quick look at this now.

Comment 14

[Zac C (:zac)]

Attachment: github pr

Just doing a trial run on Travis to fix this test.

If it works I'll revert the changes to `script` file.

Comment 15

[Bob Silverberg [:bsilverberg]]

Comment on attachment 8337832 [details] [review]
github pr

LGTM, just need to undo the script change.

Comment 16

[Bob Silverberg [:bsilverberg]]

Landed on master in https://github.com/mozilla-b2g/gaia/commit/671a7c1fae1c32a77b2675636c7efdf089fd6443

We should bring this into v1.2 as well.

Comment 17

[Zac C (:zac)]

Attachment: github pr

Still not happy with this so I'm fixing a second failure which is when the "add to favorite" button is slightly off screen and not tapped by Marionette properly.

Comment 18

[Florin Strugariu [:Bebe]]

Comment on attachment 8338406 [details] [review]
github pr

works like a charm

Comment 19

[[:AndreiH]]

Attachment: Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/14110

Pointer to Github pull-request

Comment 20

[[:AndreiH]]

Comment on attachment 8339142 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/14110

UPLIFTED TO V1.2

Comment 21

[[:AndreiH]]

Comment on attachment 8338406 [details] [review]
github pr

Works nicely!

Comment 22

[Bob Silverberg [:bsilverberg]]

Comment on attachment 8339142 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/14110

This PR is causing 3 tests to fail, which can be seen on Travis. I am surprised that this PR was submitted at all, considering that the test it was meant to fix is now broken. Were the tests run against v1.2?

Comment 23

[Bob Silverberg [:bsilverberg]]

Comment on attachment 8338406 [details] [review]
github pr

Landed on master in https://github.com/mozilla-b2g/gaia/commit/6f6cd8caac5d44762cbff38c97517dc1d48278ba

Comment 24

[Bob Silverberg [:bsilverberg]]

Attachment: Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/14127

Pointer to Github pull-request

Comment 25

[Bob Silverberg [:bsilverberg]]

Comment on attachment 8339259 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/14127

Uplift to v1.2

Comment 26

[[:AndreiH]]

Comment on attachment 8339259 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/14127

lgtm, my bad, I did not run it on device. brought in both commits

Comment 27

[Florin Strugariu [:Bebe]]

can't we do the 2 pulls in one ?

They fix the same test right?

Comment 28

[Bob Silverberg [:bsilverberg]]

Bebe, that is irrelevant now, as the 2nd part of the fix has already landed in both master and v1.2. Now we just need a working version of the first part of the fix for v1.2.

In the future, yes, it would make sense to do a single PR with both commits for the uplift.

Comment 29

[Trif Andrei-Alin[:AlinT]]

merged in v1.2 with commit:
https://github.com/mozilla-b2g/gaia/commit/1b67b551be11daf33dad8387a7153cc52e5cefa2

Comment 30

[Florin Strugariu [:Bebe]]

Thanks Bob

Comment 31

[Trif Andrei-Alin[:AlinT]]

Marking this bug as RF, since it landed in master and v1.2.
Thanks guys!

Comment 32

[Trif Andrei-Alin[:AlinT]]

Comment on attachment 8339142 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/14110

pull 14110 was closed after the merge of pull 14127, the review are no longer needed