[UI Tests] Intermittent Travis UI Test Failure: TEST-UNEXPECTED-FAIL | test_add_contact_to_favorites.py test_add_contact_to_favorites.TestAddContactToFavorite.test_add_contact_to_favorite | AssertionError: u'Add as Favorite' != 'Remove as Favorite'

RESOLVED FIXED

Status

Firefox OS
Gaia::UI Tests
P1
normal
RESOLVED FIXED
5 years ago
5 years ago

People

(Reporter: julienw, Assigned: zac)

Tracking

unspecified
Other
Gonk (Firefox OS)

Firefox Tracking Flags

(b2g-v1.2 affected)

Details

Attachments

(4 attachments, 2 obsolete attachments)

(Reporter)

Description

5 years ago
Traceback (most recent call last):

File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.6.2-py2.7.egg/marionette/marionette_test.py", line 143, in run

testMethod()

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/contacts/test_add_contact_to_favorites.py", line 25, in test_add_contact_to_favorite

self.assertEqual(contact_details.add_remove_text, 'Remove as Favorite')

TEST-UNEXPECTED-FAIL | test_add_contact_to_favorites.py test_add_contact_to_favorites.TestAddContactToFavorite.test_add_contact_to_favorite | AssertionError: u'Add as Favorite' != 'Remove as Favorite'
(Assignee)

Comment 1

5 years ago
OK the wait here on line #66 is terrible, it's doing almost nothing:
https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/apps/contacts/regions/contact_details.py#L66

After the tap we should wait for something in/on the page to change state properly.
OS: Linux → Gonk (Firefox OS)
Priority: -- → P1
Hardware: x86_64 → Other
Assignee: nobody → bob.silverberg
Status: NEW → ASSIGNED
Created attachment 832913 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/13746

Pointer to Github pull-request
Attachment #832913 - Flags: review?(zcampbell)
Attachment #832913 - Flags: review?(trifandreialin)
Attachment #832913 - Flags: review?(moz.teodosia)
Attachment #832913 - Flags: review?(florin.strugariu)
Attachment #832913 - Flags: review?(andrei.hutusoru)
Created attachment 832917 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/13747

Pointer to Github pull-request
Comment on attachment 832917 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/13747

Same change for v1.2
Attachment #832917 - Flags: review?(zcampbell)
Attachment #832917 - Flags: review?(trifandreialin)
Attachment #832917 - Flags: review?(moz.teodosia)
Attachment #832917 - Flags: review?(florin.strugariu)
Attachment #832917 - Flags: review?(andrei.hutusoru)
Attachment #832913 - Flags: review?(florin.strugariu) → review+

Updated

5 years ago
Attachment #832913 - Flags: review?(andrei.hutusoru) → review-
Attachment #832917 - Flags: review?(florin.strugariu) → review+

Updated

5 years ago
Attachment #832917 - Flags: review?(moz.teodosia) → review+
(Assignee)

Comment 5

5 years ago
Comment on attachment 832913 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/13746

I haven't been able to run it but the logic looks sound.
Attachment #832913 - Flags: review?(zcampbell) → review+

Comment 6

5 years ago
Can someone else please test that on desktopb2g? Because this fails for me intermittently
(Assignee)

Comment 7

5 years ago
I can test on desktopb2g.
(Assignee)

Comment 8

5 years ago
What is the error you're getting Andrei?
(Assignee)

Updated

5 years ago
Attachment #832913 - Flags: review?(trifandreialin)
Attachment #832913 - Flags: review?(moz.teodosia)
(Assignee)

Comment 10

5 years ago
Comment on attachment 832917 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/13747

r+
Attachment #832917 - Flags: review?(zcampbell)
Attachment #832917 - Flags: review?(trifandreialin)
Attachment #832917 - Flags: review?(andrei.hutusoru)
Attachment #832917 - Flags: review+
(Assignee)

Comment 12

5 years ago
Reverted masteR:
https://github.com/mozilla-b2g/gaia/commit/52978d943425c086482a946e63842e83d5d2b2de

Reverted v1.2:
https://github.com/mozilla-b2g/gaia/commit/3d68f43721911869be14b0876e9d58e817f90e20

This has made stability worse.

It is waiting for visible text which may be off-screen, hence the wait is failing.
(Assignee)

Comment 13

5 years ago
Having a quick look at this now.
Assignee: bob.silverberg → zcampbell
(Assignee)

Comment 14

5 years ago
Created attachment 8337832 [details] [review]
github pr

Just doing a trial run on Travis to fix this test.

If it works I'll revert the changes to `script` file.
Attachment #832913 - Attachment is obsolete: true
Attachment #832917 - Attachment is obsolete: true
Attachment #8337832 - Flags: review?(fyen)
Attachment #8337832 - Flags: review?(bob.silverberg)
Comment on attachment 8337832 [details] [review]
github pr

LGTM, just need to undo the script change.
Attachment #8337832 - Flags: review?(bob.silverberg) → review+
Attachment #8337832 - Flags: review?(fyen)
Landed on master in https://github.com/mozilla-b2g/gaia/commit/671a7c1fae1c32a77b2675636c7efdf089fd6443

We should bring this into v1.2 as well.
status-b2g-v1.2: --- → affected
(Assignee)

Comment 17

5 years ago
Created attachment 8338406 [details] [review]
github pr

Still not happy with this so I'm fixing a second failure which is when the "add to favorite" button is slightly off screen and not tapped by Marionette properly.
Attachment #8338406 - Flags: review?(florin.strugariu)
Attachment #8338406 - Flags: review?(bob.silverberg)
Comment on attachment 8338406 [details] [review]
github pr

works like a charm
Attachment #8338406 - Flags: review?(florin.strugariu) → review+

Comment 19

5 years ago
Created attachment 8339142 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/14110

Pointer to Github pull-request

Comment 20

5 years ago
Comment on attachment 8339142 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/14110

UPLIFTED TO V1.2
Attachment #8339142 - Flags: review?(trifandreialin)
Attachment #8339142 - Flags: review?(florin.strugariu)
Attachment #8339142 - Flags: review?(bob.silverberg)

Comment 21

5 years ago
Comment on attachment 8338406 [details] [review]
github pr

Works nicely!
Attachment #8338406 - Flags: review+
Comment on attachment 8339142 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/14110

This PR is causing 3 tests to fail, which can be seen on Travis. I am surprised that this PR was submitted at all, considering that the test it was meant to fix is now broken. Were the tests run against v1.2?
Attachment #8339142 - Flags: review?(bob.silverberg) → review-
Created attachment 8339259 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/14127

Pointer to Github pull-request
Comment on attachment 8339259 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/14127

Uplift to v1.2
Attachment #8339259 - Flags: review?(viorela.ioia)
Attachment #8339259 - Flags: review?(trifandreialin)
Attachment #8339259 - Flags: review?(florin.strugariu)
Attachment #8339259 - Flags: review?(andrei.hutusoru)
Attachment #8339259 - Flags: review?(viorela.ioia) → review+

Comment 26

5 years ago
Comment on attachment 8339259 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/14127

lgtm, my bad, I did not run it on device. brought in both commits
Attachment #8339259 - Flags: review?(andrei.hutusoru) → review+
can't we do the 2 pulls in one ?

They fix the same test right?
Flags: needinfo?(bob.silverberg)
Flags: needinfo?(andrei.hutusoru)
Bebe, that is irrelevant now, as the 2nd part of the fix has already landed in both master and v1.2. Now we just need a working version of the first part of the fix for v1.2.

In the future, yes, it would make sense to do a single PR with both commits for the uplift.
Flags: needinfo?(bob.silverberg)
Attachment #8339259 - Flags: review?(trifandreialin) → review+
Attachment #8339259 - Flags: review?(florin.strugariu)
Thanks Bob
Flags: needinfo?(andrei.hutusoru)
Marking this bug as RF, since it landed in master and v1.2.
Thanks guys!
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
Comment on attachment 8339142 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/14110

pull 14110 was closed after the merge of pull 14127, the review are no longer needed
Attachment #8339142 - Flags: review?(trifandreialin)
Attachment #8339142 - Flags: review?(florin.strugariu)
You need to log in before you can comment on or make changes to this bug.