Closed Bug 1003175 Opened 6 years ago Closed 6 years ago

[v1.4] Find a suitable wait in cards view method

Categories

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

ARM
Gonk (Firefox OS)
defect
Not set

Tracking

(b2g-v1.4 affected)

RESOLVED FIXED
Tracking Status
b2g-v1.4 --- affected

People

(Reporter: viorela, Assigned: AndreiH)

References

Details

(Whiteboard: [xfail])

Attachments

(6 files)

Test_cards_view_kill_apps_with_two_apps fails intermittently in the latest master build, we should investigate and fix the failure. 

Gaia      725a23802708eb70e3d7e8a2ce7179adbac806e4
Gecko     https://hg.mozilla.org/mozilla-central/rev/d7c07694f339
BuildID   20140429040202
Version   32.0a1
ro.build.version.incremental=eng.tclxa.20131223.163538
ro.build.date=Mon Dec 23 16:36:04 CST 2013

Stacktrace: 
Traceback (most recent call last):
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/.env/local/lib/python2.7/site-packages/marionette_client-0.7.6-py2.7.egg/marionette/marionette_test.py", line 163, in run
testMethod()
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/tests/functional/cards_view/test_cards_view_kill_apps_with_two_apps.py", line 33, in test_kill_app_from_cards_view
self.cards_view.close_app(self._test_apps[1])
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/apps/system/regions/cards_view.py", line 44, in close_app
self.wait_for_element_not_present(*self._app_card_locator(app))
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 35, in wait_for_element_not_present
lambda m: not m.find_element(by, locator))
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/.env/local/lib/python2.7/site-packages/marionette_client-0.7.6-py2.7.egg/marionette/wait.py", line 143, in until
cause=last_exc)
TimeoutException: TimeoutException: Timed out after 30.1 seconds

I was able to reproduce the failure locally, but not manually. I think we need to improve the wait in close_app() method, as the close button is not tapped.
No-jun, can you have a look into this?

Looks like the HTML changed. Have a look in the Gaia commit history and find it to get a clue about what happened/why and check it hasn't been backed out again.
Flags: needinfo?(npark)
It looks like the close button is not tapped, (or more like, what the automation taps is not the proper close button) so there's something wrong with     _close_buttons_locator = (By.CSS_SELECTOR, '#cards-view li[data-origin*="%s"] .close-card') now.

I looked at the code change in gaia/apps/system/js/cards_view.js, and following checkin was made during the last weekend:
d7b12959c6369e8259b8d99e11cbb2e2297d66d4
I suspect this could be it, but not sure how to proceed further.  could you help?
Flags: needinfo?(npark) → needinfo?(zcampbell)
No-jun, in this case you need to dig into all of the debug options available to you, look at the html output, dump the html source, replicate it manually, use app manager to see what happens when you turn cards view on and off.
Flags: needinfo?(zcampbell)
This was reproduced in the latest master build

Gaia      badc73ee7f108fa631150bded0cc57e92aad810e
Gecko     https://hg.mozilla.org/mozilla-central/rev/e19812f56952
BuildID   20140430040206
Version   32.0a1
ro.build.version.incremental=324
ro.build.date=Thu Dec 19 14:04:55 CST 2013

Traceback (most recent call last):
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/.env/local/lib/python2.7/site-packages/marionette_client-0.7.6-py2.7.egg/marionette/marionette_test.py", line 163, in run
testMethod()
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/tests/functional/cards_view/test_cards_view_kill_apps_with_two_apps.py", line 33, in test_kill_app_from_cards_view
self.cards_view.close_app(self._test_apps[1])
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/apps/system/regions/cards_view.py", line 44, in close_app
self.wait_for_element_not_present(*self._app_card_locator(app))
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 35, in wait_for_element_not_present
lambda m: not m.find_element(by, locator))
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/.env/local/lib/python2.7/site-packages/marionette_client-0.7.6-py2.7.egg/marionette/wait.py", line 143, in until
cause=last_exc)
TimeoutException: TimeoutException: Timed out after 30.0 seconds
For clarity - this is a bug that requires updating the test, rather than making a fix in gaia/gecko, right?
(In reply to Jason Smith [:jsmith] from comment #5)
> For clarity - this is a bug that requires updating the test, rather than
> making a fix in gaia/gecko, right?

Correct.
This fails intermittently now on another step, and this requires a better wait.

Gaia      386b5478eb9c3970972966123517e993e8a1092a                         
Gecko     https://hg.mozilla.org/mozilla-central/rev/e2e1b19fcffc          
BuildID   20140502040202                                                   
Version   32.0a1  

Traceback (most recent call last):
06:20:17   File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/.env/local/lib/python2.7/site-packages/marionette_client-0.7.7-py2.7.egg/marionette/marionette_test.py", line 163, in run
06:20:17     testMethod()
06:20:17   File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/tests/functional/cards_view/test_cards_view_kill_apps_with_two_apps.py", line 33, in test_kill_app_from_cards_view
06:20:17     self.cards_view.close_app(self._test_apps[1])
06:20:17   File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/apps/system/regions/cards_view.py", line 44, in close_app
06:20:17     self.wait_for_element_not_present(*self._app_card_locator(app))
06:20:17   File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 35, in wait_for_element_not_present
06:20:17     lambda m: not m.find_element(by, locator))
06:20:17   File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/.env/local/lib/python2.7/site-packages/marionette_client-0.7.7-py2.7.egg/marionette/wait.py", line 143, in until
06:20:17     cause=last_exc)
06:20:17 TimeoutException: TimeoutException: Timed out after 30.1 seconds
06:20:17 TEST-UNEXPECTED-FAIL | test_cards_view_kill_apps_with_two_apps.py test_cards_view_kill_apps_with_two_apps.TestCardsViewTwoApps.test_kill_app_from_cards_view |
Whiteboard: [xfail]
Reproduced intermittently on local device with:
Gaia      386b5478eb9c3970972966123517e993e8a1092a                         
Gecko     https://hg.mozilla.org/mozilla-central/rev/e2e1b19fcffc          
BuildID   20140502040202                                                   
Version   32.0a1  

The button has enough time to be visible. The CSS selector is fine according to the App Manager's console.
Assignee: nobody → jlorenzo
(In reply to jlorenzo from comment #8)
> 
> The button has enough time to be visible. The CSS selector is fine according
> to the App Manager's console.

What can happen, and is possibly happening in this case, is that the close button is visible but still moving. So marionette calculates its location as soon as it is visible and then aims a tap there. however after marionette calculates but before it taps, the element keeps moving. Then the location of the tap is wrong and has no effect!
(In reply to Zac C (:zac) from comment #9)
> What can happen, and is possibly happening in this case, is that the close
> button is visible but still moving.

It may be the problem. Jason and I tried to add a timer (2 seconds) before tapping any close button. I ran the test 9 times and no failures to report. Nevertheless, replacing the timer by a wait-until-displayed condition on the close button didn't solve the problem.

Is there any workaround to solve this?
I spoke to mdas, she proposed to add a wait_for_condition which waits until element.position stops changing.

Here is a patch waiting for the close button to stop being animated.
Attachment #8416782 - Flags: feedback?(zcampbell)
Comment on attachment 8416782 [details] [review]
Proposed patch to wait for the close button

Hi Johan, the wait seems to be good, but we can merge all 3 of those method/properties into `is_app_displayed`. We don't need to abstract it so much.

When we call `is_app_displayed` we can assume the no-transition part too.
Attachment #8416782 - Flags: feedback?(zcampbell) → feedback-
Blocks: 1006469
Comment on attachment 8416782 [details] [review]
Proposed patch to wait for the close button

Refactored patch.
Attachment #8416782 - Flags: review?(zcampbell)
Comment on attachment 8416782 [details] [review]
Proposed patch to wait for the close button

r+ works very well!!

Just need to squash this into 1 commit (as per Gaia rules) and then it's good to merge.
Attachment #8416782 - Flags: review?(zcampbell) → review+
Comment on attachment 8416782 [details] [review]
Proposed patch to wait for the close button

r+, very nice patch :)
Merged:
https://github.com/mozilla-b2g/gaia/commit/b7de32b72cf13df7e6fd69c7eb29783c2a32f3b5
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Reopening this for v1.4
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Attachment #8420906 - Flags: review?(zcampbell)
Attachment #8420906 - Flags: review?(florin.strugariu)
Comment on attachment 8420906 [details] [review]
PR https://github.com/mozilla-b2g/gaia/pull/19158

this is failing on jenkins
Attachment #8420906 - Flags: review?(florin.strugariu) → review-
Comment on attachment 8420906 [details] [review]
PR https://github.com/mozilla-b2g/gaia/pull/19158

Updated
Attachment #8420906 - Flags: review- → review?(florin.strugariu)
this is still failing on v1.4 desktop build, I need to do some more investigation before the final review
Assignee: jlorenzo → andrei.hutusoru
Summary: Investigate failure in test_cards_view_kill_apps_with_two_apps.py → [v1.4] Investigate failure in test_cards_view_kill_apps_with_two_apps.py
Comment on attachment 8420906 [details] [review]
PR https://github.com/mozilla-b2g/gaia/pull/19158

re-r? us when you're ready :)
Attachment #8420906 - Flags: review?(zcampbell)
Attachment #8420906 - Flags: review?(florin.strugariu)
Comment on attachment 8420906 [details] [review]
PR https://github.com/mozilla-b2g/gaia/pull/19158

This is finally for review.
After trying a lot of stuff from the HTML 'style' attribute, nothing worked good.
Desktop v1.4 or master were intermittently failing.

I think this is a good wait to go with.
Attachment #8420906 - Flags: review?(zcampbell)
Attachment #8420906 - Flags: review?(florin.strugariu)
I cherry-picked the commit to master and it works well. Can you make the pull to master and then we can go from there?

I'll test on v1.4 too.
Master PR
Attachment #8422364 - Flags: review?(zcampbell)
Attachment #8422364 - Flags: review?(florin.strugariu)
Comment on attachment 8422364 [details] [review]
MASTER PR https://github.com/mozilla-b2g/gaia/pull/19243

r+! works well
Attachment #8422364 - Flags: review?(zcampbell) → review+
Comment on attachment 8420906 [details] [review]
PR https://github.com/mozilla-b2g/gaia/pull/19158

r+ also!
works well.
Attachment #8420906 - Flags: review?(zcampbell) → review+
Attachment #8422364 - Flags: review+
Travis returned a error for the master pull.

let's wait for it to finish
Attachment #8420906 - Flags: review?(florin.strugariu) → review+
Attachment #8422364 - Flags: review?(florin.strugariu) → review+
This started failing quite frequently on travis master once this landed.

Examples of failures: 
https://travis-ci.org/mozilla-b2g/gaia/jobs/25163827#L2384
https://travis-ci.org/mozilla-b2g/gaia/jobs/25162330#L2373

Had to back this out: https://github.com/mozilla-b2g/gaia/commit/63f68c7b467d84f747a9549cc17b988f1e60314a

Traceback (most recent call last):

File "/home/travis/build/mozilla-b2g/gaia/travis_venv/local/lib/python2.7/site-packages/marionette_client-0.7.7-py2.7.egg/marionette/marionette_test.py", line 163, in run

testMethod()

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/cards_view/test_cards_view_with_two_apps.py", line 49, in test_that_app_can_be_launched_from_cards_view

cards_view.wait_for_cards_view_not_displayed()

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/apps/system/regions/cards_view.py", line 54, in wait_for_cards_view_not_displayed

self.wait_for_element_not_displayed(*self._cards_view_locator)

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 48, in wait_for_element_not_displayed

lambda m: not m.find_element(by, locator).is_displayed())

File "/home/travis/build/mozilla-b2g/gaia/travis_venv/local/lib/python2.7/site-packages/marionette_client-0.7.7-py2.7.egg/marionette/wait.py", line 143, in until

cause=last_exc)

TimeoutException: TimeoutException: Timed out after 10.1 seconds

TEST-UNEXPECTED-FAIL | test_cards_view_with_two_apps.py test_cards_view_with_two_apps.TestCardsView.test_that_app_can_be_launched_from_cards_view |

----------------------------------------------------------------------

Ran 1 test in 30.663s

FAILED (errors=1)
Sorry about that Kevin, thanks for doing the backout.
Strange, I have runt this locally on desktop build a lot of times just to be sure :|
Summary: [v1.4] Investigate failure in test_cards_view_kill_apps_with_two_apps.py → [v1.4] Find a suitable wait in cards view method
Update the xfail bug for test_cards_view_with_two_apps.
Attachment #8424795 - Flags: review?(florin.strugariu)
Attachment #8424795 - Flags: review?(andrei.hutusoru)
Attachment #8424795 - Flags: review?(florin.strugariu)
Attachment #8424795 - Flags: review?(andrei.hutusoru)
Attachment #8424795 - Flags: review+
Attachment #8426287 - Flags: review?(zcampbell)
Attachment #8426287 - Flags: review?(andrei.hutusoru)
Attachment #8426287 - Flags: review?(florin.strugariu)
Comment on attachment 8426287 [details] [review]
Github PR: https://github.com/mozilla-b2g/gaia/pull/19422

Travis is happy with this. was run with repeat https://travis-ci.org/mozilla-b2g/gaia/jobs/25686690
I am going to give this an r+ because I cannot find something else better to wait for, and I did some investigation... and it still failed. So I agree with that extra wait, works just fine.
Attachment #8426287 - Flags: review?(andrei.hutusoru) → review+
looks OK let's see what Gaia Try tells us about it:

https://tbpl.mozilla.org/?tree=Gaia-Try&rev=fedf09533129
Attachment #8426287 - Flags: review?(zcampbell)
Attachment #8426287 - Flags: review?(florin.strugariu)
Attachment #8426287 - Flags: review+
https://github.com/mozilla-b2g/gaia/commit/dc1fdb0d26c3e9c94fe8233ec4e3d38ff6ac9abf
Status: REOPENED → RESOLVED
Closed: 6 years ago6 years ago
Resolution: --- → FIXED
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Attachment #8427683 - Flags: review?(florin.strugariu)
Attachment #8427683 - Flags: review?(andrei.hutusoru)
Status: REOPENED → RESOLVED
Closed: 6 years ago6 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.