1003175 - [v1.4] Find a suitable wait in cards view method

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Description

[Viorela Ioia [:viorela]]

Test_cards_view_kill_apps_with_two_apps fails intermittently in the latest master build, we should investigate and fix the failure. 

Gaia      725a23802708eb70e3d7e8a2ce7179adbac806e4
Gecko     https://hg.mozilla.org/mozilla-central/rev/d7c07694f339
BuildID   20140429040202
Version   32.0a1
ro.build.version.incremental=eng.tclxa.20131223.163538
ro.build.date=Mon Dec 23 16:36:04 CST 2013

Stacktrace: 
Traceback (most recent call last):
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/.env/local/lib/python2.7/site-packages/marionette_client-0.7.6-py2.7.egg/marionette/marionette_test.py", line 163, in run
testMethod()
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/tests/functional/cards_view/test_cards_view_kill_apps_with_two_apps.py", line 33, in test_kill_app_from_cards_view
self.cards_view.close_app(self._test_apps[1])
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/apps/system/regions/cards_view.py", line 44, in close_app
self.wait_for_element_not_present(*self._app_card_locator(app))
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 35, in wait_for_element_not_present
lambda m: not m.find_element(by, locator))
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/.env/local/lib/python2.7/site-packages/marionette_client-0.7.6-py2.7.egg/marionette/wait.py", line 143, in until
cause=last_exc)
TimeoutException: TimeoutException: Timed out after 30.1 seconds

I was able to reproduce the failure locally, but not manually. I think we need to improve the wait in close_app() method, as the close button is not tapped.

Comment 1

[Zac C (:zac)]

No-jun, can you have a look into this?

Looks like the HTML changed. Have a look in the Gaia commit history and find it to get a clue about what happened/why and check it hasn't been backed out again.

Comment 2

[No-Jun Park [:njpark]]

It looks like the close button is not tapped, (or more like, what the automation taps is not the proper close button) so there's something wrong with     _close_buttons_locator = (By.CSS_SELECTOR, '#cards-view li[data-origin*="%s"] .close-card') now.

I looked at the code change in gaia/apps/system/js/cards_view.js, and following checkin was made during the last weekend:
d7b12959c6369e8259b8d99e11cbb2e2297d66d4
I suspect this could be it, but not sure how to proceed further.  could you help?

Comment 3

[Zac C (:zac)]

No-jun, in this case you need to dig into all of the debug options available to you, look at the html output, dump the html source, replicate it manually, use app manager to see what happens when you turn cards view on and off.

Comment 4

[Florin Strugariu [:Bebe]]

This was reproduced in the latest master build

Gaia      badc73ee7f108fa631150bded0cc57e92aad810e
Gecko     https://hg.mozilla.org/mozilla-central/rev/e19812f56952
BuildID   20140430040206
Version   32.0a1
ro.build.version.incremental=324
ro.build.date=Thu Dec 19 14:04:55 CST 2013

Traceback (most recent call last):
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/.env/local/lib/python2.7/site-packages/marionette_client-0.7.6-py2.7.egg/marionette/marionette_test.py", line 163, in run
testMethod()
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/tests/functional/cards_view/test_cards_view_kill_apps_with_two_apps.py", line 33, in test_kill_app_from_cards_view
self.cards_view.close_app(self._test_apps[1])
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/apps/system/regions/cards_view.py", line 44, in close_app
self.wait_for_element_not_present(*self._app_card_locator(app))
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 35, in wait_for_element_not_present
lambda m: not m.find_element(by, locator))
File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/.env/local/lib/python2.7/site-packages/marionette_client-0.7.6-py2.7.egg/marionette/wait.py", line 143, in until
cause=last_exc)
TimeoutException: TimeoutException: Timed out after 30.0 seconds

Comment 5

[Jason Smith [:jsmith]]

For clarity - this is a bug that requires updating the test, rather than making a fix in gaia/gecko, right?

Comment 6

[Zac C (:zac)]

(In reply to Jason Smith [:jsmith] from comment #5)
> For clarity - this is a bug that requires updating the test, rather than
> making a fix in gaia/gecko, right?

Correct.

Comment 7

[[:AndreiH]]

This fails intermittently now on another step, and this requires a better wait.

Gaia      386b5478eb9c3970972966123517e993e8a1092a                         
Gecko     https://hg.mozilla.org/mozilla-central/rev/e2e1b19fcffc          
BuildID   20140502040202                                                   
Version   32.0a1  

Traceback (most recent call last):
06:20:17   File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/.env/local/lib/python2.7/site-packages/marionette_client-0.7.7-py2.7.egg/marionette/marionette_test.py", line 163, in run
06:20:17     testMethod()
06:20:17   File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/tests/functional/cards_view/test_cards_view_kill_apps_with_two_apps.py", line 33, in test_kill_app_from_cards_view
06:20:17     self.cards_view.close_app(self._test_apps[1])
06:20:17   File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/apps/system/regions/cards_view.py", line 44, in close_app
06:20:17     self.wait_for_element_not_present(*self._app_card_locator(app))
06:20:17   File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 35, in wait_for_element_not_present
06:20:17     lambda m: not m.find_element(by, locator))
06:20:17   File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.non-smoketest/.env/local/lib/python2.7/site-packages/marionette_client-0.7.7-py2.7.egg/marionette/wait.py", line 143, in until
06:20:17     cause=last_exc)
06:20:17 TimeoutException: TimeoutException: Timed out after 30.1 seconds
06:20:17 TEST-UNEXPECTED-FAIL | test_cards_view_kill_apps_with_two_apps.py test_cards_view_kill_apps_with_two_apps.TestCardsViewTwoApps.test_kill_app_from_cards_view |

Comment 8

[Johan Lorenzo [:jlorenzo]]

Reproduced intermittently on local device with:
Gaia      386b5478eb9c3970972966123517e993e8a1092a                         
Gecko     https://hg.mozilla.org/mozilla-central/rev/e2e1b19fcffc          
BuildID   20140502040202                                                   
Version   32.0a1  

The button has enough time to be visible. The CSS selector is fine according to the App Manager's console.

Comment 9

[Zac C (:zac)]

(In reply to jlorenzo from comment #8)
> 
> The button has enough time to be visible. The CSS selector is fine according
> to the App Manager's console.

What can happen, and is possibly happening in this case, is that the close button is visible but still moving. So marionette calculates its location as soon as it is visible and then aims a tap there. however after marionette calculates but before it taps, the element keeps moving. Then the location of the tap is wrong and has no effect!

Comment 10

[Johan Lorenzo [:jlorenzo]]

(In reply to Zac C (:zac) from comment #9)
> What can happen, and is possibly happening in this case, is that the close
> button is visible but still moving.

It may be the problem. Jason and I tried to add a timer (2 seconds) before tapping any close button. I ran the test 9 times and no failures to report. Nevertheless, replacing the timer by a wait-until-displayed condition on the close button didn't solve the problem.

Is there any workaround to solve this?

Comment 11

[Johan Lorenzo [:jlorenzo]]

Attachment: Proposed patch to wait for the close button

I spoke to mdas, she proposed to add a wait_for_condition which waits until element.position stops changing.

Here is a patch waiting for the close button to stop being animated.

Comment 12

[Zac C (:zac)]

Comment on attachment 8416782 [details] [review]
Proposed patch to wait for the close button

Hi Johan, the wait seems to be good, but we can merge all 3 of those method/properties into `is_app_displayed`. We don't need to abstract it so much.

When we call `is_app_displayed` we can assume the no-transition part too.

Comment 13

[Johan Lorenzo [:jlorenzo]]

Comment on attachment 8416782 [details] [review]
Proposed patch to wait for the close button

Refactored patch.

Comment 14

[Zac C (:zac)]

Comment on attachment 8416782 [details] [review]
Proposed patch to wait for the close button

r+ works very well!!

Just need to squash this into 1 commit (as per Gaia rules) and then it's good to merge.

Comment 15

[Zac C (:zac)]

Comment on attachment 8416782 [details] [review]
Proposed patch to wait for the close button

r+, very nice patch :)

Comment 16

[Zac C (:zac)]

Merged:
https://github.com/mozilla-b2g/gaia/commit/b7de32b72cf13df7e6fd69c7eb29783c2a32f3b5

Comment 17

[[:AndreiH]]

Reopening this for v1.4

Comment 18

[[:AndreiH]]

Attachment: PR https://github.com/mozilla-b2g/gaia/pull/19158

Comment 19

[Florin Strugariu [:Bebe]]

Comment on attachment 8420906 [details] [review]
PR https://github.com/mozilla-b2g/gaia/pull/19158

this is failing on jenkins

Comment 20

[[:AndreiH]]

Comment on attachment 8420906 [details] [review]
PR https://github.com/mozilla-b2g/gaia/pull/19158

Updated

Comment 21

[[:AndreiH]]

this is still failing on v1.4 desktop build, I need to do some more investigation before the final review

Comment 22

[Zac C (:zac)]

Comment on attachment 8420906 [details] [review]
PR https://github.com/mozilla-b2g/gaia/pull/19158

re-r? us when you're ready :)

Comment 23

[[:AndreiH]]

Comment on attachment 8420906 [details] [review]
PR https://github.com/mozilla-b2g/gaia/pull/19158

This is finally for review.
After trying a lot of stuff from the HTML 'style' attribute, nothing worked good.
Desktop v1.4 or master were intermittently failing.

I think this is a good wait to go with.

Comment 24

[Zac C (:zac)]

I cherry-picked the commit to master and it works well. Can you make the pull to master and then we can go from there?

I'll test on v1.4 too.

Comment 25

[[:AndreiH]]

Attachment: MASTER PR https://github.com/mozilla-b2g/gaia/pull/19243

Master PR

Comment 26

[Zac C (:zac)]

Comment on attachment 8422364 [details] [review]
MASTER PR https://github.com/mozilla-b2g/gaia/pull/19243

r+! works well

Comment 27

[Zac C (:zac)]

Comment on attachment 8420906 [details] [review]
PR https://github.com/mozilla-b2g/gaia/pull/19158

r+ also!
works well.

Comment 28

[Florin Strugariu [:Bebe]]

Travis returned a error for the master pull.

let's wait for it to finish

Comment 29

[Florin Strugariu [:Bebe]]

Merged on master https://github.com/mozilla-b2g/gaia/commit/ac106ed7fda48543d1609fbaed2e66ff8f63a7d5

Comment 30

[Kevin Grandon :kgrandon]

This started failing quite frequently on travis master once this landed.

Examples of failures: 
https://travis-ci.org/mozilla-b2g/gaia/jobs/25163827#L2384
https://travis-ci.org/mozilla-b2g/gaia/jobs/25162330#L2373

Had to back this out: https://github.com/mozilla-b2g/gaia/commit/63f68c7b467d84f747a9549cc17b988f1e60314a

Traceback (most recent call last):

File "/home/travis/build/mozilla-b2g/gaia/travis_venv/local/lib/python2.7/site-packages/marionette_client-0.7.7-py2.7.egg/marionette/marionette_test.py", line 163, in run

testMethod()

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/cards_view/test_cards_view_with_two_apps.py", line 49, in test_that_app_can_be_launched_from_cards_view

cards_view.wait_for_cards_view_not_displayed()

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/apps/system/regions/cards_view.py", line 54, in wait_for_cards_view_not_displayed

self.wait_for_element_not_displayed(*self._cards_view_locator)

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 48, in wait_for_element_not_displayed

lambda m: not m.find_element(by, locator).is_displayed())

File "/home/travis/build/mozilla-b2g/gaia/travis_venv/local/lib/python2.7/site-packages/marionette_client-0.7.7-py2.7.egg/marionette/wait.py", line 143, in until

cause=last_exc)

TimeoutException: TimeoutException: Timed out after 10.1 seconds

TEST-UNEXPECTED-FAIL | test_cards_view_with_two_apps.py test_cards_view_with_two_apps.TestCardsView.test_that_app_can_be_launched_from_cards_view |

----------------------------------------------------------------------

Ran 1 test in 30.663s

FAILED (errors=1)

Comment 31

[Zac C (:zac)]

Sorry about that Kevin, thanks for doing the backout.

Comment 32

[[:AndreiH]]

Strange, I have runt this locally on desktop build a lot of times just to be sure :|

Comment 33

[Viorela Ioia [:viorela]]

Attachment: Github PR: https://github.com/mozilla-b2g/gaia/pull/19380

Update the xfail bug for test_cards_view_with_two_apps.

Comment 34

[Florin Strugariu [:Bebe]]

https://github.com/mozilla-b2g/gaia/commit/2194dd5edb8ab7baadab0874de96c6b8edf5c90a

Comment 35

[Viorela Ioia [:viorela]]

Attachment: Github PR: https://github.com/mozilla-b2g/gaia/pull/19422

Comment 36

[[:AndreiH]]

Comment on attachment 8426287 [details] [review]
Github PR: https://github.com/mozilla-b2g/gaia/pull/19422

Travis is happy with this. was run with repeat https://travis-ci.org/mozilla-b2g/gaia/jobs/25686690
I am going to give this an r+ because I cannot find something else better to wait for, and I did some investigation... and it still failed. So I agree with that extra wait, works just fine.

Comment 37

[Florin Strugariu [:Bebe]]

looks OK let's see what Gaia Try tells us about it:

https://tbpl.mozilla.org/?tree=Gaia-Try&rev=fedf09533129

Comment 38

[Florin Strugariu [:Bebe]]

https://github.com/mozilla-b2g/gaia/commit/dc1fdb0d26c3e9c94fe8233ec4e3d38ff6ac9abf

Comment 39

[Viorela Ioia [:viorela]]

Attachment: PR for v1.4: https://github.com/mozilla-b2g/gaia/pull/19588

Comment 40

[[:AndreiH]]

Comment on attachment 8427683 [details] [review]
PR for v1.4: https://github.com/mozilla-b2g/gaia/pull/19588

Merged on v1.4

https://github.com/mozilla-b2g/gaia/commit/9a3be9be5e89b99e6137bd9266b6823a1fbfc88d