888103 - More realistic app startup times on firefoxos

Status: RESOLVED FIXED

(Testing Graveyard :: Eideticker, defect)

Description

[William Lachance (:wlach)]

Right now the eideticker startup tests start each app synthetically, using the gaiatest framework:

https://github.com/mozilla/eideticker/blob/master/src/tests/b2g/applaunching/startup.py#L21

It would be more realistic to test how long it takes from the time that an app is pressed on the home screen. We should update our tests to do that. There's two parts to this:

1. Somehow finding the position of the app icon to press (swiping to the right screen if necessary?)
2. Modifying eideticker to press the icon, and record the time that it was pressed (depends on bug 888102)

Comment 1

[Jonathan Griffin (:jgriffin)]

It looks like we should be able to click the app's button on the grid.  See e.g., 

https://github.com/mozilla/gaia-ui-tests/blob/master/gaiatest/tests/homescreen/test_homescreen_launch_app.py

Comment 2

[William Lachance (:wlach)]

Attachment: Start apps by pressing on their icon on the homescreen

Ok this turned out to be somewhat urgent so I did a first pass at implementation today. Unfortunately my current solution duplicates a lot of code in gaiatest, but the way that code is structured isn't straightforwardly reusable. Thoughts?

Comment 3

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 810802 [details] [diff] [review]
Start apps by pressing on their icon on the homescreen

Review of attachment 810802 [details] [diff] [review]:
-----------------------------------------------------------------

::: src/tests/b2g/applaunching/startup.py
@@ +52,4 @@
>          # gain
>          apps.set_permission('Camera', 'geolocation', 'deny')
>  
> +        visible_icon_locator = (By.CSS_SELECTOR, 'li.icon[aria-label="%s"]' % self.appname)

There's a homescreen app object that should help with some of this. We should probably add anything else we need to the app object too. It looks like the following (untested) should work:

from gaiatest.apps.homescreen.app import Homescreen
homescreen = Homescreen(self.device.marionette)
homescreen.switch_to_homescreen_frame()
while homescreen.homescreen_has_more_pages:
    appicon = self.device.marionette.find_element(homescreen._homescreen_icon_locator % self.appname)
    if appicon.is_displayed():
        break
    homescreen.go_to_next_page()

Comment 4

[William Lachance (:wlach)]

Attachment: 0001-Bug-888103-Start-apps-by-pressing-on-their-icon-on-h.patch

Awesome suggestion, using the new homescreen object cleaned up the code quite a bit.

Comment 5

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 814648 [details] [diff] [review]
0001-Bug-888103-Start-apps-by-pressing-on-their-icon-on-h.patch

Review of attachment 814648 [details] [diff] [review]:
-----------------------------------------------------------------

::: src/tests/b2g/applaunching/startup.py
@@ +17,4 @@
>          # gain
>          apps.set_permission('Camera', 'geolocation', 'deny')
>  
> +        self.homescreen = apps.launch('Homescreen')

I've seen some issues with this, where we had two instances of the Homescreen app running. If we're confident we already have the homescreen running then we should probably skip this launch.

Comment 6

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 814648 [details] [diff] [review]
0001-Bug-888103-Start-apps-by-pressing-on-their-icon-on-h.patch

Review of attachment 814648 [details] [diff] [review]:
-----------------------------------------------------------------

I just tried this with the 'Music' app and instead of navigating to the correct page it taps the 'Music' collection from everything.me I think we should be able to avoid this by switching to the first page of local apps before attempting to find the icon.

Comment 7

[William Lachance (:wlach)]

Landed with suggested fixes:

https://github.com/mozilla/eideticker/commit/abae9ea5dc5db53aff2d13259abb6ecfbdbe6a1a

Comment 8

[William Lachance (:wlach)]

(In reply to Dave Hunt (:davehunt) from comment #6)
> Comment on attachment 814648 [details] [diff] [review]
> 0001-Bug-888103-Start-apps-by-pressing-on-their-icon-on-h.patch
> 
> Review of attachment 814648 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> I just tried this with the 'Music' app and instead of navigating to the
> correct page it taps the 'Music' collection from everything.me I think we
> should be able to avoid this by switching to the first page of local apps
> before attempting to find the icon.

Oh crud, I didn't realize that this got an r- before pushing. Could you look at what I committed and give it a retroactive r+ if it's ok? I essentially just did what you suggested.

Comment 9

[Dave Hunt [:davehunt] [he/him] ⌚BST]

That's probably because I changed my review, sorry about that. Your commit looks good to me, and is what I did when running these tests recently.