1598059 - [meta] Add raptor-browsertime support for google chrome on android

Status: RESOLVED FIXED

(Testing :: Raptor, enhancement, P3)

Description

[Robert Wood [:rwood]]

I believe one of the decision factors for wanting to switch from the raptor-webext to the browsertime tool is the ability to use it with the Chrome browser android app. This bug is to add support for that, both locally as well as in production, via mach raptor --browsertime.

Just some questions to consider when it comes to the production side:

• Where will the Chrome app come from (including updates) for our production CI?
• Does the 'chromedriver' used with Chrome desktop work with Chrome android or is there a separate one required? Where will it come from (and be updated from)?
• Can the Chrome app be automatically installed and updated on our android devices at bitbar? Is there support for that in mozharness and the bitbar container/worker?
• Will the mitmproxy recordings made for Firefox android work on Chrome android (assuming so)?
• Will running the browsertime tests on Chrome android provide stable data allowing for regression detection (how noisy/stable is it)?

Comment 1

[Andrew Erickson [:aerickson]]

(In reply to Robert Wood [:rwood] from comment #0)

I believe one of the decision factors for wanting to switch from the raptor-webext to the browsertime tool is the ability to use it with the Chrome browser android app. This bug is to add support for that, both locally as well as in production, via mach raptor --browsertime.

Just some questions to consider when it comes to the production side:

• Where will the Chrome app come from (including updates) for our production CI?

The only official way to get Chrome for Android is from the Google Play store. I've asked Bitbar if it's possible for our devices to install Google Play apps. If they can't be logged in with a user, then we'd have to harvest APKs ourselves using a tool like https://github.com/ClaudiuGeorgiu/PlaystoreDownloader. It requires a Google account to operate, but we'd only have to run it on one server and then store the APK and have the others install it.

• Does the 'chromedriver' used with Chrome desktop work with Chrome android or is there a separate one required? Where will it come from (and be updated from)?

Not sure if it is the same or if it works, but it would be run in the docker container and be updated in the Dockerfile or the tests could install/update it (more flexible).

• Can the Chrome app be automatically installed and updated on our android devices at bitbar? Is there support for that in mozharness and the bitbar container/worker?

I've asked Bitbar about installation/updates. Not sure about mozharness supporting updating apps.

• Will the mitmproxy recordings made for Firefox android work on Chrome android (assuming so)?
• Will running the browsertime tests on Chrome android provide stable data allowing for regression detection (how noisy/stable is it)?

Comment 2

[Andrew Erickson [:aerickson]]

Android chromedriver docs: https://chromedriver.chromium.org/getting-started/getting-started---android

• it does seem to be the same binary for desktop and android

Comment 3

[Andrew Erickson [:aerickson]]

Bitbar says: "We can have chrome installed on the devices and disable it from clean-up scripts and enable auto updates."

Comment 4

[Robert Wood [:rwood]]

Thanks for the info Andrew, much appreciated!

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

if this is P5, then is there a real driving factor for all the browsertime work? I know there are some nice to have benefits, but my understanding is chrome on android was the #1 reason for exploring browsertime.

Comment 6

[Tarek Ziadé (:tarek)]

it's a driving factor as much as some other reasons, but there's a pile of work before we can do it. we'll get to it, but right now we have a lot of work to finish this quarter to finish maturing browsertime integration. The goal is to finish all bugs in https://bugzilla.mozilla.org/show_bug.cgi?id=1574182 that are P1 and P2 then move to P3-P5

P1 does not mean important and P5 not important in the way I triage. P1 means way more urgent than P5

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Tarek Ziadé (:tarek) from comment #6)

P1 does not mean important and P5 not important in the way I triage. P1 means way more urgent than P5

P4 is only for bots, and P5 mainly for intermittent failures. So this has to stay as P3, and would have to be re-prioritized once all dependencies have been fixed.

Comment 9

[Dave Hunt [:davehunt] [he/him] ⌚BST]

:sparky can we close this?