1562870 - Generate Conditioned profiles

Status: RESOLVED FIXED

(Testing :: Raptor, enhancement, P2)

Description

[Tarek Ziadé (:tarek)]

Create a Taskcluster Fetch job that produces/updates a collection of profiles into reusable artifacts

Comment 1

[Tarek Ziadé (:tarek)]

Notes to myself:

• add a task file here https://searchfox.org/mozilla-central/source/taskcluster/ci/fetch/
• add a standalone profile builder here https://searchfox.org/mozilla-central/source/taskcluster/scripts/misc/

Comment 2

[Nick Alexander :nalexander [he/him]]

I think there are a couple of different things that will be needed here.

1. some kind of "heavy" conditioning, which is done over time or relatively infrequently. This one is all about populating data in some way, wants versioning, etc. It's important that this can be parameterized to accommodate different behaviours and scenarios.

2. quick "light" conditioning that mostly just bumps timestamps and ensures that periodic tasks are up-to-date (like safe browsing downloads). This one consumes a "heavy" profile and ensures its ready for a specific test or tests. This one is all about ensuring timestamps are fresh, doesn't need to persist for a long time, and needs to be fast enough to run before each and every tests.

Comment 3

[Tarek Ziadé (:tarek)]

The work is happening now in https://github.com/tarekziade/conditioned-profile and will eventually be added in our tree

Comment 4

[Tarek Ziadé (:tarek)]

Attachment: Bug 1562870 - generate conditioned profiles r?gbrown,bc,aerickson

Generates conditioned profiles
This is a work in progress. Pushing it so the patch is grabbable

Comment 5

[Tarek Ziadé (:tarek)]

status update: (will do one more Friday before I leave for 2 weeks) Nick I am :ni you so you get that update

try of the day (contains a cold and a heavy profile for the latest nighly, generated from linux)
https://treeherder.mozilla.org/#/jobs?repo=try&revision=3f798181b2371f07a0304f4e3c044d834fd35748

The permanent link to the latest version of the indexed artifacts is https://tools.taskcluster.net/index/gecko.v2.try.latest.firefox/conditioned-profile
So if you want the heavy one, this is the permalink:

https://index.taskcluster.net/v1/task/gecko.v2.try.latest.firefox.conditioned-profile/artifacts/public/conditioned_profile/profile-linux-heavy-days-vanilla.tgz

next step is to push and vendor everything in mozilla-central so the CLI can be used from the task without downloading my binary release

Comment 6

[Tarek Ziadé (:tarek)]

Nicholas,

Following your comment at https://phabricator.services.mozilla.com/D38211#inline-232434 , I wanted to give here more details about my plans, to make sure it sounds good to you and we are aligned.

I think the key points of this project is to make sure:
1/ anyone that needs a profile can grab it using an URL of a zipped directory
2/ it's easy to change+add scenarii that produces said directoty, using a webdriver client
3/ the TC task runs smoothly for all the gecko flavors we're going to use to build profiles

After some discussions with David Burns about how android+webdriver is/will work in our testing environment, my plan to make this work for Android is to run geckodriver on the host, driving the target android build from within an emulator.

I am aware of https://bugzilla.mozilla.org/show_bug.cgi?id=1525126 and I know that there's a similar effort going on for WPT, so it seems to be
the path forward. If you think there's a better way please let's talk asap. I will surely ping Geoff as well to validate this model (I was planning to have him + someone from the TC team as reviewers once my try build is using a fully vendored version of the code -- today probably)

About the tools used: I am not using Marionette as you mentioned, the current code was recycled from a previous project that uses Arsenic, which is an equivalent of the selenium Python client. It drives geckodriver, chromedriver etc.. the same way, and I will probably need to tweak its behavior for the emulator, but I don't think that will require much work on that side.

I am completely agnostic about the tool that's used to produce the profile, as long as it produces the right HTTP webdriver calls, it's an implementation detail. An arsenic scenario is just a function that drives the browser, like other tools would do -- see https://github.com/tarekziade/conditioned-profile/blob/master/condprof/scenario/heavy.py#L57-L86

So, I am curious to understand what would be the benefit of moving to another toolchain for the MVP, as long as the profiles are built and it's easy to modify/add new scenarii. If there's a bigger plan in your mind, happy to discuss it. Once everything is plugged and working, using webdriver client A or webdriver B should not be too hard since everything is loosely coupled in the project -- but I can't think of a use case that's not possible with any webdriver client.

Comment 7

[Tarek Ziadé (:tarek)]

mach try with vendored code - https://treeherder.mozilla.org/#/jobs?repo=try&revision=8976072e6b5c51f591400abdd141035652d00ac7

Comment 8

[Geoff Brown [:gbrown]]

Thinking about extending this work to android, I am reminded of https://searchfox.org/mozilla-central/source/testing/mozharness/scripts/android_emulator_pgo.py, which generates and retrieves fennec profiles for pgo builds; I believe :mshal is the expert on android pgo. android_emulator_pgo.py and all of our existing android continuous integration tasks rely on the AndroidMixin, https://searchfox.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/testing/android.py, to run and manage the emulator (or connect to a physical device on bitbar) and mozdevice, https://searchfox.org/mozilla-central/source/testing/mozbase/mozdevice/mozdevice, to access the device (install an app, launch an app, push/pull files).

Comment 9

[Geoff Brown [:gbrown]]

Be aware of https://searchfox.org/mozilla-central/source/testing/mozbase/mozprofile/mozprofile and https://searchfox.org/mozilla-central/source/testing/profiles, which might have useful features that could be leveraged.

Comment 10

[Tarek Ziadé (:tarek)]

Thanks Geoff, I updated the patch and added you and pmoore as reviewers + added on Phab the list of stuff that remains to be done.
I will be back on that work in August

Comment 11

[Tarek Ziadé (:tarek)]

try https://treeherder.mozilla.org/#/jobs?repo=try&revision=aae4fc563ebdc5877872200fe7da8e60c53b30fa

Comment 12

[Tarek Ziadé (:tarek)]

try run https://treeherder.mozilla.org/#/jobs?repo=try&revision=0a9eb942d2adffd0afe4065a1a06656d53dd9388

Comment 13

[Tarek Ziadé (:tarek)]

fresh try run https://treeherder.mozilla.org/#/jobs?repo=try&revision=a44d8f388e1a56f9f6f22766b35123d8c3d157fd

Comment 14

[Tarek Ziadé (:tarek)]

Depends on https://github.com/mozilla-mobile/fenix/issues/5156

Comment 15

[Tarek Ziadé (:tarek)]

Try run on g5, p2s, windows, linux, macOs https://treeherder.mozilla.org/#/jobs?repo=try&revision=b16df4e604f6e53a9f3bb3ea97241209913a40e1&selectedJob=265876917

Comment 16

[Tarek Ziadé (:tarek)]

Geoffrey, do you have a bit of time to do a new review ? I think it's ready to land (besides the bitbar worker types that we'll need to tweak)

Comment 17

[Geoff Brown [:gbrown]]

Done!

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e0ba466d216a
generate conditioned profiles r=gbrown,bc,aerickson

Comment 19

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/e0ba466d216a

Comment 20

[Nick Alexander :nalexander [he/him]]

This has landed so clearing NI.