1537944 - Add conditioned-profile support for desktop Firefox in Raptor

Status: RESOLVED FIXED

(Testing :: Raptor, task, P1)

Description

[Robert Wood [:rwood]]

Add the ability for Raptor to use preconditioned browser profiles. Perf team will provide more details for the profile conditions to use.

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Currently the only conditioning Raptor does is wait 30 seconds for the browser to settle. We're considering alternative conditioning strategies such as load an unrelated website. This bug is about investigating if we can avoid conditioning the browser for every test in a suite, and instead to perform the conditioning up-front.

This would involve an initial browser launch, execute the conditioning step (wait 30 seconds, etc), taking a copy of the profile, and then closing the browser. Subsequent browser launches would use a clone of the conditioned profile. This should save us the repeated cost of conditioning the profile, which is especially valuable as we explore more heavy-duty conditioning, or extending the settle time to longer than 30 seconds.

Comment 2

[Marian Raiciof [:marauder]]

Attachment: Bug 1537944 Add support for up-front profile conditioning in Raptor

Comment 3

[Marian Raiciof [:marauder]]

The patch is not finished yet, is still a work in progress.

Comment 4

[Dave Hunt [:davehunt] [he/him] ⌚BST]

I think we should abort this work and instead consider integrating Tarek's work on generating profiles for use in automation. What do you think, Tarek?

Comment 5

[Tarek Ziadé (:tarek)]

It makes a lot of sense to me yeah

Comment 6

[Tarek Ziadé (:tarek)]

See bug 1562870

Comment 7

[Tarek Ziadé (:tarek)]

I pushed a first version of an empty profile conditionning at https://github.com/tarekziade/conditioned-profile

You can try this command line to generate the profile for firefox:

cp-creator --firefox {FIREFOX_BINARY} --scenarii cold /path/to/profile/dir

Later versions will let us use chrome as well, since the tool uses a webdriver-based client.

In term of dependencies, I think the easiest and simplest way to have that tool available in raptor will be to use a binary release of conditioned-profile like what we're doing with mitmproxy 4, via tooltool or a task fetch. That will avoid having to vendor all the conditioned profile dependencies.

Comment 9

[Tarek Ziadé (:tarek)]

You can start to play with this if you grab the pending patch to use testing/condprof

This is how the API is called (more documentation soon)

>>> from condprof.client import get_profile
>>> get_profile(target_dir, target_platform, scenario, age, customization)

to grab the cold profile on win64, that's a week or less old, and extract it on the current dir:

get_profile(".", "win64", "cold", "days", "default")

Comment 10

[Stephen Donner [:stephend] Not actively reading bugmail]

Attachment: [WIP] Bug 1537944: Add support for up-front profile conditioning in Raptor. r?#perftest, ?tarek

Comment 11

[Stephen Donner [:stephend] Not actively reading bugmail]

After conferring with Rob yesterday, we though morphing this bug to cover initial conditioned-profile support for Firefox desktop, in Raptor (with Browsertime soon to follow, over in bug 1589267), to more-easily break up tasks and start landing code.

Comment 12

[Stephen Donner [:stephend] Not actively reading bugmail]

Attachment: bug1537944.diff

In lieu of my non-working ./mach try and moz-phab (arc) environments right now, I just want to attach this (working, passing w/existing tests) -- current, working state-of-progress, diff.

As soon as my environments are restored, I'll get a Phabricator issue going, once it's also closer to landing: https://phabricator.services.mozilla.com/D47716#1512149.

Comment 13

[Stephen Donner [:stephend] Not actively reading bugmail]

Attachment: Bug 1537944. Add conditioned-profile support for desktop Firefox in Raptor. r?#perftest, ?tarek

Comment 14

[Tarek Ziadé (:tarek)]

Attachment: plug_condprof.patch

This is how you can hook the conditioned profile code as a dependency. It will make both "mach raptor" and "mach python-test" run.

I guess you can use this as a basis for your patch to add next the code in raptor.py

Comment 15

[Tarek Ziadé (:tarek)]

Try run with my patch: https://treeherder.mozilla.org/#/jobs?repo=try&revision=7bd1b771cd68fcd0c523a958e66486f9015ef7ab

(to demonstrate that the imports work in tests and raptor runs)

Comment 16

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to Tarek Ziadé (:tarek) from comment #14)

Created attachment 9103867 [details] [diff] [review]
plug_condprof.patch

This is how you can hook the conditioned profile code as a dependency. It will make both "mach raptor" and "mach python-test" run.

I guess you can use this as a basis for your patch to add next the code in raptor.py

Thx; this fixed it for me; clearing need-info.

Comment 18

[Stephen Donner [:stephend] Not actively reading bugmail]

I've pushed the latest to Phab (https://bugzilla.mozilla.org/attachment.cgi?id=9103064 / https://phabricator.services.mozilla.com/D49970), and here's try: https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=97e44d9d8418c6e81201c30b68bacca0103f8631. Updated details of what's been tested and implemented are in Phabricator.

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6d7fa567184e
Add conditioned-profile support for desktop Firefox in Raptor.  r=perftest-reviewers,tarek,sparky, ?tarek

Comment 20

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/6d7fa567184e