Closed Bug 1537944 Opened 1 year ago Closed 3 months ago

Add conditioned-profile support for desktop Firefox in Raptor

Categories

(Testing :: Raptor, task, P1)

Version 3
task

Tracking

(firefox72 fixed)

RESOLVED FIXED
mozilla72
Tracking Status
firefox72 --- fixed

People

(Reporter: rwood, Assigned: stephend)

References

(Blocks 3 open bugs)

Details

Attachments

(2 files, 3 obsolete files)

Add the ability for Raptor to use preconditioned browser profiles. Perf team will provide more details for the profile conditions to use.

Currently the only conditioning Raptor does is wait 30 seconds for the browser to settle. We're considering alternative conditioning strategies such as load an unrelated website. This bug is about investigating if we can avoid conditioning the browser for every test in a suite, and instead to perform the conditioning up-front.

This would involve an initial browser launch, execute the conditioning step (wait 30 seconds, etc), taking a copy of the profile, and then closing the browser. Subsequent browser launches would use a clone of the conditioned profile. This should save us the repeated cost of conditioning the profile, which is especially valuable as we explore more heavy-duty conditioning, or extending the settle time to longer than 30 seconds.

Assignee: nobody → marian.raiciof
Status: NEW → ASSIGNED
Priority: P3 → P1

The patch is not finished yet, is still a work in progress.

Depends on: 1547747
Priority: P1 → P2

I think we should abort this work and instead consider integrating Tarek's work on generating profiles for use in automation. What do you think, Tarek?

Flags: needinfo?(tarek)

It makes a lot of sense to me yeah

Flags: needinfo?(tarek)
Attachment #9063500 - Attachment is obsolete: true
Depends on: 1562870

I pushed a first version of an empty profile conditionning at https://github.com/tarekziade/conditioned-profile

You can try this command line to generate the profile for firefox:

cp-creator --firefox {FIREFOX_BINARY} --scenarii cold /path/to/profile/dir

Later versions will let us use chrome as well, since the tool uses a webdriver-based client.

In term of dependencies, I think the easiest and simplest way to have that tool available in raptor will be to use a binary release of conditioned-profile like what we're doing with mitmproxy 4, via tooltool or a task fetch. That will avoid having to vendor all the conditioned profile dependencies.

Duplicate of this bug: 1576837

You can start to play with this if you grab the pending patch to use testing/condprof

This is how the API is called (more documentation soon)

>>> from condprof.client import get_profile
>>> get_profile(target_dir, target_platform, scenario, age, customization)

to grab the cold profile on win64, that's a week or less old, and extract it on the current dir:

get_profile(".", "win64", "cold", "days", "default")
Assignee: marian.raiciof → stephen.donner
Type: enhancement → task
Priority: P2 → P1
Attachment #9097551 - Attachment description: Bug 1537944: Add support for up-front profile conditioning in Raptor. r?#perftest, ?tarek → [WIP] Bug 1537944: Add support for up-front profile conditioning in Raptor. r?#perftest, ?tarek

After conferring with Rob yesterday, we though morphing this bug to cover initial conditioned-profile support for Firefox desktop, in Raptor (with Browsertime soon to follow, over in bug 1589267), to more-easily break up tasks and start landing code.

Summary: Add support for up-front profile conditioning in Raptor → Add conditioned-profile support for desktop Firefox in Raptor
Attached file bug1537944.diff (obsolete) —

In lieu of my non-working ./mach try and moz-phab (arc) environments right now, I just want to attach this (working, passing w/existing tests) -- current, working state-of-progress, diff.

As soon as my environments are restored, I'll get a Phabricator issue going, once it's also closer to landing: https://phabricator.services.mozilla.com/D47716#1512149.

Attachment #9097551 - Attachment is obsolete: true
Attachment #9102706 - Attachment is obsolete: true

This is how you can hook the conditioned profile code as a dependency. It will make both "mach raptor" and "mach python-test" run.

I guess you can use this as a basis for your patch to add next the code in raptor.py

Flags: needinfo?(stephen.donner)

Try run with my patch: https://treeherder.mozilla.org/#/jobs?repo=try&revision=7bd1b771cd68fcd0c523a958e66486f9015ef7ab

(to demonstrate that the imports work in tests and raptor runs)

(In reply to Tarek Ziadé (:tarek) from comment #14)

Created attachment 9103867 [details] [diff] [review]
plug_condprof.patch

This is how you can hook the conditioned profile code as a dependency. It will make both "mach raptor" and "mach python-test" run.

I guess you can use this as a basis for your patch to add next the code in raptor.py

Thx; this fixed it for me; clearing need-info.

Flags: needinfo?(stephen.donner)
Duplicate of this bug: 1591669
Whiteboard: leave-open
Whiteboard: leave-open
Pushed by sdonner@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6d7fa567184e
Add conditioned-profile support for desktop Firefox in Raptor.  r=perftest-reviewers,tarek,sparky, ?tarek
Status: ASSIGNED → RESOLVED
Closed: 3 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla72
You need to log in before you can comment on or make changes to this bug.