Closed Bug 1409046 Opened 2 years ago Closed 2 years ago

Prevent running pip install on every ./mach talos-test executions

Categories

(Testing :: Talos, enhancement)

enhancement
Not set

Tracking

(firefox59 fixed)

RESOLVED FIXED
mozilla59
Tracking Status
firefox59 --- fixed

People

(Reporter: ochameau, Assigned: ochameau)

Details

(Whiteboard: [PI:January])

Attachments

(1 file)

Everytime you run:
  ./mach talos-test --activeTests damp

A bunch of "pip install" are executed first:
  pip install --timeout 120 pip>=1.5
  pip install --timeout 120 psutil>=3.1.1
  pip install --timeout 120 mozsystemmonitor==0.3
  pip install --timeout 120 jsonschema==2.5.1
  pip install --timeout 120 functools32==3.2.3-2
  pip install --timeout 120 blobuploader==1.2.4
  pip install --timeout 120 -r /mnt/desktop/gecko/testing/config/mozbase_requirements.txt
  pip install --timeout 120 -r /mnt/desktop/gecko/testing/talos/requirements.txt


It ends up slowing down any contribution to talos. All these pip install should ideally be done once: on first run, after clobber or less frequently than everytime.

Also, may be calling it only twice would already speed this up:
  pip install --timeout 120 pip>=1.5
  pip install --timeout 120 -r /mnt/desktop/gecko/testing/talos/requirements.txt

I imagine we could put all dependencies into talos/requirements.txt?
See this file which pull another requirements.txt via "-r ":
  http://searchfox.org/mozilla-central/source/testing/config/telemetry_tests_requirements.txt#1
Whiteboard: [PI:October]
Whiteboard: [PI:October] → [PI:November]
Whiteboard: [PI:November] → [PI:January]
Or better, we could prevent having to re-setup the whole virtualenv at each run?
Any suggestion on how to improve talos startup speed?
I'm willing to contribute, but it is hard to find my way through the whole codebase.
$ ./mach talos-test --activeTests damp --cycles 1 --tppagecycles 1 --subtests cold.inspector

Running this command taks 26s on which 18s (70%) is spend doing preliminary python setup and only 7s (30%) actually opening Firefox and running the test.
It makes any contribution to Talos test scripts very annoying.

Would you be able to mentor me to look into that?
Here is a patch that fixes this locally, it surely introduces issues if you don't clobber and update pip dependencies, but this fix all slow startup issues!

Let's see if that breaks try runs:
  https://treeherder.mozilla.org/#/jobs?repo=try&revision=19961875495021c96123092e4f3b525e422112b1
Comment on attachment 8941362 [details]
Bug 1409046 - Prevent re-creating virtualenv everytime ./mach talos is run.

https://reviewboard.mozilla.org/r/211680/#review217474

overall this looks like the right direction and not too hacky...lets see what rwood has to say
Attachment #8941362 - Flags: review?(rwood)
Comment on attachment 8941362 [details]
Bug 1409046 - Prevent re-creating virtualenv everytime ./mach talos is run.

https://reviewboard.mozilla.org/r/211680/#review217498

LGTM, I am bit concerned by the talos virtualenv falling out of date when building without clobber. However I don't think the requirements are updated very often so this issue would probably be pretty rare.

I think we should give it a try, and if we find devs are running into issues because of an outdated talos virtualenv we can reconsider this. Thanks!
Attachment #8941362 - Flags: review?(rwood) → review+
Thanks for the fast review!
Assignee: nobody → poirot.alex
Pushed by apoirot@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f46f4e29d4a0
Prevent re-creating virtualenv everytime ./mach talos is run. r=rwood
https://hg.mozilla.org/mozilla-central/rev/f46f4e29d4a0
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla59
You need to log in before you can comment on or make changes to this bug.