1409046 - Prevent running pip install on every ./mach talos-test executions

Status: RESOLVED FIXED

(Testing :: Talos, enhancement)

Description

[Alexandre Poirot [:ochameau]]

Everytime you run:
  ./mach talos-test --activeTests damp

A bunch of "pip install" are executed first:
  pip install --timeout 120 pip>=1.5
  pip install --timeout 120 psutil>=3.1.1
  pip install --timeout 120 mozsystemmonitor==0.3
  pip install --timeout 120 jsonschema==2.5.1
  pip install --timeout 120 functools32==3.2.3-2
  pip install --timeout 120 blobuploader==1.2.4
  pip install --timeout 120 -r /mnt/desktop/gecko/testing/config/mozbase_requirements.txt
  pip install --timeout 120 -r /mnt/desktop/gecko/testing/talos/requirements.txt


It ends up slowing down any contribution to talos. All these pip install should ideally be done once: on first run, after clobber or less frequently than everytime.

Also, may be calling it only twice would already speed this up:
  pip install --timeout 120 pip>=1.5
  pip install --timeout 120 -r /mnt/desktop/gecko/testing/talos/requirements.txt

I imagine we could put all dependencies into talos/requirements.txt?
See this file which pull another requirements.txt via "-r ":
  http://searchfox.org/mozilla-central/source/testing/config/telemetry_tests_requirements.txt#1

Comment 1

[Alexandre Poirot [:ochameau]]

Or better, we could prevent having to re-setup the whole virtualenv at each run?
Any suggestion on how to improve talos startup speed?
I'm willing to contribute, but it is hard to find my way through the whole codebase.

Comment 2

[Alexandre Poirot [:ochameau]]

$ ./mach talos-test --activeTests damp --cycles 1 --tppagecycles 1 --subtests cold.inspector

Running this command taks 26s on which 18s (70%) is spend doing preliminary python setup and only 7s (30%) actually opening Firefox and running the test.
It makes any contribution to Talos test scripts very annoying.

Would you be able to mentor me to look into that?

Comment 3

[Alexandre Poirot [:ochameau]]

Attachment: Bug 1409046 - Prevent re-creating virtualenv everytime ./mach talos is run.

Review commit: https://reviewboard.mozilla.org/r/211680/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/211680/

Comment 4

[Alexandre Poirot [:ochameau]]

Here is a patch that fixes this locally, it surely introduces issues if you don't clobber and update pip dependencies, but this fix all slow startup issues!

Let's see if that breaks try runs:
  https://treeherder.mozilla.org/#/jobs?repo=try&revision=19961875495021c96123092e4f3b525e422112b1

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8941362 [details]
Bug 1409046 - Prevent re-creating virtualenv everytime ./mach talos is run.

https://reviewboard.mozilla.org/r/211680/#review217474

overall this looks like the right direction and not too hacky...lets see what rwood has to say

Comment 6

[Robert Wood [:rwood]]

Comment on attachment 8941362 [details]
Bug 1409046 - Prevent re-creating virtualenv everytime ./mach talos is run.

https://reviewboard.mozilla.org/r/211680/#review217498

LGTM, I am bit concerned by the talos virtualenv falling out of date when building without clobber. However I don't think the requirements are updated very often so this issue would probably be pretty rare.

I think we should give it a try, and if we find devs are running into issues because of an outdated talos virtualenv we can reconsider this. Thanks!

Comment 7

[Alexandre Poirot [:ochameau]]

Thanks for the fast review!

Comment 8

[Alexandre Poirot [:ochameau]]

Comment on attachment 8941362 [details]
Bug 1409046 - Prevent re-creating virtualenv everytime ./mach talos is run.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/211680/diff/1-2/

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f46f4e29d4a0
Prevent re-creating virtualenv everytime ./mach talos is run. r=rwood

Comment 10

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/f46f4e29d4a0