632716 - Buildbot masters should use stable versions of Python modules

Status: RESOLVED WORKSFORME

(Infrastructure & Operations Graveyard :: CIDuty, task, P4)

Description

[Rail Aliiev [:rail]]

From time to time we hit problems installing latest versions of Python modules on masters. Sometimes we may have a situation, when masters use not exactly the same version of a particular module (harder to debug a problem).

It woul be great to have exactly the same modules on all masters.

We could use pip' http://pip.openplans.org/#freezing-requirements feature.

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

We should also be careful not to make these installs dependent on downloading things from pypi -- the freezing accomplishes fixed versions, but will still download things from the wilds of the internet.

Comment 2

[John Ford [:jhford] CET/CEST Berlin Time]

could we also download the tarballs and use their setup.py script?

Comment 3

[Chris AtLee [:catlee]]

(In reply to comment #2)
> could we also download the tarballs and use their setup.py script?

we should have locally whatever eggs or tarballs pip requires

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

Sure, but there's no need to go that far.  A directory of downloaded tarballs, along with the right options to pip, should do the trick.

 pip --find-links=/path/to/tarballs --no-index install buildbot

Comment 5

[Chris AtLee [:catlee]]

(In reply to comment #4)
> Sure, but there's no need to go that far.  A directory of downloaded tarballs,
> along with the right options to pip, should do the trick.
> 
>  pip --find-links=/path/to/tarballs --no-index install buildbot

It would be nice to have these checked in somewhere, in case the remote location goes away at some point.

Comment 6

[Dustin J. Mitchell [:dustin] (he/him)]

My plan is to put these in the puppet-files svn repository, although that hasn't been created yet (bug 632716)

Comment 7

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Found in triage.

Anything left to do here, or is this superseded by the recently-completed python upgrade work?