871798 - Using Virtualenvs for deploying socorro

Status: RESOLVED DUPLICATE of bug 907861

(Infrastructure & Operations Graveyard :: WebOps: Socorro, task)

Description

[Selena Deckelmann :selenamarie :selena]

I'm hoping to get SQLAlchemy 0.8.x supported on socorro database systems. 

It looks like 0.6.0 is what's currently available.

Can someone investigate what it will take to support this?

Comment 1

[Selena Deckelmann :selenamarie :selena]

So, in talking with Rob about this more, he thinks we should revisit using virtualenvs for deployments. 

Are there objections from WebOps regarding the use of virtualenvs? We have a rigged-up version of this in $SOCORRO/thirdparty, but could make better use of a true virtualenv.  

And not have custom libraries deployed that are not the same as our production systems.

Comment 2

[Lonnen :lonnen]

I'm not sure about virtual envs, but I know there is a strong objection to using pip. Vendor allowed devs to vet a lib from pip and then check it in, cosigning that it wasn't malicious.

I've waffled on this issue over the last year. In node, ruby, puppet, etc I'm happy to vendor. Vendor + submodule is a giant pain in python, though.

I think Erik Rose's peep mitigates the risk of installing fresh from pip, which is why I filed https://bugzilla.mozilla.org/show_bug.cgi?id=894493.

Comment 3

[Erik Rose [:erik][:erikrose]]

Second. peep + virtualenv means the only thing we're trusting is Github, which we have the odd habit of trusting. :-) (That's assuming virtualenv doesn't hit the network to download pip or setuptools or anything.) I suppose we could start signing commits to get around that one last hole if we wanted (http://mikegerwitz.com/papers/git-horror-story.html), but I'd say fix the big hole first.

Comment 4

[Brandon Burton [:solarce]]

I don't see any reason not to try this, since we're already using the Jenkins produced tarball as our deployment artifact.

The only impacts I see it having on webops are

1. Some tweaks to the mod_wsgi bits of the Apache configuration
2. Some tweaks to the push script for 'manage.py' invocations to make sure the venv is activated before they are run.

It seems like the actual legwork to make this happen would be done by webtools? If so do we want to move this bug to another component to track progress or make a new bug?

Comment 5

[Robert Helmer [:rhelmer]]

This was actually done over in bug 907861 (and dependent webops bugs for apache changes)