869270 - vagrant/puppet cannot find sqlalchemy

Status: RESOLVED FIXED

(Socorro :: Infra, task)

Description

[Robert Helmer [:rhelmer]]

Vagrant uses Puppet to bootstrap a Socorro VM, including running setupdb_app.py --fakedata

setupdb_app.py now requires sqlalchemy, but it's only in dev.txt ... the current puppet manifests do a "make install" then put /data/socorro/thirdparty on PYTHONPATH which won't work because thirdparty/ only includes dependencies from prod.txt

Perhaps we should switch this over to using a virtualenv?

Comment 1

[Selena Deckelmann :selenamarie :selena]

(In reply to Robert Helmer [:rhelmer] from comment #0)
> Vagrant uses Puppet to bootstrap a Socorro VM, including running
> setupdb_app.py --fakedata
> 
> setupdb_app.py now requires sqlalchemy, but it's only in dev.txt ... the
> current puppet manifests do a "make install" then put
> /data/socorro/thirdparty on PYTHONPATH which won't work because thirdparty/
> only includes dependencies from prod.txt
> 
> Perhaps we should switch this over to using a virtualenv?

Is this really a production dependency? None of the app uses it -- only the database servers and things talking directly to the database. I'm concerned about deploying it everywhere when we really don't need it.

Comment 2

[Robert Helmer [:rhelmer]]

(In reply to Selena Deckelmann :selenamarie :selena from comment #1)
> (In reply to Robert Helmer [:rhelmer] from comment #0)
> > Vagrant uses Puppet to bootstrap a Socorro VM, including running
> > setupdb_app.py --fakedata
> > 
> > setupdb_app.py now requires sqlalchemy, but it's only in dev.txt ... the
> > current puppet manifests do a "make install" then put
> > /data/socorro/thirdparty on PYTHONPATH which won't work because thirdparty/
> > only includes dependencies from prod.txt
> > 
> > Perhaps we should switch this over to using a virtualenv?
> 
> Is this really a production dependency? None of the app uses it -- only the
> database servers and things talking directly to the database. I'm concerned
> about deploying it everywhere when we really don't need it.

No not really, but vagrant/puppet currently expect setupdb_app.py to use production dependencies, which is why I think we should switch that over to using a virtualenv in that environment instead (dev.txt could be installed there)

Comment 3

[Selena Deckelmann :selenamarie :selena]

(In reply to Robert Helmer [:rhelmer] from comment #2)

> No not really, but vagrant/puppet currently expect setupdb_app.py to use
> production dependencies, which is why I think we should switch that over to
> using a virtualenv in that environment instead (dev.txt could be installed
> there)

Ah, right. Sorry -- didn't read to the end there. 

+1 to using a virtualenv!

Comment 4

[Robert Helmer [:rhelmer]]

r? https://github.com/mozilla/socorro/pull/1236

Comment 5

[[github robot]]

Commits pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/a7f90e44f1e682a5e3af734db1a241b3ef59cc39
bug 869270 - use a specially-named virtualenv for setupdb, since it needs dev-only sqlalchemy dep now

https://github.com/mozilla/socorro/commit/e9b9dbfd08ff847713104d05a0ade03405716503
bug 869270 - make virtualenv and use it for setupdb_app

https://github.com/mozilla/socorro/commit/8a0f04f6cbe20360eb3ffcc0c0900ba54f47d709
bug 869270 - make sure the postgres roles are mentioned

https://github.com/mozilla/socorro/commit/b3449fb2f9f0fa7d64aebe26f66d3bf7df1c9955
bug 869270 - backfill_matviews populates product_versions

https://github.com/mozilla/socorro/commit/fc5cf6ab318f95d142f7c50293517e890cb87569
Merge pull request #1236 from rhelmer/bug869270-setupdb-use-virtualenv

Bug869270 setupdb use virtualenv

Comment 6

[Selena Deckelmann :selenamarie :selena]

r+!

Comment 7

[[github robot]]

Commit pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/5e72aa8ffbe60d99812945fabe82a6103c5ffd8f
Bug 869270 - Ignore new socorro-vagrant-virtualenv directory.