511229 - Find Long term solution for managing Application Dependencies

Status: RESOLVED WONTFIX

(Webtools Graveyard :: Cesium, defect)

Description

[Mark Hahnenberg (:mhahnenberg)]

Cesium (and many other apps, Django or otherwise) have multiple external dependencies (e.g. South for DB schema migrations) that sometimes require multiple steps to get setup on a staging or production server.  When changes are made to these dependencies (e.g. a new version comes out), IT has to go back to these machines and manually move stuff around.  In order to remain flexible and be able to iterate rapidly, some solution should be generated to automate these setup processes to reduce both the amount of time devs spend figuring out how to get their code onto staging/production servers and the amount of time IT spends setting these sorts of things up.

One idea is to use fabric, a Python remote deployment tool, allowing devs to write custom deployment scripts without having to actually touch the staging/production servers.  Any other suggestions are welcome.

Comment 1

[Dave Dash [:davedash, :dd] (assign all bugs to mbrandt)]

Shyamn, Jeremy,

Do you guys have opinions on this?

If I created a fabric script that managed the deployment of an app would this be something that IT would be willing to use?

Fabric lets you run simple commands, e.g.:

fab production install

would run the production command and the install command.  Production might just set some variables, e.g. what hosts to hit, and install would run the right svn commands, etc.

I've got scripts that work very reliably for Django+SVN that we could use.  If this works for Cesium we can later extend it to other projects.

Comment 2

[Jeremy Orem [:oremj]]

I'd rather have this tie in with puppet if possible.  Justdave, what do you think?

Comment 3

[Dave Dash [:davedash, :dd] (assign all bugs to mbrandt)]

Yeah if puppet is what you guys are using, we should probably just start packaging up all web apps with puppet - I hear its a steep learning curve, but its good.

Comment 4

[Dave Miller [:justdave]]

Getting the dependencies packaged up as RPMs makes it really easy for puppet to deploy them (because then it can just pull them from our repo and install), and it also makes them easy to upgrade/remove later.  I'm happy to help with building RPMs if needed.  But puppet can also run scripts and whatnot, so if you have an installer script or somesuch, it can do that, too (it'd need a way to check and see if it's already been run or not though, because puppet does everything in its list every time it runs).  Do be aware that the web clusters generally don't have access to the Internet, so stuff like SVN pulls from upstream usually won't work.

In general you're probably not going to have to worry about telling puppet what to do, we'll take care of that part if you just give instructions for what needs to be installed.

(In reply to comment #0)
> When changes
> are made to these dependencies (e.g. a new version comes out), IT has to go
> back to these machines and manually move stuff around.

We stopped having to screw with this when we got serious about setting up puppet.  Anything new that goes into the production web servers gets configured in puppet already these days.  This is a recent change in the last month or so though, which is probably why you're used to us having to manually screw with stuff up until now.

So essentially, this is worksforme now, as long as the IT folks doing the setup remember to make puppet do it.

Comment 5

[Jeff Balogh (:jbalogh)]

Cesium is dead.  RESOLVED => EXPIRED