1520769 - Run django migrations on container start on elmo webheads

Status: RESOLVED FIXED

(Webtools Graveyard :: Elmo, enhancement)

Description

[Axel Hecht [:Pike]]

Right now, we're not running any migrations on elmo in AWS.

Doing so is also a bit tricky, as we have different infrastructure on that database.

That's why we most often won't have "atomic" migrations between a10n and elmo.

Trick is to add data first, then code to use data, then remove code using old data, then old data.

Or real bad-ass downtimes (we might need that for some parts).

I've got a patch with a migration (data changes only), and I'm adding ./manage migrate to the docker startup. Seems that socorro does that, too, from what I've found.

Filing a bug to track that decision.

That rides along with a patch to remove old ContentTypes. Django doesn't delete those by default when removing models or apps, so there's a ton of them left over.

Comment 1

[Axel Hecht [:Pike]]

Attachment: PR for data migration and docker changes

Brian (or Miles), can you review the docker changes?

I got an f+ from Matjaz over IRC for the django migration itself.

Comment 2

[Brian Pitts]

"Right now, we're not running any migrations on elmo in AWS."

This isn't true. We run the migrations for django every time we launch a new web server.

https://github.com/mozilla-services/cloudops-deployment/blob/master/projects/elmo/puppet/modules/elmo/manifests/migrate.pp
https://github.com/mozilla-services/cloudops-deployment/blob/master/projects/elmo/puppet/yaml/type/elmo.webapp.yaml#L6

Running it every time a web server launched isn't ideal, and in the migration to kubernetes we'll be able to fix this and have the migrations only run once per webapp deployment.

Adding it to container startup is a step backward.

Comment 3

[Axel Hecht [:Pike]]

Thanks, good that I asked.

Updated the PR.

Comment 4

[[github robot]]

Commit pushed to develop at https://github.com/mozilla/elmo

https://github.com/mozilla/elmo/commit/819b32af58412a15e0b452fe7ca633e1dcf9e79b
bug 1520769, clean up ContentType and permissions in db through migration.

We've taken a couple of migrations that removed apps and models that
didn't remove the content types or the permissions for them. Remove
them now.

The migration depends on that latest migration of all elmo apps with
data, so it's OK to access them virtually in the migration via get_model.

Comment 5

[Axel Hecht [:Pike]]

The patch I had here is now in prod.

The migrations didn't apply on deployment, though, but required some kicking on puppet? Miles, was that a fluke or a bug?

Comment 6

[Axel Hecht [:Pike]]

Comment on attachment 9037196 [details] [review]
PR for data migration and docker changes

Got an r- on the docker parts.

Comment 7

[Miles Crabill [:miles]]

A bug, and one that should now be fixed. The migrations were also not properly applied in stage.

I realized that the way we were configuring migrations to run had an error in logic, so they would never work correctly the first time. We have a fix PR here: https://github.com/mozilla-services/cloudops-deployment/pull/2896.

I think we can probably mark this fixed.

Comment 8

[Axel Hecht [:Pike]]

'k. The migrations we had only really showed up on the admin interface, so I couldn't actually check on stage due the lack of log-in ;-).