1084003 - Attachment and wiki database migrations are kaputt

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: Code Cleanup, defect)

Description

[Jannis Leidel [:jezdez]]

What did you do?
================
1. added a new field to the wiki revision model
2. tried to create a new migration for it

What happened?
==============
The generated migration file contained previously already applied migrations (https://gist.github.com/jezdez/65e34975307c22b3af74).

The commit https://github.com/mozilla/kuma/commit/58093b04fbf56de7f679f400f4bb681f393cf76d moves some of the wiki migrations into the attachments app, which shouldn't have happened, as the migrations were already applied, which also completely ignores the fact that the dict at the bottom of a migration file represents the state of all models in that app and the related models of that app.

In other words, the migrations applied for the new attachements app are wrong. The migrations of the wiki app are broken.

What should have happened?
==========================
Instead the attachments app should have gotten an empty initial migration that would have depended on the last wiki migration (http://south.readthedocs.org/en/latest/dependencies.html). The name of the attachment models in the wiki migrations should have been renamed to match the new app name.

Is there anything else we should know?
======================================

Comment 1

[Luke Crouch [:groovecoder]]

:jezdez - this "blocker" bug is paging me every 30 minutes. Do we need to drop everything and fix this?

Comment 2

[Māris Fogels [:mars] (please needinfo)]

Should we declare a push freeze?  (Is there an off-button for the push bot we can use to enforce this?)

Comment 3

[Luke Crouch [:groovecoder]]

This doesn't break existing environments, as they don't try to run the wiki migrations. So this doesn't break pushes. It's really annoying for new vm's though. :/

:jezdez - I didn't have time to fix this today. If you're up for poking at it tomorrow, feel free to take this bug from me.

Comment 4

[Jannis Leidel [:jezdez]]

It "blocks" development, so I'm not sure how else I would interpret the "blocker" label here. Forgive me if I made Bugzilla send you notifications, I wasn't aware of a setup. I think it'd be great if you could explain what each level triggers so that I don't accidentally raise flags without a real issue.

Comment 5

[Luke Crouch [:groovecoder]]

"blocker" is the only level that sends notifications, and it only does so until the bug is assigned. Once it's assigned, it doesn't page anymore. :mars - did we write the bug notification policy down anywhere?

Comment 6

[[github robot]]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/b2c43c0a92cca504e81b4ea2fb7ea4c205c6fc3d
bug 1084003 - require two peer review and at least one owner for database migration changes

https://github.com/mozilla/kuma/commit/0aa20a149999bcc3bd81270fcc1c8f60553a2198
Merge pull request #2835 from jezdez/migration-reviews

bug 1084003 - require two peer review and at least one owner for database migration changes

Comment 7

[Māris Fogels [:mars] (please needinfo)]

I don't think it has been writen it down.  FWIW, blocker is fine to use if it blocks further development, but under the current rules for the pager, this bug does have to be assigned to someone if it's that severe (even if we won't be working on it for a while).  FYI, severity:critical will page people, too.

Comment 8

[MDN Team (:mdn-dev)]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/43bff6d0cae43dba19d04dc6e9d18bf311a276a2
fix bug 1084003 - repair migrations after attachment split

This moves back three of the original wiki migrations into the wiki app and adds a new initial migration to the attachments app that renames the tables.
It also backfills the south history table to remember those three migrations (since they are still effectively applied).
The wiki app has a new empty migration that only depends on the last migration of the attachements app, so that migrating the wiki app will run the attachements app migration first.

This also gets rid of old schematic migrations that have been applied before by running the schematic command.
Some new South migrations were added as the result of that (as replacements):

  - a new data migration in the devmo app that tries to create a default site object.
  - an initial and a data migration for the feeder app (both faked during deploy)

The migration in the users app that deleted the old and unneeded user profile related tables is not handled by exception catching, for initial setups of an empty database.

The script that is run on the dev server during deploy has been modified to run the appropriate migrate command to reset the database there.

The chief deploy script has been modified to *only* run the appropriate migrations when the deployed git tag is "attachments-split-cleanup" since chief runs the script from disk, not after it has pulled from git.

The puppet manifests have *not* been updated and instead migration steps will be provided via email to the users with a current develoment environment. The steps are:

	python2.6 vendor/src/schematic/schematic migrations/
	python2.6 manage.py migrate feeder --delete-ghost-migrations --fake --noinput
	python2.6 manage.py migrate wiki --noinput
	python2.6 manage.py migrate --noinput

https://github.com/mozilla/kuma/commit/20ea79a4acbb8906028186ac59970619bda110c7
Merge pull request #2850 from jezdez/bug1084003-2

fix bug 1084003 - repair migrations after attachment split