1005946 - Test migration process on production database

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Engagement, task)

Description

[Osmose [:osmose, :mkelly]]

Before we actually go live, I think we should test out the migration and nightly update process on a copy of the production database to see how long it will take and what errors might pop up. 

I suggest that we do this on the staging server since it's closer to what prod will be like. This may depend on when QA is okay with us munging the staging server as they're doing some QA on it as soon as we finish the stage update. Once we're satisfied we can restore the media directory and database, which should put stage back to normal.

Bug 1005212 describes the update procedure: we'll have to overwrite the stage DB with the prod data, then set up a mirror of the prod data for the import process. We'll also have to copy the data from prod's media directory to stage before running the migrations, so that the image files it's expecting are on the filesystem.

Once we've migrated the data and checked to see if it looks legit (a short manual glance is enough to make me happy), we should then run ./bin/nightly_update.sh to see how much time the nightly update will take. Then another quick look to check the database data, and we should be good!

Comment 1

[[github robot]]

Commit pushed to master at https://github.com/mozilla/affiliates

https://github.com/mozilla/affiliates/commit/89d330063a46b242722227835a2c91724bb274fb
Bug 1005946: Ignore links that point to nonexistent users.

Comment 2

[Osmose [:osmose, :mkelly]]

We ran the migration and nightly_update on a copy of the prod data, and both commands only took a few minutes to complete, so we're good on this point. A few things we learned:

- The celery_tasksetmeta table may not exist, but is listed for deletion in the instructions in bug 1005212. It's fine to just ignore it not existing.

- The migrations may fail on a banner migration that updates the width and height fields on banner variations. I suspect this is due to multidb making the read operations go to the slave database, which may not reflect the changes from the migrations immediately. Re-running the migrations seems to fix this.

- Once we're done, we need to create a userprofile object for the admin user and invalidate the user cache if the admin wants to log into the site.

Comment 3

[Chris Turra [:cturra]]

migration testing in dev/stage complete and push to prod now done :)