880569 - Write a migration script to publish projects to the MakeAPI

Status: RESOLVED FIXED

(Webmaker Graveyard :: Popcorn Maker, defect)

Description

[Jon Buckley [:jbuck]]

As discussed in bug 878096, we need to write a migration script that will publish all existing Popcorn projects to the MakeAPI so that when users log in, their projects will be visible in the My Projects dashboard.

This will likely be a one-off script so prettiness is not required, just working code. We can import the data from prod into staging for testing purposes when the script is written.

Comment 1

[Chris DeCairos (:cade)]

Attachment: https://github.com/mozilla/popcorn.webmaker.org/pull/53

Le Script.

Comment 2

[Jon Buckley [:jbuck]]

Comment on attachment 761690 [details] [review]
https://github.com/mozilla/popcorn.webmaker.org/pull/53

Looks pretty good! But I think it can be improved with the use of async, and we should definitely considered hard-coding the utils/stores bits, since I'm currently 2/3rds of the way to removing them.

Comment 3

[Chris DeCairos (:cade)]

Comment on attachment 761690 [details] [review]
https://github.com/mozilla/popcorn.webmaker.org/pull/53

Okay, I've used async to ensure that the process of resolving remixes, publishing to the makeAPI and re-saving the make id into the project is done synchronously. I had a bug where I had things being done in parallel and it DDOS'ed my local log in server.

I also removed the use of the utils file, and ripped out the URL generating functions I needed.

Total run time on the 29K rows in the backup data I have took ~6+ minutes

Comment 4

[Chris DeCairos (:cade)]

Attachment: https://github.com/mozilla/MakeAPI/pull/82

This Patch to the MakeAPI supports the fact that almost none of the old projects owners will have already created a webmaker account, this sidesteps around the login server using an environment variable.

we can toggle it on for the migration then restart with it off.

https://github.com/mozilla/MakeAPI/pull/82

Comment 5

[Chris DeCairos (:cade)]

FYI - this script depends on the previous migration scripts being applied to the database. They add thumbnail and makeid columns.

Comment 6

[[github robot]]

Commit pushed to master at https://github.com/mozilla/MakeAPI

https://github.com/mozilla/MakeAPI/commit/55fcfc606c306259646d35e9b80cc205a98317f3
Bug 880569 - Support the mass migration of record with a lazy admin environment var. keep false in most cases.

Comment 7

[[github robot]]

Commit pushed to master at https://github.com/mozilla/popcorn.webmaker.org

https://github.com/mozilla/popcorn.webmaker.org/commit/5fcadbad97d44805b48c10777e9c0048ec25b395
Fix Bug 880569 - Publish Unpublished makes to the Make API