1110808 - Copy MDN files from NetApp to S3 buckets

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Community Platform, task)

Description

[Luke Crouch [:groovecoder]]

To move MDN from SCL3 to AWS, we need to change from NetApp to S3. As part of that, we will need to copy/move MDN files (demo studio uploads, wiki attachments, and static assets) from NetApp to S3 buckets.

Comment 1

[Chris Turra [:cturra]]

:relud is creating these buckets for us. once complete, i will kick off the data sync.

Comment 2

[Chris Turra [:cturra]]

i now have access! data sync to mdn-stage s3 bucket has begun.

Comment 3

[Luke Crouch [:groovecoder]]

Cool, thanks :cturra! That will let us test the code with stage S3 before we do the full switch.

Comment 4

[Chris Turra [:cturra]]

i have completed a first (initial import) pass on this sync job for both stage and prod s3 buckets. will keep this bug open to track the completion and implementation of my sync script to keep this date up to date.

Comment 5

[Chris Turra [:cturra]]

the sync script is complete and on a cron to run every 2 hours. once a timeline for the migration off of netapp is planned, please let me know so i can stop this sync script.

*important, for now, i am running a sync that will delete-removed - which means, any local (nfs) files removed will also be removed in s3. once your code is writing to s3, this will impact you, since the "local" files will no longer be present. lets be sure to co-ordinate that change together :)

:groovecoder - i am tagging a needinfo flag for you here just to confirm you've read my message above ;) feel free to clear it anytime.

Comment 6

[Luke Crouch [:groovecoder]]

Thanks :cturra. I'm leaving the needinfo on to remind myself every week. :)