859340 - [mkt] Move Netapp NFS shared storage to S3

Status: RESOLVED WONTFIX

(Cloud Services :: Operations: Marketplace, task, P3)

Description

[Jason Thomas [:jason]]

As part of the Move Marketplace to Dedicated BU, we will need to move off the Netapp NFS shared storage.

Tasks:

1. Create S3 bucket and user for zamboni.
2. Sync NFS content to S3 bucket.
3. Modify CDN origin to point to S3 bucket.
4. Modify zamboni configuration to use S3, coordinate with devs.
5. Test!

I would like to begin on -dev this week, I will coordinate in #remora.

Comment 1

[Jason Thomas [:jason]]

-dev content is synced to s3 bucket, https://github.com/mozilla/zamboni/pull/733 for settings changes.

Comment 2

[Jason Thomas [:jason]]

@clouserw | jason: can you describe changes for krupa?  does that make us use S3 instead of our CDN?

When zamboni save files they go to s3 instead of the local fs; both app uploads and screenshots, and any other content.

S3 will become the origin of our CDN. In production our CDN hostname will rename the same, just pointed to AWS S3 instead.

I think we will need to QA tasks that write to file storage. @clouserw for specifics on what needs to be QA.

Comment 4

[Jason Thomas [:jason]]

Instead of breaking -dev, we are going to use -altdev. https://github.com/mozilla/zamboni/pull/740

Comment 5

[Jeremy Orem [:oremj]]

Let's revisit this once altdev is working again.

Comment 6

[Wil Clouser [:clouserw]]

Rumor has it this is making too many requests to S3 and we're not sure why.  Jason - can you fill us in on all of the details you have?

Comment 7

[Greg Cox [:gcox]]

As part of phx1exit under bug 1227213, all AMO volumes have been filer-unmounted and will be shredded.

Comment 8

[Jason Thomas [:jason]]

marketplace will stay on cloud nfs storage for now.