1457816 - S3 expiration policy for Firefox & mobile nightlies

Status: VERIFIED FIXED

(Cloud Services :: Operations: Product Delivery, task)

Description

[Nick Thomas [:nthomas] (UTC+12)]

Could you please add an expiration policy which removes keys once they are 10 days old from these bucket+prefix combinations:

net-mozaws-prod-delivery-firefox   /pub/firefox/nightly/latest- 
net-mozaws-prod-delivery-firefox   /pub/firefox/nightly/partials

net-mozaws-prod-delivery-archive   /pub/mobile/nightly/latest-

Previous key versions can be removed after 5 days, as these are heavily overwritten but we fail forward.

Comment 1

[Jeremy Orem [:oremj]]

I've added these rules to the stage bucket. Let's give it a few days, then let me know if everything looks good.

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

Hmm, I'm not sure what was present on stage before the rules were added. Looks like no partials directory at least. Do we have bucket logging enabled ? Looks like we'd get S3.CREATE.DELETEMARKER and S3.EXPIRE.OBJECT operations (https://docs.aws.amazon.com/AmazonS3/latest/dev/lifecycle-and-other-bucket-config.html)

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

ni for the bucket logging question in comment #2.

Comment 4

[Jeremy Orem [:oremj]]

Yes, we do have bucket logs enabled. I'll check it out, but it probably won't be until after the all hands.

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

Great! With the logging telling us what the expiration is doing, and versioning providing a restoration path if things go wrong, then I'm pretty happy we'll be OK.

Comment 6

[Jeremy Orem [:oremj]]

I've turned these rules on in production.

Comment 7

[Nick Thomas [:nthomas] (UTC+12)]

Looking good to me, thanks.