1145300 - Nightly build artifacts should never expire on s3

Status: RESOLVED WONTFIX

(Release Engineering :: Applications: MozharnessCore, defect)

Description

[Michael Shal [:mshal]]

We'll want to keep nightly build artifacts around forever, or as close as reasonably possible to "forever" that we can get. Right now all artifacts are set to expire after 1 year.

Comment 1

[[e-mail] (:bhearsum)]

I don't think we want to keep _all_ nightly build artifacts around forever. There's no reason to keep MARs around for a very long time, since their contents are the same as the installers, but in a less useful format for historical purposes.

There might be other things too, I'm having trouble finding the cronjobs that clean things up on FTP right now...

Comment 2

[[e-mail] (:bhearsum)]

I got a hold of the clean-up cronjobs for FTP today. Here's the summary of what we delete from /pub/mozilla.org/firefox/nightly, excluding anything in latest-$branch directories:
* Windows installers and XPIs from all platforms that are 6 months or older
* All MAR files and test packages that are 1 month or older
* All localized builds that are more than 5 days old

There's also a few other weird things that happen that we may need to handle:
* We have a script that grabs a debug build from tinderbox-builds and archives it in nightly once per day. I think this is to give us a permanent nightly archive of debug builds without doing extra builds every night.
* We clean up the "latest-$branch" dirs after every uplift because they end up with two sets of files (one with the previous version, one with the current). I'm not sure if we'll have an equivalent of these in the s3 world.
* A bunch of clean-up of tinderbox-builds.

We don't necessarily need to cargo cult all of the above - I just wanted to echo it as a starting point.

Comment 3

[Michael Shal [:mshal]]

Nightly build artifacts are preserved for some time by Beetmover (eg: bug 1282188, bug 1308042, and many others), so we don't need to change the index for them explicitly. So this is a WONTFIX.

Comment 4

[Michael Shal [:mshal]]

Bug 1317747 is also relevant for beetmover for those interested.