1102686 - move "old try archives" from people to somewhere else (S3? static cluster?)

Status: RESOLVED FIXED

(Developer Services :: General, task)

Description

[:Atoll]

per :hwine, there are a bunch of "old try archives" on people:~hwine/ that should be given a more permanent home somewhere else (with a redirect, if that's appropriate).

filing for hwine triage.

Comment 1

[Hal Wine [:hwine] (leaving Jan 17, ping on slack if needed)]

I hope to get to this post Mozlandia - would like to leave here so it's not forgotten. the NI will keep it on my radar.

Comment 2

[Hal Wine [:hwine] (leaving Jan 17, ping on slack if needed)]

Greg -- the primary driver for these is bug 1017253, which I definitely don't want to reopen. I know you're working on massive improvements to that issue.

My question is twofold:
 a) will your solution include anything for past history? (thus making
    these tarballs redundant)

 b) If not, does dev services already have an S3 bucket that
    could/should hold these?

If the answer to both is "no" I'll find some space within releng's
systems.

Comment 3

[Gregory Szorc [:gps]]

I plan to backfill the archived try repos into the headless repository. When complete, that will make the archived Try repos obsolete.

We don't currently have an S3 bucket set aside from the Try bundles. That is pretty high on my TODO list.

If you need to move these off people, why don't you move them into the NFS mount on hgssh? How about /repo/hg/bak/?

Comment 4

[Hal Wine [:hwine] (leaving Jan 17, ping on slack if needed)]

(In reply to Gregory Szorc [:gps] from comment #3)
> I plan to backfill the archived try repos into the headless repository. When
> complete, that will make the archived Try repos obsolete.

Nice - I'm assuming that is likely a Q2 goal, so I just need storage for 6 months.

> If you need to move these off people, why don't you move them into the NFS
> mount on hgssh? How about /repo/hg/bak/?

They need to remain web retrievable, and (afaik) we don't have (and shouldn't have, imo) non-repository URL space served by hg webheads.

For 6 months, it may be reasonable to leave them where they are. I just get pinged everytime space is an issue on people ;)

Comment 5

[Greg Cox [:gcox]]

(In reply to Hal Wine [:hwine] (use NI) from comment #4)
> I'm assuming that is likely a Q2 goal, so I just need storage for 6 months.

Ding goes a timer.  This '6 months' was written 10 months ago.
Are the files already moved / going to be moved (blocker bug?) / obsolete and deletable?

Comment 6

[Hal Wine [:hwine] (leaving Jan 17, ping on slack if needed)]

:gps - I don't see any need to keep these, do you?  If you agree, I'll rm them, otherwise I'll move to the vcs-archive area.

What's there now:
[[e-mail]3 ~]$ ll -h public_html/try_history/*
-r--r--r-- 1 hwine hwine 1.5G Jun  5  2014 public_html/try_history/try-reset-2014-02-22-1225.tgz
-r--r--r-- 1 hwine hwine 3.6G Aug 14  2014 public_html/try_history/try-reset-2014-08-13-1826.tar.bz2

Comment 7

[Gregory Szorc [:gps]]

I'd like to keep them and eventually import them into the headless try repo -- for posterity because why not.

Somewhere on hgssh1 there is a directory of old try "backups." I can't find the path right now because I'm locked out of the VPN. That or S3 would be an appropriate location for these archives.

Comment 8

[:Atoll]

Is this still needed? If so, I can try to help out.

Comment 9

[:Atoll]

Hi folks! This bug is *really* old, and I'm sorry to reawaken it, but it seemed important to everyone back at the time, so it'd be a shame to just idle the bug out without checking in first.

Is this something that y'all still need and wish to see done?

Comment 10

[Gregory Szorc [:gps]]

We can either copy them to hgssh4:/repo/hg/bak/old_try_repos (or similar directory name) or upload them to an S3 bucket. I'd prefer the latter because it is likely cheaper than space on a Netapp volume.

I would do this if I had access to the files in ~hwine.

Comment 11

[:Atoll]

(hwine, i can grant his access with your auth)

Comment 12

[:Atoll]

with your *permission. (sudo qualifies as auth, confusion not intended.)

Comment 13

[Hal Wine [:hwine] (leaving Jan 17, ping on slack if needed)]

Go for it! Glad to see them disappear!

ni me again when done, and I can delete them.

Comment 14

[:Atoll]

Moved ~hwine/public_html/try_history/ to ~gszorc/public_html/try_history/ with appropriate permissions updates. n? :gps to migrate them away from people; okay to RESO FIXE this bug once they're deleted.

Comment 15

[Gregory Szorc [:gps]]

Uploaded archives to S3 at moz-vcs-archive/try-archives (part of moz-devservices account).

Deleted directory from my people account.