948148 - hg.mozilla.org repository pruning

Status: RESOLVED WONTFIX

(Developer Services :: Mercurial: hg.mozilla.org, defect)

Description

[Ben Kero [:bkero]]

In an effort to save space towards our move to hg using local disk I'd propose to prune some old repositories. In particular are these repositories, their size, and how long it's been since their last change.

Many of the groups will need to be contacted to see if they still care about the code, and releng will need to be consulted to see if they still have procedures that depend on these repositories.

The proposal was to move the unwanted items onto an 'hg-archive' volume in case someone realizes they need something later.

actionmonkey - 2008/07 - 520MB
actionmonkey-tamarin - 2008/07 - 82MB
cvs-trunk-mirror (maybe important for history?) - 07/2009 - 500MB
mobile-browser - 03/2011 - 13MB
penelope - 2010/10 - 3.2MB
tamarin-central - 2010/03 - 63MB
tamarin-tracing - 2008-10 - 83MB
tracemonkey - 2011/07 - 1.1GB
xforms - 2012/02 - 2.5MB

incubator/offscreen - 402MB
incubator/symbian - 322MB
incubator/taskfox -  394MB

l10n/l20n - 2009/07 - never used

MANY things in labs/

projects/2007-configure-rewrite - 2007/08 - 400MB
projects/mccoy - 2009/06 - 1.2MB
projects/firefox-lorentz - 2010/04 - 791MB
projects/addonsmgr - 2010/05 - 453MB
projects/kraken - 2010/11 - 5.4MB
projects/private-browsing - 2011/05 - 1.2GB
projects/places - 2011/07 - 1.2GB
projects/electrolysis - 2011/09 - 485MB
projects/devtools - 2012/02 - 1.4GB
projects/devtools2 - 2011/10 - 910MB
projects/nanojit-central - 2012/01 - 4.7MB
projects/addon-sdk-release - 2012/03 - 19MB
projects/addon-sdk-beta - 2012/03 - 20MB
projects/accessibility - 2012/04 - 492MB
projects/jaegermonkey - 2012/05 - 1.2GB

integration/mozilla-inbound-dead - 2013/05 - 1.1GB (pushlog broke once, just a dup)

releases/comm-miramar - 2011/09 - 153MB
releases/mobile-1.1 - 2010/06 - 12MB
releases/mobile-2.0 - 2011/04 - 9.7MB
releases/mozilla-2.1 - 2011/04 - 1.2GB
releases/mozilla-2.0 - 709MB
releases/comm-1.9.1 - 97MB
releases/comm-2.0 - 153MB
releases/mobile-5.0 - 2.9MB
releases/mozilla-miramar - 805MB
releases/mozilla-1.9.1 - 544MB
releases/mobile-6.0  - 3.9MB
releases/mozilla-mobile-6.0 - 709MB
releases/comm-miramar - 153MB
releases/mozilla-mobile-5.0 - 1.2GB
releases/comm-1.9.2 - 103MB

weave-l10n/ - the whole thing - newest change was 2012/11, before that 2011/07 - 6.3MB

Comment 1

[Justin Wood (:Callek)]

For clarity when you say "prune" do you mean "rm -rf $1" or something else?

I can see many of those being marked/setup as read-only and not actively handled/managed beyond hgweb/clone-over-ssh, but I can't easily see most of these being removed.

Comment 2

[Ben Kero [:bkero]]

I mean like I state in the third paragraph, wherein these repositories are moved onto a backup archive somewhere never to be seen again -- unless requested.

This is an effort to save disk space since currently the hg web mirrors are running at 95% disk full with 16GB available.

Comment 3

[Ben Kero [:bkero]]

Another thing to be considered would be to archive user repositories of users who are not with us anymore. Do we have a policy on how long we retain/make available for those?

Comment 4

[Corey Shields [:cshields]]

Hal is there anything you can do to help coordinate ownership of these repos so we can move this forward?  Thanks!

Comment 5

[Axel Hecht [:Pike]]

This is actually tricky for the l10n dashboard automation.

That doesn't expect repositories to go away, so we'll need to figure out what to do with that.

For at least some of those repos, it might be just fine to throw the l10n dashboard data into a backup and remove it. Not sure if that's for all of that, and I'm also not sure how to actually have a conversation about that. Doesn't feel like I'm the only person that should make the call, but I also don't know if anyone else cares.

I'll attach a list of repos the l10n dashboard knows about in a second. There's more food for thought in there, too.

Comment 6

[Axel Hecht [:Pike]]

Attachment: repos known to the l10n dashboard, don't remove without coordination with l10n

Comment 7

[Hal Wine [:hwine] use NI!]

Based on offline chats with cshields, and in similar light as comment 5, there are not any repos under release/* gaia/* integration/* that can go completely offline.

Whether certain repos in projects/* can go offline is up to the owners of those repos (or their descendants).

However, not all of these repositories need to be given the same high service level as our active ones. There are repositories that could be made read-only, and not served in an HA fashion. Ideally, the URL would remain the same (symlinks to the rescue?), or the "old" url(s) should serve a page on how to find the archive location.

Let me know if that is an option we can pursue.

Comment 8

[Ben Kero [:bkero]]

Got permission from jorendorff to move to the archive: actionmonkey actionmonkey-tamarin tamarin-central tamarin-tracing tracemonkey, so they were moved to /repo/hg/retired_repos/, then removed from hgweb.config, then deleted from the hgweb hosts.

Comment 9

[Ben Kero [:bkero]]

Talked to robcee about removing devtools and devtools2 projects, so done

Comment 10

[Ben Kero [:bkero]]

Talked to luke about removing projects/jaegermonkey and projects/nanojit. No problems there.

Comment 11

[Ludovic Hirlimann [:Usul]]

Mark can you comment on :

releases/comm-miramar
releases/comm-1.9.1 
releases/comm-1.9.2
releases/comm-2.0

I believe penelope is a dead project but ain't sure whom to check with (Gerv looked at it not too long ago so he might be a good candidate).

Comment 12

[Axel Hecht [:Pike]]

Those are referenced by the l10n infrastructure still, and need code/database changes on my side to go away.

Comment 13

[Mark Banner (:standard8)]

(In reply to Ludovic Hirlimann [:Usul] from comment #11)
> releases/comm-miramar
> releases/comm-1.9.1 
> releases/comm-1.9.2
> releases/comm-2.0

As Alex said, these are still referenced by the L10n infra. Also per comment 7 these shouldn't really go away, as they are effectively release archives.

mxr.mozilla.org also references some of these repo, and it is likely socorro might as well.

They could certainly move to a lower service level space, but I'd kinda expect all the same urls to work etc (referencing old repos can be useful now and again).

Comment 14

[Ryan Watson [:w0ts0n]]

hgweb5.dmz.scl3.mozilla.com alerting for low disk.

Comment 15

[Axel Hecht [:Pike]]

In bug 1003132, I modified the l10n automation to stop polling all repositories it knows about. I flagged repos that I don't care about, and if we were to be archived, it won't break the l10n automation anymore.

I didn't investigate if any of these have other stakeholders, fwiw.

I stopped polling the following repository trees:

releases/gaia-l10n/v1_0_1
releases/gaia-l10n/v1_1
releases/l10n-miramar
releases/l10n-mozilla-1.9.1
releases/l10n-mozilla-1.9.2
releases/l10n-mozilla-2.0
testpilot-l10n
weave-l10n

I also stopped polling the following individual repos:

mobile-browser
projects/firefox-lorentz
releases/comm-1.9.1
releases/comm-1.9.2
releases/comm-2.0
releases/comm-miramar
releases/mobile-1.1
releases/mobile-2.0
releases/mozilla-1.9.1
releases/mozilla-1.9.2
releases/mozilla-2.0
releases/mozilla-2.1
releases/mozilla-miramar

Comment 17

[Kendall Libby [:fubar] (he/him)]

hgweb7 hit 95% today while resetting cedar. need to do something Real Soon Now.

Comment 18

[Ludovic Hirlimann [:Usul]]

Fri 04:53:28 PDT [5185] hgweb7.dmz.scl3.mozilla.com:Disk - All is CRITICAL: DISK CRITICAL - free space: / 15396 MB (5% inode=56%)

Comment 19

[Ben Kero [:bkero]]

I've removed the repositories found in comment 15.

Comment 20

[Ed Morley [:emorley]]

(In reply to Ben Kero [:bkero] from comment #19)
> I've removed the repositories found in comment 15.

If you're looking for some more targets, I've had a quick glance and I some of the following look promising (but do double-check with owners)...

Repos that were temporary backups, so can be removed now:
https://hg.mozilla.org/projects/holly-new/ (ask #releng, this is a rentable repo backup)

Repos that were imported/merged into another, so may not be required any more:
https://hg.mozilla.org/mobile-browser/ (ask :mfinkle ; can't remember if the history was preserved during import)

Repos that are clones of mozilla-central & but are no longer hooked up to releng automation and no recent commits:
https://hg.mozilla.org/projects/accessibility/ (ask :MarcoZ or :tbsaunde)
https://hg.mozilla.org/projects/ionmonkey/ (speak to the js team)
https://hg.mozilla.org/projects/places/ (ask :mak)
https://hg.mozilla.org/projects/private-browsing/ (ask :ehsan or :jdm)
https://hg.mozilla.org/projects/profiling/ (ask :ehsan)
https://hg.mozilla.org/projects/2007-configure-rewrite/ (ask :ted)

Empty repos (would be nice to remove to clean up the listings on hg.m.o):
https://hg.mozilla.org/tbbuild/
https://hg.mozilla.org/automation/fennecpm/
https://hg.mozilla.org/automation/firebug/

Repos possibly moved to github:
https://hg.mozilla.org/labs (I don't know these well enough to say for sure for each - but I think you may be able to remove a bunch of these after speaking to owners - last commits quite a while ago and https://github.com/mozilla?query=weave)

(https://github.com/mozilla?query=addon-sdk ; ask :erikvold)
https://hg.mozilla.org/projects/addon-sdk-beta/
https://hg.mozilla.org/projects/addon-sdk-jetperf-tests/
https://hg.mozilla.org/projects/addon-sdk-release/

Also perhaps the deprecated/* repos under here?
https://hg.mozilla.org/services

Comment 21

[Ryan Watson [:w0ts0n]]

More warnings:
Thu 06:43:23 PDT [5115] hgweb7.dmz.scl3.mozilla.com:Disk - All is CRITICAL: DISK CRITICAL - free space: / 14605 MB (5% inode=56%): (http://m.mozilla.org/Disk+-+All)

Comment 22

[david garvey:dgarvey]

hgweb5.dmz.scl3.mozilla.com reach 90% today which is a warning.

Comment 23

[david garvey:dgarvey]

Guys,

hgweb3.dmz.scl3.mozilla.com has just reached critical 95%.

Comment 24

[Ryan Watson [:w0ts0n]]

nagios-scl3	 Thu 02:13:05 PDT [5021] hgweb3.dmz.scl3.mozilla.com:Disk - All is WARNING: DISK WARNING - free space: / 16144 MB

Comment 25

[Ben Kero [:bkero]]

Back to the queue