Trim rsync modules (May 2012 edition)

RESOLVED FIXED

Status

Release Engineering
General
P2
normal
RESOLVED FIXED
6 years ago
5 years ago

People

(Reporter: nthomas, Assigned: nthomas)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments, 1 obsolete attachment)

(Assignee)

Description

6 years ago
From the motd:
---------------------------------------------------------------------------
            R E L E A S E S - R S Y N C . M O Z I L L A . O R G
---------------------------------------------------------------------------

                   Current module sizes (updated 2012-05-22)

                   mozilla-prereleases: 363GB
                      mozilla-releases: 132GB
                       mozilla-current: 31GB

We're getting close enough to the next merge that we can trim down releases as well as prereleases.
Sea 2.8 and 2.9 can go, I'll get Jens to update our website.
(In reply to Justin Wood (:Callek) from comment #1)
> Sea 2.8 and 2.9 can go, I'll get Jens to update our website.

No action needed for SM since I'm taking care of that as part of website updates nowadays. :-)

jens@holodoc:~/Dropbox/Mozilla/smpo > grep -r 'mirrorred = true' *
src/releases/2.10b2.en.html:[%- mirrorred = true -%]
src/releases/2.9.1.en.html:[%- mirrorred = true -%]
src/releases/index.en.html:[%- mirrorred = true -%]
(Assignee)

Comment 3

6 years ago
Created attachment 627890 [details] [diff] [review]
[svn] initial cleanup

Highlights
* drop all of mobile/ because all downloads happen through Google Play, and we don't have any bouncer entries active for these recent releases/betas anyway
* drop SM 2.8 and 2.9 per earlier comment
* drop FF 3.6.25/26/27, carry only 3.6.28/update/win32 otherwise (for 3.6.12 and older in update loops)
* rewrite a bunch of 'keep only update/win32' excludes more compactly, eg FF 8.0.1 and many others (I'm planning to revisit why we're still carrying these really old releases on mirrors)
* almost no traffic on 10.0.3esr's, drop them from all mirrors
* drop a selection of older betas for FF, TB, SM
* keep only TB 3.1.20/update/win32 for end of branch
Attachment #627890 - Flags: review?(rail)
(Assignee)

Updated

6 years ago
Attachment #627890 - Flags: review?(aki)
(Assignee)

Comment 4

6 years ago
Comment on attachment 627890 [details] [diff] [review]
[svn] initial cleanup

The update/win32 trickery isn't working as expected, I'll redo this after further testing.
Attachment #627890 - Attachment is obsolete: true
Attachment #627890 - Flags: review?(rail)
Attachment #627890 - Flags: review?(aki)
Duplicate of this bug: 759425
(Assignee)

Comment 6

6 years ago
Created attachment 628281 [details] [diff] [review]
[svn] initial cleanup, v2

Swapped back to the verbose way of excluding all but update/win32. Otherwise comment #3 applies.
Attachment #628281 - Flags: review?(rail)
(Assignee)

Comment 7

6 years ago
Comment on attachment 628281 [details] [diff] [review]
[svn] initial cleanup, v2

Aki, just looking for confirmation that we don't need mobile/releases on the mirrors.
Attachment #628281 - Flags: review?(aki)

Comment 8

6 years ago
Comment on attachment 628281 [details] [diff] [review]
[svn] initial cleanup, v2

I think that's fine.
We were talking about releasing official Android builds off our website with the appropriate bouncer entries and AUS updates, but that seems to have died down.
Attachment #628281 - Flags: review?(aki) → review+
Attachment #628281 - Flags: review?(rail) → review+
(Assignee)

Comment 9

6 years ago
Comment on attachment 628281 [details] [diff] [review]
[svn] initial cleanup, v2

Committed revision 39607.
Attachment #628281 - Flags: checked-in+
(Assignee)

Comment 10

6 years ago
Created attachment 628555 [details] [diff] [review]
[svn] followup for mozilla-releases

I committed this as a followup in revision 39618 so that mozilla-releases is in sync with mozilla-prereleases, which means the motd will report the correct size for releases.

We now have 
  mozilla-prereleases down to 184G
  mozilla-releases down to 73G, and we'd expect 35-40G more on Tuesday
Attachment #628555 - Flags: checked-in+
(Assignee)

Updated

6 years ago
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
(Assignee)

Comment 11

6 years ago
I did some Check Now trimming over the weekend for releases only carried on the internal mirrors. Still need to do that for releases on the mirrors, which take longer to clean up.

rsync currently doesn't have the --max-delete argument on them. Still
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 12

6 years ago
I'm disabling these mirrors because they haven't synced recently
* http://mozilla.mirror.ihug.co.nz - last sync April 23, weight 7
* http://mirror.dacentec.com/mozilla - last sync May 3, weight 23
* http://mozilla.fastsoft.net - last sync April 26, weight 88

which gets everything in the attachments down to a weight of 1 and I've flipped Check Now.
Status: REOPENED → RESOLVED
Last Resolved: 6 years ago6 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.