674517 - update mozilla-releases rsync exclude file to not push any betas to mirrors

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[[e-mail] (:bhearsum)]

We haven't been pushing Firefox Betas to the full mirror network, and it looks like Thunderbird and SeaMonkey have been doing the same. To do this, it's required that we update the exclude file before pushing each individual release. We should update the rsyncd files with more general excludes to make this updates less frequent or wholly unnecessary.

We could add (and equivalent rules for other products):
- firefox/releases/*b*

which should block all Betas going forward from hitting the mirrors.

Or if that seems to unsafe, we could add one like:
- firefox/releases/6.0b*

The downside to that is it becomes easy to forget to do.

Anyone have any thoughts on either of this, or alternative ideas?

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

The first suggestion sounds great to me, except that using '*.*b*' would be slightly clearer when we come to do future maintenance.

For T'bird and SeaMonkey's information - when we just use the internal mirrors we have 3 mozilla machines plus a couple of CDNs carrying the files, a total weight of about 2000 in bouncer. Considering we used to serve Firefox betas from ftp.m.o this is lots of capacity.

Comment 2

[[e-mail] (:bhearsum)]

I made this change:
-bash-3.2$ diff -Naur rsyncd-mozilla-releases.exclude /pub/mozilla.org/zz/rsyncd-mozilla-releases.exclude 
--- rsyncd-mozilla-releases.exclude	2011-07-28 08:14:52.000000000 -0700
+++ /pub/mozilla.org/zz/rsyncd-mozilla-releases.exclude	2011-07-28 08:19:22.035721000 -0700
@@ -19,8 +19,6 @@
 - firefox/releases/1.0rc1
 - firefox/releases/1.0rc2
 - firefox/releases/deerpark
-- firefox/releases/1.5b1
-- firefox/releases/1.5b2
 - firefox/releases/1.5rc1
 - firefox/releases/1.5rc2
 - firefox/releases/1.5rc3
@@ -40,9 +38,6 @@
 - firefox/releases/1.5.0.12
 - firefox/releases/latest-1.5
 - firefox/releases/bonecho
-- firefox/releases/2.0b1
-- firefox/releases/2.0b2
-- firefox/releases/2.0b3
 - firefox/releases/2.0rc1
 - firefox/releases/2.0rc2
 - firefox/releases/2.0rc3
@@ -77,11 +72,6 @@
 - firefox/releases/latest-2.0
 - firefox/releases/partners
 - firefox/releases/granparadiso
-- firefox/releases/3.0b1
-- firefox/releases/3.0b2
-- firefox/releases/3.0b3
-- firefox/releases/3.0b4
-- firefox/releases/3.0b5
 - firefox/releases/3.0rc1
 - firefox/releases/3.0rc2
 - firefox/releases/3.0rc3
@@ -114,11 +104,6 @@
 - firefox/releases/3.0.19-real-real/win32
 - firefox/releases/latest-3.0
 - firefox/releases/shiretoko
-- firefox/releases/3.1b1
-- firefox/releases/3.1b2
-- firefox/releases/3.1b3
-- firefox/releases/3.5b4
-- firefox/releases/3.5b99
 - firefox/releases/3.5rc1
 - firefox/releases/3.5rc2
 - firefox/releases/3.5rc3
@@ -144,11 +129,6 @@
 - firefox/releases/3.5.19
 - firefox/releases/latest-3.5
 - firefox/releases/namoroka
-- firefox/releases/3.6b1
-- firefox/releases/3.6b2
-- firefox/releases/3.6b3
-- firefox/releases/3.6b4
-- firefox/releases/3.6b5
 - firefox/releases/3.6rc1
 - firefox/releases/3.6rc2
 - firefox/releases/3.6
@@ -169,32 +149,12 @@
 - firefox/releases/3.6.16
 - firefox/releases/3.6.17
 - firefox/releases/devpreview
-- firefox/releases/4.0b1
-- firefox/releases/4.0b2
-- firefox/releases/4.0b3
-- firefox/releases/4.0b4
-- firefox/releases/4.0b5
-- firefox/releases/4.0b6
-- firefox/releases/4.0b7
-- firefox/releases/4.0b8
-- firefox/releases/4.0b9
-- firefox/releases/4.0b10
-- firefox/releases/4.0b11
-- firefox/releases/4.0b12
 - firefox/releases/4.0rc1
 - firefox/releases/4.0rc2
 - firefox/releases/4.0
 - firefox/releases/4.0.1
 - firefox/releases/latest-4.0
-- firefox/releases/5.0b1
-- firefox/releases/5.0b2
-- firefox/releases/5.0b3
-- firefox/releases/5.0b5
-- firefox/releases/5.0b6
-- firefox/releases/5.0b7
-- firefox/releases/6.0b1
-- firefox/releases/6.0b2
-- firefox/releases/6.0b3
+- firefox/releases/*.*b*
 - mobile/releases/wince
 - mobile/releases/winmo
 - mobile/releases/1.0rc1
@@ -270,17 +230,10 @@
 - seamonkey/releases/2.1a1
 - seamonkey/releases/2.1a2
 - seamonkey/releases/2.1a3
-- seamonkey/releases/2.1b1
-- seamonkey/releases/2.1b2
-- seamonkey/releases/2.1b3
-- seamonkey/releases/2.1b3-bad
-- seamonkey/releases/2.1b3-real
 - seamonkey/releases/2.1rc1
 - seamonkey/releases/2.1rc2
 - seamonkey/releases/2.1
-- seamonkey/releases/2.2b1
-- seamonkey/releases/2.2b2
-- seamonkey/releases/2.2b3
+- seamonkey/releases/*.*b*
 - thunderbird/releases/0.1
 - thunderbird/releases/0.2
 - thunderbird/releases/0.3
@@ -303,9 +256,6 @@
 - thunderbird/releases/1.0rc
 - thunderbird/releases/1.1a1
 - thunderbird/releases/1.1a2
-- thunderbird/releases/1.5b1
-- thunderbird/releases/1.5b2
-- thunderbird/releases/1.5b1
 - thunderbird/releases/1.5rc1
 - thunderbird/releases/1.5rc2
 - thunderbird/releases/1.5
@@ -320,8 +270,6 @@
 - thunderbird/releases/1.5.0.13
 - thunderbird/releases/1.5.0.14
 - thunderbird/releases/2.0a1
-- thunderbird/releases/2.0b1
-- thunderbird/releases/2.0b2
 - thunderbird/releases/2.0.0.0rc1
 - thunderbird/releases/2.0.0.0
 - thunderbird/releases/2.0.0.4
@@ -349,10 +297,6 @@
 - thunderbird/releases/3.0a1
 - thunderbird/releases/3.0a2
 - thunderbird/releases/3.0a3
-- thunderbird/releases/3.0b1
-- thunderbird/releases/3.0b2
-- thunderbird/releases/3.0b3
-- thunderbird/releases/3.0b4
 - thunderbird/releases/3.0rc1
 - thunderbird/releases/3.0rc2
 - thunderbird/releases/3.0
@@ -376,8 +320,6 @@
 - thunderbird/releases/3.0.11/win32
 - thunderbird/releases/latest-3.0
 - thunderbird/releases/3.1a1
-- thunderbird/releases/3.1b1
-- thunderbird/releases/3.1b2
 - thunderbird/releases/3.1rc1
 - thunderbird/releases/3.1rc2
 - thunderbird/releases/3.1
@@ -394,7 +336,7 @@
 - thunderbird/releases/3.3a1
 - thunderbird/releases/3.3a2
 - thunderbird/releases/3.3a3
-- thunderbird/releases/5.0b1
+- thunderbird/releases/*.*b*
 - calendar/sunbird/releases/0.2
 - calendar/sunbird/releases/0.3
 - calendar/sunbird/releases/0.3.1

Comment 3

[Dave Miller [:justdave]]

I notice there's alphas in there, too, should those be done the same way?

Comment 4

[Justin Wood (:Callek)]

(In reply to comment #3)
> I notice there's alphas in there, too, should those be done the same way?

Doesn't hurt, but we are no longer producing new alpha's and across all products we already have every alpha on the exclude list afaict.

Of note while I was verifying the above:

http://releases.mozilla.org/pub/mozilla.org/camino/releases/ has many old files (dating back to 2006 it seems)

http://releases.mozilla.org/pub/mozilla.org/mobile/releases/ has latest-alpha and latest-beta dirs that are non-empty.

http://releases.mozilla.org/pub/mozilla.org/xulrunner/releases/ has 1.9.0 and 1.9.1 builds still available, given their Geckos are obsolete should we relegate those to ftp/internal mirrors?

Comment 5

[[e-mail] (:bhearsum)]

As Callek says, we don't ship alphas anymore, so there's not too much benefit to adding a similar thing fon them. Nick mentioned that there's probably a lot of shrinking on older releases we could be doing though, so we could do the alphas plus a bunch of other stuff in a followup. Filed bug 675116 on this.

Comment 6

[[e-mail] (:bhearsum)]

Aki, can we do this for mobile/, too?

Comment 7

[Aki Sasaki (not active)]

Sure.

Comment 8

[[e-mail] (:bhearsum)]

(In reply to comment #7)
> Sure.

Added, and I removed the other references to Betas for mobile/