Closed Bug 536808 Opened 11 years ago Closed 7 years ago

preferred links to old releases are dead (archive.mozilla.org only shows current/latest)

Categories

(Infrastructure & Operations Graveyard :: WebOps: Product Delivery, task)

task
Not set
trivial

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: mrmazda, Unassigned)

Details

(Whiteboard: [need apache rewrite map])

e.g. : http://releases.mozilla.org/pub/mozilla.org/firefox/releases/ only shows current/latest version dirs which is probably a newly established behavior. However, http://archive.mozilla.org/pub/mozilla.org/firefox/releases/ shows exactly the same thing, while http://stage.mozilla.org/pub/mozilla.org/firefox/releases/ apparently shows everything http://releases.mozilla.org/pub/mozilla.org/firefox/releases/ used to show. The current behavior results in a massive number of dead links on http://www.mozilla.org/ports/os2/ and probably a lot of other places. It also means the only way to offer access to old builds seems to be via dissemination of the existence of http://stage.mozilla.org/ to people that maybe shouldn't know about it.
It's been that way for a few years actually.  The only possible change might be to the list of stuff being excluded from getting sent to the releases pool.

Changing the protocol from http to ftp should still get around the redirects.

It'd probably be possible to set up a RedirectMap script which would read the excludes file from the rsync module and not redirect anything that's on the exclude list.  That'd be more of a project to pull of, but probably the correct long-term solution.
Assignee: server-ops → justdave
Component: Server Operations → Server Operations: Projects
(In reply to comment #1)
> It'd probably be possible to set up a RedirectMap script which would read the
> excludes file from the rsync module and not redirect anything that's on the
> exclude list.  That'd be more of a project to pull of, but probably the correct
> long-term solution.

Nick and I discussed this on IRC the other day and decided we'd like to go ahead and do this.  It'll be better for performance this way than pointing people at another server that's reading the same filesystem over NFS from the FTP server.  In reality it's probably not going to be that hard to pull off, but will probably take a couple hours of walking the tree fixing up the exclude list to be more accurate so we can ditch all the funky bind mounts.

I'll try to tackle this myself after I get a few other things caught up, but Nick's welcome to attempt it himself if he's feeling bored before I get to it. :)
Component: Server Operations: Projects → Server Operations
Stage 1 of this is done.  The bind mounts on surf and dm-download02 are gone, the mozilla-releases module is now using the full archive.m.o docroot with an exclude file.  The exclude file in use is located at zz/rsync-mozilla-releases.exclude within the ftp tree.  Note to anyone that edits this, the specific excludes need to happen before the includes, and the global catch-all exclude needs to be last, after the includes.
Severity: normal → trivial
[need apache rewrite map]
Component: Server Operations → Server Operations: Projects
Whiteboard: [need apache rewrite map]
Assignee: justdave → server-ops-webops
Component: Server Operations: Projects → WebOps: Product Delivery
Product: mozilla.org → Infrastructure & Operations
QA Contact: mzeier → nmaul
just triaging some old bugs and this appears to have been resolved already. each of the links make their way back to the ftp cluster.


$ curl -IL http://releases.mozilla.org/pub/mozilla.org/firefox/releases/
HTTP/1.1 301 Moved Permanently
Server: Apache
X-Backend-Server: pp-web01
Vary: Accept-Encoding
Content-Type: text/html; charset=iso-8859-1
Date: Mon, 26 May 2014 21:33:20 GMT
Location: http://download.cdn.mozilla.net/pub/mozilla.org/firefox/releases/
Transfer-Encoding: chunked
Connection: Keep-Alive
X-Cache-Info: cached

HTTP/1.1 200 OK
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
Cache-Control: max-age=300
Content-Type: text/html;charset=UTF-8
Date: Mon, 26 May 2014 21:33:41 GMT
Expires: Mon, 26 May 2014 21:38:41 GMT
Last-Modified: Mon, 26 May 2014 21:29:54 GMT
Server: ECAcc (sjc/4EC2)
X-Backend-Server: ftp4.dmz.scl3.mozilla.com
X-Cache: HIT
X-Cache-Info: caching
Content-Length: 92503


$ curl -I http://archive.mozilla.org/pub/mozilla.org/firefox/releases/
HTTP/1.1 200 OK
Server: Apache
X-Backend-Server: ftp1.dmz.scl3.mozilla.com
Cache-Control: max-age=300
Content-Type: text/html;charset=UTF-8
Date: Mon, 26 May 2014 21:33:30 GMT
Expires: Mon, 26 May 2014 21:38:30 GMT
Transfer-Encoding: chunked
Access-Control-Allow-Origin: *
Connection: Keep-Alive
X-Cache-Info: caching



$ curl -I http://stage.mozilla.org/pub/mozilla.org/firefox/releases/
HTTP/1.1 200 OK
Server: Apache
X-Backend-Server: ftp2.dmz.scl3.mozilla.com
Cache-Control: max-age=300
Content-Type: text/html;charset=UTF-8
Date: Mon, 26 May 2014 21:33:59 GMT
Expires: Mon, 26 May 2014 21:38:59 GMT
Transfer-Encoding: chunked
Access-Control-Allow-Origin: *
Connection: Keep-Alive
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.