538429 - Support version-less and platform-less download-eu.m.o links

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Alex Buchanan [:abuchanan]]

For the Browser Choice ballot, we want to give links to Microsoft that are a simple as possible.  This means removing the version number and platform.  

Example:
current = download-eu.m.o/?product=firefox-3.5.7&os=win&lang=af
target = download-eu.m.o/?lang=af

(There could be a way to abstract the language code too, if we feel that's necessary)

That way, we avoid changing the download URL being served by Microsoft, and we have easy control over which version gets served, etc.

I was thinking this could be done with some Apache rewrite rules, or a simple PHP redirect script, or maybe even some Bouncer magic.

Thoughts on the best approach?  Should we go as far as to detect language on our side, making the download link:  download-eu.mozilla.org  ?

Comment 1

[Alex Buchanan [:abuchanan]]

I found a related bug 398366, for implementing product=firefox-latest in Bouncer

Comment 2

[Alex Buchanan [:abuchanan]]

Dave, how do you want to implement this?  A simple PHP script could pull the most recent version from our Firefox product details class and issue a redirect.

As an Apache redirect, it would need to be updated manually with each Firefox release, which is not ideal.

I'm not sure what it would take to patch Bouncer to allow ?product=firefox-latest, but I can look into it if you want to go the route.

Comment 3

[Dave Miller [:justdave]]

Having it somewhere that build can update as part of their release process would be ideal.  Screwing with the apache config every time we have a release wouldn't be fun.  Looking something up on an external site over http wouldn't be good either, that site gets too much traffic to get away with that.  It would need to look it up either from the database or the local filesystem.

Comment 4

[Alex Buchanan [:abuchanan]]

(In reply to comment #3)
> Having it somewhere that build can update as part of their release process
> would be ideal.  Screwing with the apache config every time we have a release
> wouldn't be fun.  Looking something up on an external site over http wouldn't
> be good either, that site gets too much traffic to get away with that.  It
> would need to look it up either from the database or the local filesystem.

agreed.  

Pulling from product-details == local file system, by means of SVN externals.
And, if SVN auto updates, we won't have to touch it at all, since product-details gets updated for mozilla.com with each release.

The only caveat is we issue yet another redirect.  That seems reasonable to me (because we use redirects so often elsewhere), but I'm curious if you feel there will be performance, scale, or response issues with that extra redirect (on top of the redirects we already have for l10n, etc)


I still think patching Bouncer would be a great long-term fix.  I'm looking into bug 398366 to see if I could patch that soon.  But, I fear it is outside of the scope of this bug / project.

Comment 5

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Needed for the EU Ballot release, so adding blocking bug.

Comment 6

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

justdave: I just talked with morgamic, coop about euballot downloads. How easy it would be for you to setup a redirect from: 

download-eu.m.o/?lang=af 

...to:
download.m.o/?product=firefox-latest-euballot&os=win&lang=af

(this would allow us to easily handle the euballot builds in bouncer, like our other builds going through bouncer)

Comment 7

[Alex Buchanan [:abuchanan]]

(In reply to comment #6)
> download.m.o/?product=firefox-latest-euballot&os=win&lang=af

Is there some Bouncer change / config planned to make product=firefox-latest-euballot work?  

Also, mrz specifically asked that download-eu.m.o be separate from download.m.o.  Is that change OK with him?

Thanks.

Comment 8

[Patrick Finch]

(In reply to comment #6)
> justdave: I just talked with morgamic, coop about euballot downloads. How easy
> it would be for you to setup a redirect from: 
> 
> download-eu.m.o/?lang=af 
> 
> ...to:
> download.m.o/?product=firefox-latest-euballot&os=win&lang=af
> 
> (this would allow us to easily handle the euballot builds in bouncer, like our
> other builds going through bouncer)

(In reply to comment #6)
> justdave: I just talked with morgamic, coop about euballot downloads. How easy
> it would be for you to setup a redirect from: 
> 
> download-eu.m.o/?lang=af 
> 
> ...to:
> download.m.o/?product=firefox-latest-euballot&os=win&lang=af
> 
> (this would allow us to easily handle the euballot builds in bouncer, like our
> other builds going through bouncer)

Confirmed that MSFT can accept these changes per John's request.  Asked for updates to Learn More in bug 546754.

Comment 9

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #7)
> (In reply to comment #6)
> > download.m.o/?product=firefox-latest-euballot&os=win&lang=af
> 
> Is there some Bouncer change / config planned to make
> product=firefox-latest-euballot work?  

Entry "firefox-latest-euballot" added to bouncer, which will point to symlink on ftp.m.o which points to builds. This means we can advance easily with every dot release.

Comment 10

[Dave Miller [:justdave]]

(In reply to comment #9)
> Entry "firefox-latest-euballot" added to bouncer, which will point to symlink
> on ftp.m.o which points to builds. This means we can advance easily with every
> dot release.

You mean releases.m.o, right?  I wouldn't expect ftp to handle that kind of load.

Comment 11

[Dave Miller [:justdave]]

that would also mean you wouldn't get per-version download stats out of the bouncer logs.

Comment 12

[Dave Miller [:justdave]]

per discussion on IRC we need to do this as a redirect in Apache, the symlink trick in comment 9 is going to break sentry and cause users to get 404s.

Comment 13

[Dave Miller [:justdave]]

Long term plan: get a bug filed as a feature request for Tuxedo to implement this redirect application-side and allow the redirect destination to be edited in the admin UI along with the product defintions.

Short term plan: we set up the redirect in apache from product=firefox-latest-euballot to product=<current-eu-version> and we'll need to update that redirect every time there's a release.

Comment 14

[Nick Thomas [:nthomas] (UTC+12)]

Please also trick sentry into checking for sv-SE instead of zh-TW for file locations matching /euballot/i.

Comment 15

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Per irc, we've dropped the symlink idea. Instead the plan is to:

1) delete "latest-euballot" symlink on stage (done).
2) put "firefox-3.6-euballot" in bouncer. (done). RelEng keep adding new euballot entries to bouncer for each new release.
3) change bouncer / apache to take "firefox-latest-euballot" and redirect to the version-specific link in bouncer. For example, for today "firefox-latest-euballot" should redirect to "firefox-3.6-euballot". RelEng will file IT bug to update this redirect for every new release.
4) sentry to be updated to check for sv-SE, not tr, as we believe this is the last "real" repack of the set.

This bug to track that work, reopening and reassigning back to justdave.

Comment 16

[Dave Miller [:justdave]]

(In reply to comment #15)
> 4) sentry to be updated to check for sv-SE, not tr, as we believe this is the
> last "real" repack of the set.

Done.

+        if ($filepath =~ m!/euballot/!i) {
+            $filepath =~ s@/en-US/@/sv-SE/@;
+        }

Sending        perl/sentry.pl
Transmitting file data .
Committed revision 62604.


> This bug to track that work, reopening and reassigning back to justdave.

Now that the above is done, justdave is now on vacation until March 1st (possibly later, since I'm 3 days late starting my vacation already).

Comment 17

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #16)
> (In reply to comment #15)
> > 4) sentry to be updated to check for sv-SE, not tr, as we believe this is the
> > last "real" repack of the set.
> 
> Done.
> 
> +        if ($filepath =~ m!/euballot/!i) {
> +            $filepath =~ s@/en-US/@/sv-SE/@;
> +        }
> 
> Sending        perl/sentry.pl
> Transmitting file data .
> Committed revision 62604.

Sorry I meant /euballot/i in the perl regexp sense, rather than directories. The files are going to live in firefox/releases/3.6/win32-EUballot/, so the above won't match.

Comment 18

[Dave Miller [:justdave]]

Here's our Apache rewrite:

> # Bug 538429: redirect firefox-latest-euballot to current version
> RewriteCond %{QUERY_STRING} ^(.*)product=firefox-latest-euballot(.*)$
> RewriteRule /(.*)$ http://download-eu.mozilla.org/$1?%1product=firefox-3.6-euballot%2 [R=302,L]

took pm-app-dist06 out of the pool and tested it there first, seems to work, pushing it out to the cluster now.

Comment 19

[Dave Miller [:justdave]]

-        if ($filepath =~ m!/euballot/!i) {
+        if ($filepath =~ m!-euballot/!i) {

Sending        perl/sentry.pl
Transmitting file data .
Committed revision 62605.


rewrites are now live on the entire cluster, you're now clear to let QA have at it.

Comment 20

[Stephen Donner [:stephend] Not actively reading bugmail]

Reopening:

http://download-eu.mozilla.org/?os=win&lang=en-GB is a 404
http://download-eu.mozilla.org/?lang=de is a 404

Comment 21

[Nick Thomas [:nthomas] (UTC+12)]

Bug 546754 says the final form for the urls is 
 http://download-eu.mozilla.org/?product=firefox-latest-euballot&os=win&lang=de

Comment 22

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to comment #21)
> Bug 546754 says the final form for the urls is 
>  http://download-eu.mozilla.org/?product=firefox-latest-euballot&os=win&lang=de

That bug and this are convoluted, but basically, iirc:

* work was done in this bug to support version-less and platform-less download links from download-eu.m.o, but was supplanted by bug 546754?  Messy and confusing, but OK.

Comment 23

[Dave Miller [:justdave]]

(In reply to comment #20)
> Reopening:
> 
> http://download-eu.mozilla.org/?os=win&lang=en-GB is a 404
> http://download-eu.mozilla.org/?lang=de is a 404

http://download.mozilla.org/?os=win&lang=en-GB is also 404 (and always has been).

The instructions were for platformless and versionless downloads, not productless.  Productless has never worked, and never will (there's not much point to it).

Comment 24

[Dave Miller [:justdave]]

FYI I'm on vacation this week, if you need immediate attention you should reassign to server-ops when you reopen.

And bug 546754 appears completely unrelated to this.  I think that one was for the links to appear on the browserchoice website, which was based on the links we made possible on this bug by fixing bouncer.