439568 - Locale selection by Accept-Language header misbehaving with short locale identifiers

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Public Pages, defect, P4)

Description

[shawn]

User-Agent:       Opera/9.50 (Macintosh; Intel Mac OS X; U; en)
Build Identifier: 

Browsing with Opera, which (for my install, at least) defaults to sending:

en,ja;q=0.9,fr;q=0.8,de;q=0.7,es;q=0.6,it;q=0.5,nl;q=0.4,sv;q=0.3,nb;q=0.2

This redirects me to https://addons.mozilla.org/ja/firefox/ instead of https://addons.mozilla.org/en-US/firefox/

In the Steps to Reproduce, I've included how to reproduce in Firefox.

Reproducible: Always

Steps to Reproduce:
1. In the browser preferences, set the preferred languages to [en], [ja], [en-us], in that order.
2. Go to http://addons.mozilla.org/
Actual Results:  
http://addons.mozilla.org/ redirects me to https://addons.mozilla.org/ja/firefox/

Expected Results:  
http://addons.mozilla.org/ should redirect me to https://addons.mozilla.org/en-US/firefox/

Comment 1

[Fred Wenzel [:wenzel]]

Yes it's true: en does not currently expand to en-US. Since the next best match is ja, it forwards you there.

Comment 5

[Matthew Somerville]

Sorry for the dupe - I did search first. Just to paraphrase what I said there: the HTTP specification states: "A language-range [in an Accept-Language
request header] matches a language-tag [in a variant] if it exactly
equals the tag, or if it exactly equals a prefix of the tag such that
the first tag character following the prefix is "-"." - ie. the request
can be a prefix of the document language, and so en first in my Accept-Language header should match a server document variant in en-us and be returned that. I'd be happy to take a look at the bit of the code, if someone could point out where it is.

Comment 6

[Fred Wenzel [:wenzel]]

No problem, it's easy to miss a bug when searching. The code is here: function detectCurrentLanguage() in http://viewvc.svn.mozilla.org/vc/addons/trunk/site/app/config/language.php?view=markup .

When fixing this, we'll also need to add another test here: http://viewvc.svn.mozilla.org/vc/addons/trunk/site/app/tests/languageConfig.test.php?view=markup . There are some in there (de-de -> de, for example), but this bug is not covered.

Comment 7

[Matthew Somerville]

Attachment: Patch to (hopefully) fix bug plus additional tests.

Comment 8

[Matthew Somerville]

I've attached a patch that I hope fixes this bug. language.php had two different ways of parsing the HTTP_ACCEPT_LANGUAGE header, both of which had bugs (e.g. one ignored quality values, the other didn't work with more than two languages provided in the header). Calls to detectCurrentLanguage would call getFallbackLanguage (and so parse HTTP_ACCEPT_LANGUAGE) no matter whether the argument to the function was true or false - so I assume that parameter was actually unneeded, removed the further calls to it in bootstrap.php, and consolidated the HTTP_ACCEPT_LANGUAGE parsing into one function. I've added three additional tests that were broken with the current handling and are hopefully fixed by my reworking, plus changed one test that was actually testing the opposite of what the RFC states. Let me know if anything's unclear or if anyone spots anything wrong - this is my first patch to Bugzilla, so I'm not sure of the next steps, but hopefully someone will. Phew :)

Comment 9

[fcp]

(In reply to comment #8)
> plus changed one test that was actually
> testing the opposite of what the RFC states.

If I understand it correctly, this is not only a change in the test, but it is also a change in the behavior.  Is this correct?

If so, I am a little concerned about this change.  Currently AMO serves Japanese pages as the language ja instead of ja-JP.  IE 7 and IE 8 send out ja-JP to represent Japanese.  If a user configures IE 7/8 to try Japanese and the US English in this order, it results in
  Accept-Language: ja-JP,en-US;q=0.5
If you make the suggested change to AMO, he/she will see the English version instead of the Japanese version, and this is not expected or desirable.

I do not know what the RFC-correct behavior is, but maybe you are right.  In that case, the correct solution might be to serve the Japanese pages in the language ja-JP instead of ja.

Comment 10

[fcp]

(In reply to comment #9)
> IE 7 and IE 8 send out ja-JP to represent Japanese.

This behavior is explained in detail in an article on IEBlog:
http://blogs.msdn.com/ie/archive/2006/10/17/accept-language-header-for-internet-explorer-7.aspx

To make sure, I am not arguing whether Microsoft is correct or not or whether the proposed patch is good or not, but I am just pointing out that applying the proposed patch alone will probably cause a trouble to a certain number of users.

Comment 11

[Matthew Somerville]

I didn't know about IE7/8, that's frankly shocking. As the RFC even clearly states: "As an example, users might assume that on selecting "en-gb", they will be served any kind of English document if British English is not available. A user agent might suggest in such a case to add "en" to get the best matching behavior." :-(

"If I understand it correctly, this is not only a change in the test, but it is
also a change in the behavior.  Is this correct?" - yes, that's correct, as the current behaviour is against the RFC. But as I said, I didn't know IE7/8 had totally broken behaviour.

So armed with that new information, I'm afraid I'm not sure what to suggest. To sum up, AMO current behaviour is to match de-DE in request to de at the server, but not vice-versa; RFC says en in request should match en-US at server and not vice-versa. I guess the patch could be changed so it does *both* - ie. en in request would match en-US at server, and de-DE in request would match de at server, which is non RFC-compliant, but would prevent issues in IE7/8. I'd be happy to make that change if it was decided that was best. We certainly don't want to break it for IE7/8, but a modified patch would still allow "en, fr;q=0.8" to work (which was my Accept-Language when I found this problem) and return English, not French.

Comment 12

[Matthew Somerville]

Attachment: Patch as above, but will allow e.g. de-de in request to match de at server.

Comment 13

[Matthew Somerville]

Second patch still fixes this bug (and related others as above with respect to qvalues and multiple entries in the AL header), but only adds new tests, doesn't alter current behaviour with regards to specific code in the request matching a generic one at the server. As a pragmatist and given the information about IE7/8, I'd say this patch is probably preferable, but I'll leave that decision to someone else :)

Comment 14

[fcp]

Thank you for the reply, Matthew!

(In reply to comment #11)
> I didn't know about IE7/8, that's frankly shocking. As the RFC even clearly
> states: "As an example, users might assume that on selecting "en-gb", they will
> be served any kind of English document if British English is not available. A
> user agent might suggest in such a case to add "en" to get the best matching
> behavior." :-(

IE 7/8 do not do that.  :(  I am not sure whether this means IE 7/8 are broken, though.

I think that there are four options here.  In what follows, a-l stands for accept-language.

(0) Leave the current behavior as it is.  That is, a-l:xx-XX matches the xx documents but a-l:xx does not match the xx-XX documents.
(1) As Matthew proposed in comment #8, make a-l:xx match the xx-?? documents, and stop making a-l:xx-XX match the xx documents (patch in comment #7).
(2) As I wrote in comment #9, follow (1), and change the language of documents from fr to fr-FR, from ja to ja-JP, and so on.
(3) As Matthew proposed in comment #11, make a-l:xx match the xx-?? documents, and a-l:xx-XX continues to match the xx documents (patch in comment #12).

Benefits of each option:
Solve this bug?  That is, serve English pages for a-l:en,fr:q=0.8 as desired?
-> (0) No, (1) Yes, (2) Yes, (3) Yes.
Is the fallback algorithm RFC compliant?
-> (0) No, (1) Yes, (2) Yes, (3) Not strictly.
Serve Japanese pages to IE 7/8 users with a-l:ja-JP,en-US:q=0.5 as desired?
-> (0) Yes, (1) No, (2) Yes, (3) Yes.
Easy to apply the change (probably)?
-> (0) -, (1) Yes, (2) Not sure (URLs will change), (3) Yes.

I would vote for (2) or (3).

Comment 15

[Wil Clouser [:clouserw]]

Locale codes are split up into xx-YY where xx is the language and YY is the country or region.  Languages can vary between country and region, usually with subtle differences (Canada and all their u's ;) but I think there are some cases where it is more drastic.

The original code was written because offing someone xx instead of xx-YY gives them a more generic version of the localization - that is pretty standard across the board.  Giving someone xx-ZZ instead of xx or xx-YY is giving someone a more specific language than what they are asking for (a regional language).  I don't think this is standard behavior anywhere (examples? or RFC?).  All things being equal, if someone asks for xx-YY which we don't have and zz which we do have, it makes more sense to offer them zz.  If the language priority is different, I can see some argument there.

So...

Case 0:  I explained why it is the way it is above.  I'm not saying there isn't room for improvement since being technically correct and being helpful isn't always the same. ;)

Case 1:  I don't like this case.  You're removing the standard behavior (xx-YY -> xx) and introducing new, more complicated behavior (I'll expand on that in case 3).

Case 2:  I don't like this one as it specifies what region the language is for.  The French used on the site isn't specifically for France so we shouldn't advertise it that way.  Same with the other locales - if they are for a specific region they are already named that way.

Case 3:  If we're going to change this, I suspect this is the closest to what we want.  Aside from sending people to a more regional language which they may not speak, one of the main problems with this is which region do you send them do?  For example, we currently support pt-PT and pt-BR on the site.  What happens when someone requests pt-YY?

Comment 16

[Matthew Somerville]

"The original code was written because offing someone xx instead of xx-YY gives
them a more generic version of the localization - that is pretty standard
across the board." - no, it is not. For example, Apache only does it as a last resort - http://httpd.apache.org/docs/2.2/content-negotiation.html states: "For example, if a client requests documents with the language en-GB for British English, the server is not normally allowed by the HTTP/1.1 standard to match that against a document that is marked as simply en. (Note that it is almost surely a configuration error to include en-GB and not en in the Accept-Language header, since it is very unlikely that a reader understands British English, but doesn't understand English in general. Unfortunately, many current clients have default configurations that resemble this.) However, if no other language match is possible and the server is about to return a "No Acceptable Variants" error or fallback to the LanguagePriority, the server will ignore the subset specification and match en-GB  against en documents. Implicitly, Apache will add the parent language to the client's acceptable language list with a very low quality value. But note that if the client requests "en-GB; q=0.9, fr; q=0.8", and the server has documents designated "en" and "fr", then the "fr" document will be returned. This is necessary to maintain compliance with the HTTP/1.1 specification and to work effectively with properly configured clients."

As they say, and I have previously quoted in comments #5 and #11, offering a more generic version is against the RFC (I am not saying that means it shouldn't still do it, though, again, as per my previous comments). User agents *should* always specifiy a generic language as well as a specific (quote from RFC in comment #11), and it turns out IE7/8 does not do that.

"Giving someone xx-ZZ instead of xx or xx-YY is giving someone a more specific language than what they are asking for (a regional language).  I don't think this is standard behavior anywhere (examples? or RFC?)." - RFC 2616 section 14.4 as I quoted above. If a server has an en-US document and the request contains en, that document matches - the request is asking for any en variant, that's what "en" means in an Accept-Language header.

"All things being equal, if someone asks for xx-YY which we don't have
and zz which we do have, it makes more sense to offer them zz." - that is quite correct, but that's not what you're saying just before. The bug with AMO, as explained, is if some asks for en and fr when AMO has en-US and fr, they are getting fr rather than en-US. Which is not only against spec, it is clearly just wrong.

"introducing new, more complicated behavior" - that obeys the HTTP RFC, and makes the site work for anyone who specifies an obvious Accept-Language header. :-)

"Aside from sending people to a more regional language which they may
not speak" - if the client asks for "en", you give them any "en" variant you may have - that is what the RFC states.

"one of the main problems with this is which region do you send them
do?  For example, we currently support pt-PT and pt-BR on the site.  What
happens when someone requests pt-YY?" Well, let's do it the RFC way, the current AMO way, and the "hybrid way":

RFC way - if they solely specify pt-YY, you should *not* return either pt-PT or pt-BR, as they have said they don't want either. You should return some other language they say they do want, or return "No match" or fallback as a last resort. If they had specified "pt", you would return whichever of pt-PT and pt-BR had a higher quality value, or failing that you could return 300 Multiple Choices or just pick one.

Current AMO way - if they solely specify pt-YY, they get en-US as fallback. If they specify solely 'pt', they also get en-US, which is clearly wrong, as they're asking for any variant of 'pt'.

Hybrid way (case 3) - Someone who just specifies pt-YY will get en-US (so as current case, as nothing matches). Someone who just specifies pt will get one of the two pt variants, as their Accept-Language header has asked for.

ie. in the case you ask about, the hybrid way matches the RFC way (as it should), and is better than the current AMO way. The time when the hybrid way doesn't match the RFC is if the request header is more specific than the server document (e.g. request of "de-DE" matching a server document of "de"), and I've said I think that should be continued due to IE7/8's behaviour.

Comment 17

[Wil Clouser [:clouserw]]

I'll follow up in a bit, but want to schedule this for 5.0.8.

Comment 18

[Wil Clouser [:clouserw]]

(In reply to comment #16)
> "The original code was written because offing someone xx instead of xx-YY gives
> them a more generic version of the localization - that is pretty standard
> across the board." - no, it is not. For example, Apache only does it as a last
> resort - http://httpd.apache.org/docs/2.2/content-negotiation.html states: "For
> example, if a client requests documents with the language en-GB for British
> English, the server is not normally allowed by the HTTP/1.1 standard to match
> that against a document that is marked as simply en. (Note that it is almost
> surely a configuration error to include en-GB and not en in the Accept-Language
> header, since it is very unlikely that a reader understands British English,
> but doesn't understand English in general. Unfortunately, many current clients
> have default configurations that resemble this.) However, if no other language
> match is possible and the server is about to return a "No Acceptable Variants"
> error or fallback to the LanguagePriority, the server will ignore the subset
> specification and match en-GB  against en documents. Implicitly, Apache will
> add the parent language to the client's acceptable language list with a very
> low quality value. But note that if the client requests "en-GB; q=0.9, fr;
> q=0.8", and the server has documents designated "en" and "fr", then the "fr"
> document will be returned. This is necessary to maintain compliance with the
> HTTP/1.1 specification and to work effectively with properly configured
> clients."

Right.  I meant it is standard across the board as a last resort fallback

> "Giving someone xx-ZZ instead of xx or xx-YY is giving someone a more specific
> language than what they are asking for (a regional language).  I don't think
> this is standard behavior anywhere (examples? or RFC?)." - RFC 2616 section
> 14.4 as I quoted above. If a server has an en-US document and the request
> contains en, that document matches - the request is asking for any en variant,
> that's what "en" means in an Accept-Language header.

Alright, section 14.4 contains:
   A language-range matches a language-tag if
   it exactly equals the tag, or if it exactly equals a prefix of the
   tag such that the first tag character following the prefix is "-".

Which sounds like the behavior you are describing is correct. I think the current behavior was implemented because of the notes after the explanation: 
      Note: This use of a prefix matching rule does not imply that
      language tags are assigned to languages in such a way that it is
      always true that if a user understands a language with a certain
      tag, then this user will also understand all languages with tags
      for which this tag is a prefix. The prefix rule simply allows the
      use of prefix tags if this is the case.

      Note: When making the choice of linguistic preference available to
      the user, we remind implementors of  the fact that users are not
      familiar with the details of language matching as described above,
      and should provide appropriate guidance. As an example, users
      might assume that on selecting "en-gb", they will be served any
      kind of English document if British English is not available. A
      user agent might suggest in such a case to add "en" to get the
      best matching behavior.

> Hybrid way (case 3) - Someone who just specifies pt-YY will get en-US (so as
> current case, as nothing matches). Someone who just specifies pt will get one
> of the two pt variants, as their Accept-Language header has asked for.
> 
> ie. in the case you ask about, the hybrid way matches the RFC way (as it
> should), and is better than the current AMO way. The time when the hybrid way
> doesn't match the RFC is if the request header is more specific than the server
> document (e.g. request of "de-DE" matching a server document of "de"), and I've
> said I think that should be continued due to IE7/8's behaviour.

I agree the above hybrid way seems most reasonable.  That means (double check this please):
en, fr    -> en-US
fr, en    -> fr
pt-YY     -> pt (against the RFC but it's current behavior and desired)
pt        -> ??? (one of our pt variants - which?)
pt, de    -> de
en-XX, fr -> fr


I'm CCing other people who may have comments.

Comment 19

[Axel Hecht [:Pike]]

As Pascal, I'll really read this tomorrow.

Just to add to it, locale codes are not "language(-REGION)?", but there are more subtags like script. We'll hit Script like Latn, Arab, Cyrl rather soon.

Comment 20

[Axel Hecht [:Pike]]

More comments:

RFC 2616 (HTTP 1.1) references RFC 1766 for language tags, which is obsoleted by RFC 4646 and RFC 4647, http://tools.ietf.org/html/rfc4646 and http://tools.ietf.org/html/rfc4647, the latter being about matching of language tags.

Not that RFC 4647 is an easy read.

Comment 21

[Matthew Somerville]

Well, I've just read it :-) Lookup (section 3.4) is what applies in this situation, I believe, and it does indeed state that the negotiation should fall back to e.g. "zh" after a "zh-Hant" in the request header if no zh-Hant document is available. Section 4.1 is also of interest. This matches (nearly) the way discussed above called "hybrid", which is the second patch I supplied.

Wil Clouser: I would have disagreed with the "pt, de" line before reading RFC4647 and it seems to say the same thing in this regard. If the request contains "pt" at the highest priority, then you should respond with a pt variant if you have one - and AMO has two. Which you respond with (as with the request of just "pt"), is up to the server; e.g. if one has more translated, you might want to do that, otherwise it really doesn't appear to matter - section 4.1 of RFC4647 says (about the same situation with Chinese) "one of them (the one considered 'default') also ought to be made available with the simple "zh"."

Other changes:
* pt-YY -> pt : This appears to be accepted by RFC4647.
* en-XX, fr -> fr : Assuming en-XX has a higher q-value, then RFC4647 appears to say that you should return the en-US by preference to this request (the example in section 3.4.1 has "fr-FR, zh-Hant" -> "fr-FR, fr, zh-Hant, zh, ...", and the section 4.1 quoted above). I'm not sure I've read that bit correctly, though, but if so, the patch needs altering for this one case, as it won't do this. It does seem quite an edge case compared to the much more important "en, fr" not working, though, which the hybrid patch definitely does fix and could always be revisited in future :)

Comment 22

[Wil Clouser [:clouserw]]

My examples left out different q values in the interest of simplicity.  The RFC states that q values default to 1.0 if unspecified.  If other values are specified they should, of course, be taken into consideration.

Comment 23

[Pascal Chevrel:pascalc]

> I agree the above hybrid way seems most reasonable.  That means (double check this please):
> en, fr    -> en-US
> fr, en    -> fr
> pt-YY     -> pt (against the RFC but it's current behavior and desired)
> pt        -> ??? (one of our pt variants - which?)
> pt, de    -> de
> en-XX, fr -> fr

So, the real question is to know what is the best user experience we want to give to our users given that theory (RFCs) does not necessarily meets practice (hundreds of millions of people using browsers that do not strictly follow RFCs) and that furthermore theory is not necessarily perfect either or fitting our own specific needs.

In the above proposal, we would send German pages to Portuguese speakers and French pages to English speakers.

A user having 'pt, de' is typically an IE6 portuguese speaker that manually added in his preferences that he ALSO speaks German. IE6 sends two-letter locale codes only.

A user having 'en-XX, fr' is a native English speaker that indicated in his preferences that he also understand French as a second choice. A typical case would be an English-Canadian speaker that added to his browser that he also understands French. This typical case is likely to be an IE7 or IE8 user since IE uses Windows 4-letter locale code from version 7 onward.

Does it make sense in terms of user experience to send French pages to an en-CA visitor while we have en-US pages just to follow RFCs?

That said, the most common non-Firefox user case we will meet are people having just one 4-letter locale code sent in accept-lang headers, like en-AU, fr-CA or ja-JP, that is IE7+ users. For those people we should just use the 2 letter locale code (en, fr, ja) and send them the pages we have for that. We can add a few checks to decide on pt-PT vs pt-BR for example (btw don't discard the possibility that one day the site will be in en-GB as well) and another solution is to inform the user that several variants of his language are supported on the site and that since we are unsure about what he wants, he can choose it manually.

Comment 24

[Matthew Somerville]

"In the above proposal, we would send German pages to Portuguese speakers and
French pages to English speakers." <-- the above lookup chart does not match what my 2nd patch does, nor what anyone wants. Read my more recent reply to that post, please :)

"A user having 'pt, de' is typically an IE6 portuguese speaker that manually
added in his preferences that he ALSO speaks German. IE6 sends two-letter
locale codes only." <-- Yes, and that currently fails in AMO and would be fixed by my patch. Try it - you get German currently, when you should get a Portuguese variant.

"A user having 'en-XX, fr' is a native English speaker that indicated in his
preferences that he also understand French as a second choice. A typical case
would be an English-Canadian speaker that added to his browser that he also
understands French. This typical case is likely to be an IE7 or IE8 user since
IE uses Windows 4-letter locale code from version 7 onward." - an IE7 user would have "en-CA, fr-FR" given *all* locales are 4-letter. This currently returns French in AMO, and would continue to return French in my patch.

"Does it make sense in terms of user experience to send French pages to an en-CA
visitor while we have en-US pages just to follow RFCs?" <-- I would say that it's what currently happens, and is not really relevant to fixing this bug, or this patch. I would just like this bug fixed.

"That said, the most common non-Firefox user case we will meet are people having
just one 4-letter locale code sent in accept-lang headers, like en-AU, fr-CA or
ja-JP, that is IE7+ users. For those people we should just use the 2 letter
locale code (en, fr, ja) and send them the pages we have for that." <-- yes, my 2nd patch does that, whilst fixing your first condition above (and the one the bug was about, and the one that made me submit the patch in the first place).

"We can add a few checks to decide on pt-PT vs pt-BR for example (btw don't discard the possibility that one day the site will be in en-GB as well) and another solution is to inform the user that several variants of his language are
supported on the site and that since we are unsure about what he wants, he can
choose it manually." <-- yes, this is always an option and up to whoever is responsible, but again, I'd say beyond the remit of fixing this bug.

Comment 25

[Pascal Chevrel:pascalc]

> Read my more recent reply to that post, please :)

You actually posted your reply while I was writing my own answer :)

>an IE7 user would have "en-CA, fr-FR" given *all* locales are 4-letter.

That's incorrect, people that set their preferences in IE6 with a 2-letter code had these preferences migrated in IE7, furthermore IE allows you to set the locale code you want in the custom field dialog.

Comment 26

[Matthew Somerville]

Okay (that wasn't clear from the MSDN link given previously); doesn't change my answer, thankfully :)

Comment 27

[Wil Clouser [:clouserw]]

(In reply to comment #23)
> > I agree the above hybrid way seems most reasonable.  That means (double check this please):
> > en, fr    -> en-US
> > fr, en    -> fr
> > pt-YY     -> pt (against the RFC but it's current behavior and desired)
> > pt        -> ??? (one of our pt variants - which?)
> > pt, de    -> de
> > en-XX, fr -> fr

If I read the above comments correctly, the above is fine (assuming no q values for now) except we'd like to have two changes:

> > pt, de    -> de
Move in the order given in the header and attempt to match a 2 letter locale with any other 4 letter locale with a matching prefix.  Unless specifically overridden, it'll just match in alphabetical order.  This would match the above to pt-BR.

> > en-XX, fr -> fr
Move in the order given in the header and attempt to match a 4 letter locale to it's 2 letter prefix.  If that fails attempt to match the 2 letter locale to any other 4 letter locale with a matching prefix.  Same rule regarding alphabetical order as above.  In this case this would match en-US.

Does that correctly summarize where we are at in this bug?

Comment 28

[Axel Hecht [:Pike]]

Please support Script tags property, sr-Latn vs sr-Cyrl, pa-Arab etc. We will add them to our Firefox universe in the not-so-distant future.

Comment 29

[Wil Clouser [:clouserw]]

(In reply to comment #28)
> Please support Script tags property, sr-Latn vs sr-Cyrl, pa-Arab etc. We will
> add them to our Firefox universe in the not-so-distant future.

Can you adjust my mappings in comment 27 to what you think they should be with that in mind?

Comment 30

[Matthew Somerville]

"Does that correctly summarize where we are at in this bug?" Sounds right to me. As far as I am aware, script tags would be supported properly too, there's no reason why they wouldn't be.

And, far from me wanting to beat a broken drum, it's always possible to revisit the code if it turns out to be a problem in the not-so-distance future, whereas it's currently broken *now* for everyone specifying "en, fr" (or as I have at the moment, "en, cy, fr, nb").

Comment 31

[Wil Clouser [:clouserw]]

If you can make a new patch from your previous one and add tests for the redirections in comment 27 I'll land it.  It should be possible to copy/modify existing tests: http://viewvc.svn.mozilla.org/vc/addons/trunk/site/app/tests/languageConfig.test.php?view=markup

Comment 32

[Wil Clouser [:clouserw]]

Alright, I'm calling this bug fixed in Zamboni (the new version of AMO).  I followed most of what we decided above, but I'm ignoring the IE6 redirects (en, fr -> en-US) because IE6 traffic isn't big enough to override someone specifically requesting fr.  I'll leave the bug open for now requesting feedback.  You can see my commented unit tests here:  http://github.com/clouserw/zamboni/blob/787df676ec7c65ac3712c382ff0763a182ee7be9/apps/amo/tests/test_redirects.py#L25

If I don't hear feedback I'll land this and close the bug.  I expect this to be live on AMO in Q2.

Comment 33

[Wil Clouser [:clouserw]]

Alright, I checked this in.  You can see the rules in the unit tests at http://github.com/jbalogh/zamboni/blob/master/apps/amo/tests/test_redirects.py#L19

Calling this fixed, but again, it won't be on the live site for a while.

Comment 34

[a_geek]

I ask that this bug be re-opened. With FF 9.0.1, I have this locale settings:
Accept-Language: en-us,en;q=0.9,de;q=0.8,zh;q=0.7,fr;q=0.6,ru;q=0.5,es;q=0.4,sv;q=0.3,ar;q=0.2

When I access addons.mozilla.org, I get the German variant of the site. Unfortunately, reproducibility is poor, and my guess is that there might be some server side caching getting in the way.