638787 - Language Detection in Chrome

Status: VERIFIED FIXED

(Websites :: demos.mozilla.org, defect, P2)

Description

[Laura Mesa [:lmesa] [:lvmesa]]

When I go to <http://webofwonder.org/> in Debian's Chromium 9, I get the
> Spanish  version of the page instead of the English version.  My
> Accept-Language  header is:
> 
>    en-US,en;q=0.8,es;q=0.6,fr-CA;q=0.4,fr;q=0.2
> 
> Therefore, I should probably  get the English version unless there is a
> reason that the Spanish version is  significantly better
> content-negotiation-wise.

Comment 1

[Austin King [:ozten]]

This is reproducable on http://addons.mozilla.org also and other Django apps that use our common middleware.

Middleware should lowercase accept language values.

CCing fwenzel, stas, and gandalf for code review...
https://github.com/mozilla/webowonder/commit/194d5178c85dbbee917b10f240a197093fb41bca

Comment 2

[Fred Wenzel [:wenzel]]

r+.

This is a small bug that is certainly present in a number of our projects, includign AMO, SUMO, Input and others.

I looked up what the standard (RFC 1766) says:
"All tags are to be treated as case insensitive; there exist conventions for capitalization of some of them, but these should not be taken to carry meaning."

In other words, Austin's solution is right.

Comment 3

[Zibi Braniecki [:zbraniecki][:gandalf]]

sounds like a good use of .lower() to me. :)

Comment 4

[James Socol [:jsocol, :james]]

I also noticed an issue while fixing this: in the current algorithm, we check for absolute matches before partial matches, meaning that this header

Accept-Language: en-gb, en, fr-fr;q=0.8, es;q=0.2

will match 'es' if you have an 'en-US' locale and an 'es' locale. It's pretty clear in this case that we should pick 'en-US'.

See bug 639818 and https://github.com/jsocol/kitsune/commit/52d827a9ce6 for a potential fix.

Comment 5

[Jeff Balogh (:jbalogh)]

Since you guys steal all of our code I am looking forward to patches that fix this in zamboni. :)

Comment 6

[Fred Wenzel [:wenzel]]

(In reply to comment #4)
> will match 'es' if you have an 'en-US' locale and an 'es' locale. It's pretty
> clear in this case that we should pick 'en-US'.

Yup, we noticed this bug on Input before as well. Good job fixing this.

Another interesting question is this:
en-gb, es;q=0.2

... I seem to remember the standard says in this case prefix-matching is still more important than exact matching with a lower quality factor (i.e., in this case we should still serve en-US over es), but I am not sure.

Comment 7

[James Socol [:jsocol, :james]]

(In reply to comment #6)
> Another interesting question is this:
> en-gb, es;q=0.2
> 
> ... I seem to remember the standard says in this case prefix-matching is still
> more important than exact matching with a lower quality factor (i.e., in this
> case we should still serve en-US over es), but I am not sure.

The patch I linked in comment 4 should do exactly that and return en-US.

Comment 8

[James Socol [:jsocol, :james]]

Fixed in SUMO here: https://github.com/jsocol/kitsune/commit/b0cb526cc9

Comment 9

[Austin King [:ozten]]

Please re-verify this bug.

(In reply to comment #8)
Very helpful, I've integrated your get_best_language code. This works much better than just lower, since it will redirect ZH to zh-TW, etc.


Fixed in
https://github.com/mozilla/webowonder/commit/2e69e51e28d2601c5a27ef929bcb73bc94e22a77

Comment 10

[raymond [:retornam] (inactive)]

verified fixed https://demos.mozilla.org/es/