Last Comment Bug 765048 - accept language header with q=0.500
: accept language header with q=0.500
Status: RESOLVED FIXED
: regression
Product: Core
Classification: Components
Component: Networking: HTTP (show other bugs)
: Trunk
: All All
: -- normal with 1 vote (vote)
: mozilla17
Assigned To: Patrick McManus [:mcmanus] PTO until Sep 6
:
Mentors:
http://panopticlick.eff.org/
Depends on:
Blocks: http-fingerprint 672448
  Show dependency treegraph
 
Reported: 2012-06-14 15:27 PDT by karl155
Modified: 2012-08-28 09:58 PDT (History)
8 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---
-
fixed
fixed


Attachments
backout patch 0 (20.68 KB, patch)
2012-07-27 18:39 PDT, Patrick McManus [:mcmanus] PTO until Sep 6
cbiesinger: review+
lukasblakk+bugs: approval‑mozilla‑aurora+
Details | Diff | Splinter Review

Description karl155 2012-06-14 15:27:44 PDT
The Accept-Language header looks a little strange on Nightly.
I did not modify it. After switching to Nightly it automatically changed
from "de-de,de;q=0.8,en-us;q=0.5,en;q=0.3"
to "en-us,en;q=0.500".
Shouldn't it be "en-us,en;q=0.5" or so?
(I don't know the correct one for en-US. But according to Panopticlick this is not the common one.)


GET / HTTP/1.1
Host: www.example.com
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:16.0) Gecko/16.0 Firefox/16.0a1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.500
Accept-Encoding: gzip, deflate
Connection: keep-alive
Comment 1 karl155 2012-07-20 07:31:08 PDT
Now Aurora (16.0a2 2012-07-19) shows this behavior too:

GET / HTTP/1.1
Host: www.example.com
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:16.0) Gecko/16.0 Firefox/16.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de-de,de;q=0.750,en-us;q=0.500,en;q=0.250
Accept-Encoding: gzip, deflate
Connection: keep-alive
Comment 2 Gordon P. Hemsley [:GPHemsley] 2012-07-20 07:57:51 PDT
This behavior was intentionally changed in bug 672448 to reduce ambiguity in quality values for those who have more than a handful of language preferences.

As of Gecko 16, the Accept-Language header now lists quality values to 3 decimal places. However, this continues to be generated automatically (the same way the 1-decimal place version was), so this will not increase fingerprintability any more than the switch from Aurora 15 to Aurora 16 would (by way of the User-Agent string). Everyone who maintains the default values will still have the same Accept-Language header. It will just differ from the old value.

As this is intended behavior (at least insofar as the code committed in the aforementioned bug is concerned), I'm going to close this as INVALID. If someone has a legitimate problem with this change, though, feel free to reopen it.
Comment 3 Dão Gottwald [:dao] 2012-07-20 11:36:10 PDT
The trailing zeros don't seem to reduce ambiguity or serve any other purpose. The concern isn't fingerprintability but useless bytes being sent with every HTTP request.
Comment 4 Gordon P. Hemsley [:GPHemsley] 2012-07-20 12:11:00 PDT
(In reply to Dão Gottwald [:dao] from comment #3)
> The trailing zeros don't seem to reduce ambiguity or serve any other
> purpose. The concern isn't fingerprintability but useless bytes being sent
> with every HTTP request.

The way the quality values are calculated, the quality values are evenly spaced between 0 and 1 based on how many language preferences are listed. If the user has a sufficient number of preferences, 1 decimal place will not be enough to disambiguate them, as multiple preferences will be assigned the same quality value. With 3 decimal places (the max allowed by the HTTP spec), this ambiguity does not occur.

Increasing the number of decimal places in the quality values of the Accept-Language header makes that header more useful for those who actually take advantage of it, which is one of things I am aiming to encourage in my implementation of BCP 47 support. This will support the efforts of creating a truly multilingual Open Web.

Of course, the alternative is to exclude trailing zeroes while also allowing up to 3 decimal places, but that code would be inherently more complex. (I'm not sure how complex, though; I imagine it'd still be well within the realm of feasibility.) The only problem, then, is whether there are implementations that do no properly parse the quality values and will list 0.2 as being less than 0.19 (as in, 2 < 19, rather than 19 < 20).
Comment 5 Dão Gottwald [:dao] 2012-07-20 12:17:24 PDT
Besides just dropping trailing zeros, another option would be to make the number of decimal places depend on the number of languages.
Comment 6 Henri Sivonen (:hsivonen) 2012-07-23 00:04:00 PDT
Sending two insignificant zero digits wastes two bytes per each request, too. I think we shouldn't bloat HTTP requests to cater to bogus server side code that doesn't know how to compare 0.2 and 0.19.

In general, I think Accept-Language has had enough time to show its viability and it has mostly failed, so I think we shouldn't make decisions with the assumption that giving it another decade will make automatic language negotiation work better.
Comment 7 Patrick McManus [:mcmanus] PTO until Sep 6 2012-07-23 04:37:18 PDT
This is definitely a small waste. Can you just file the servers (if any?) as evang bugs if they can't implement the spec?
Comment 8 Patrick McManus [:mcmanus] PTO until Sep 6 2012-07-27 07:34:30 PDT
I think we ought to make a change here by largely reverting 672448 before this goes to beta. I propose using up to 3 decimal places, but not padding with insignificant 0's.

bytes matter. and the only beneficiary of the bloat are un-named (hypothetical?) out of spec servers.

biesi?
Comment 9 Dão Gottwald [:dao] 2012-07-27 09:05:11 PDT
(In reply to Patrick McManus [:mcmanus] from comment #8)
> I propose using up to 3 decimal places, but not padding
> with insignificant 0's.

Note that doing what I said in comment 5 would let us send e.g. 0.7 instead of 0.75 when the extra 0.05 doesn't serve any purpose.
Comment 10 Christian :Biesinger (don't email me, ping me on IRC) 2012-07-27 13:31:12 PDT
yeah, let's not add useless zeroes.
Comment 11 Patrick McManus [:mcmanus] PTO until Sep 6 2012-07-27 18:39:05 PDT
Created attachment 646791 [details] [diff] [review]
backout patch 0

Gordon, I saw that you used checkin-needed to commit this code, so I'll bundle up the backout for you.

we should back it out on 17 and 16, and then you can resumbit (hopefully to 17) a version that doesn't create extra bytes that don't create differentiation. Sorry about the hassle :(

sound ok?
Comment 12 Alex Keybl [:akeybl] 2012-07-30 11:25:29 PDT
We'd consider uplifting, but it's not clear that this needs to be tracked for release given the minimal user impact.
Comment 13 Christian :Biesinger (don't email me, ping me on IRC) 2012-08-06 14:05:25 PDT
Comment on attachment 646791 [details] [diff] [review]
backout patch 0

Review of attachment 646791 [details] [diff] [review]:
-----------------------------------------------------------------

r=biesi assuming this is a straightforward backout. I'm om vacation without a laptop so a reak review is difficult:)
Comment 14 Patrick McManus [:mcmanus] PTO until Sep 6 2012-08-06 18:45:16 PDT
backout from m-c (ff17)
https://hg.mozilla.org/integration/mozilla-inbound/rev/9c94085402c9
Comment 15 Patrick McManus [:mcmanus] PTO until Sep 6 2012-08-06 18:50:26 PDT
Comment on attachment 646791 [details] [diff] [review]
backout patch 0

feature not ready for release as-is.. its been backed out of m-c and is now just on FF16.. requesting a? for backing it out of there.
Comment 16 Ed Morley [:emorley] 2012-08-07 07:34:51 PDT
https://hg.mozilla.org/mozilla-central/rev/9c94085402c9
Comment 17 Lukas Blakk [:lsblakk] use ?needinfo 2012-08-09 10:56:08 PDT
Comment on attachment 646791 [details] [diff] [review]
backout patch 0

backout approved.
Comment 18 Patrick McManus [:mcmanus] PTO until Sep 6 2012-08-09 14:28:49 PDT
https://hg.mozilla.org/releases/mozilla-aurora/rev/a5763ac8cbc3
Comment 19 Gordon P. Hemsley [:GPHemsley] 2012-08-28 09:58:48 PDT
Patrick, thanks for the assist.

For those wondering, a new patch is up for review in bug 672448. It ensures that the additional decimal places are only used when significant.

For example, if you have four languages in your Accept-Language header, they will get 0.75, 0.5, and 0.25 now, instead of their former 0.8, 0.5, and 0.3. (I hope that's not considered too much of a waste of bytes; if it is, we can discuss it in the other bug.) Two languages (as in default 'en-US') will get the 0.5 value as before.

(In reply to Henri Sivonen (not reading bugmail until 2012-09-17) (:hsivonen) from comment #6)
> Sending two insignificant zero digits wastes two bytes per each request,
> too. I think we shouldn't bloat HTTP requests to cater to bogus server side
> code that doesn't know how to compare 0.2 and 0.19.

I agree that we shouldn't cater to bogus value parsers, and my new patch does not do so.

> In general, I think Accept-Language has had enough time to show its
> viability and it has mostly failed, so I think we shouldn't make decisions
> with the assumption that giving it another decade will make automatic
> language negotiation work better.

This, however, I disagree with right now. Part of the reason that servers have not made much effort to use Accept-Language for language negotiation is because browsers did not make the effort to encourage users to use it. The BCP47 effort that I am spearheading hopes to attempt to do that.

Either way, this is a bigger topic of discussion than what this bug is for, and I don't think it should be involved in any decisions about allowing 3-digit quality values.

Note You need to log in before you can comment on or make changes to this bug.