2000765 - Firefox uses q=0.5 in Accept-Language header (with two language options), while other browsers use 0.9

Status: RESOLVED FIXED

(Core :: Networking: HTTP, defect, P2)

Description

[Daniel Holbert [:dholbert]]

STR:

1. Visit https://example.org/ with network DevTools open.
2. Inspect one of the requests and find the Accept-Language header, and see what its value is.

EXPECTED RESULTS:
In Chromium and WebKit, the value is the following:

en-US,en;q=0.9

ACTUAL RESULTS:
In Firefox, the value is the following:

en-US,en;q=0.5

The q value probably shouldn't matter too much here, but we've got at least one WebCompat bug where the server cares about it and specifically gives a 403 response when you send two options with q=0.5 -- bug 1998663.

Maybe we should adjust our algorithm so that it produces 0.9 or some higher value as in other browsers, for interoperability? Our algorithm to compute these q values lives here, FWIW:
https://searchfox.org/firefox-main/rev/89b2affdc5d2a1588763e5cb4ac046093c4136a9/netwerk/base/rust-helper/src/lib.rs#66-78

#[no_mangle]
/// Allocates an nsACString that contains a ISO 639 language list
/// notated with HTTP "q" values for output with an HTTP Accept-Language
/// header. Previous q values will be stripped because the order of
/// the langs implies the q value. The q values are calculated by dividing
/// 1.0 amongst the number of languages present.
///
/// Ex: passing: "en, ja"
///     returns: "en,ja;q=0.5"
///
///     passing: "en, ja, fr_CA"
///     returns: "en,ja;q=0.7,fr_CA;q=0.3"
pub extern "C" fn rust_prepare_accept_languages(

Comment 1

[witty.31.06]

I would like to work on this issue, can you assign this to me?

Comment 2

[Valentin Gosu [:valentin] (he/him)]

Sure. I think phabricator also assigns it to you when you submit a patch for it.
Thanks for working on this.

Comment 3

[Valentin Gosu [:valentin] (he/him)]

Note Chrome's impl:
https://source.chromium.org/chromium/chromium/src/+/main:net/http/http_util.cc;l=788-812;drc=7587309936e5da8661bdbf0475dc1d15ee913cca

std::string HttpUtil::GenerateAcceptLanguageHeader(
    const std::string& raw_language_list) {
  // We use integers for qvalue and qvalue decrement that are 10 times
  // larger than actual values to avoid a problem with comparing
  // two floating point numbers.
  const unsigned int kQvalueDecrement10 = 1;
  unsigned int qvalue10 = 10;
  base::StringTokenizer t(raw_language_list, ",");
  std::string lang_list_with_q;
  while (t.GetNext()) {
    std::string language = t.token();
    if (qvalue10 == 10) {
      // q=1.0 is implicit.
      lang_list_with_q = language;
    } else {
      DCHECK_LT(qvalue10, 10U);
      base::StringAppendF(&lang_list_with_q, ",%s;q=0.%d", language.c_str(),
                          qvalue10);
    }
    // It does not make sense to have 'q=0'.
    if (qvalue10 > kQvalueDecrement10)
      qvalue10 -= kQvalueDecrement10;
  }
  return lang_list_with_q;
}

So to match Chrome, Firefox needs to change from "divide evenly among n languages" to "fixed decrement of 0.1 per language, floored at 0.1".

Comment 4

[witty.31.06]

Attachment: Bug 2000765 - Updated q-value generation in Accept-Language Header to match chrome.r=valentin

The algorithm now generates q-value 0.9 for second language in two language option. Now q-weight of each language decrements by 0.1 (minimum 0.1) down the language list.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/0e050ae5116d
https://hg.mozilla.org/integration/autoland/rev/7a81423aa3c7
Updated q-value generation in Accept-Language Header to match chrome.r=valentin,necko-reviewers,devtools-reviewers,android-reviewers,nchevobbe,nalexander

Comment 6

[Valentin Gosu [:valentin] (he/him)]

Release Note Request (optional, but appreciated)
[Why is this notable]: The algorithm used to generate Accept-Language header has now changed to match that of Chrome. The new algorithm is simple, in that it's starts out with 0.9 for any additional languages, and drops down by 0.1 to a minimium of 0.1
[Affects Firefox for Android]: Yes.
[Suggested wording]: Firefox now uses the same quality values (q-values) in Accept-Language headers as other major browsers. The second language preference is now sent with q=0.9 instead of q=0.5, with subsequent languages decreasing by 0.1 each (minimum 0.1). This change fixes compatibility issues with some servers that incorrectly rejected requests with lower quality values.
[Links (documentation, blog post, etc)]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Accept-Language

Comment 7

[agoloman]

https://hg.mozilla.org/mozilla-central/rev/7a81423aa3c7

Comment 8

[Ryan VanderMeulen [:RyanVM]]

Added to the Fx147 relnotes.