13380 - Implement Accept-Language Header according to HTTP/1.1

Status: VERIFIED FIXED

(Core :: Internationalization, defect, P3)

Description

[Katsuhiko Momoi]

We need to implement Accept-Language backend to go with
the Accept-Language UI according to HTTP/1.1.

The final draft of the UI proposal is here:

http://rocknroll/users/momoi/publish/seamonkey/50intlui.html

(The illustration will be revised soon to relflect a proposal
for RDF-based "customize" menu, e.g. see the side-bar customize menu)

In HTTP1.1, in addition to language-range list, we need to provide
"q" values. The Pref. UI will have an ordered/ranked list of lang(s).
We then should assign decreasing values of "q" to the user selected
list. Here's the spec from RFC 2616:

------------------------------------------------------
14.4 Accept-Language

   The Accept-Language request-header field is similar to Accept, but
   restricts the set of natural languages that are preferred as a
   response to the request. Language tags are defined in section 3.10.

       Accept-Language = "Accept-Language" ":"
                         1#( language-range [ ";" "q" "=" qvalue ] )
       language-range  = ( ( 1*8ALPHA *( "-" 1*8ALPHA ) ) | "*" )

   Each language-range MAY be given an associated quality value which
   represents an estimate of the user's preference for the languages
   specified by that range. The quality value defaults to "q=1". For
   example,

       Accept-Language: da, en-gb;q=0.8, en;q=0.7

   would mean: "I prefer Danish, but will accept British English and
   other types of English." A language-range matches a language-tag if
   it exactly equals the tag, or if it exactly equals a prefix of the
   tag such that the first tag character following the prefix is "-".
   The special range "*", if present in the Accept-Language field,
   matches every tag not matched by any other range present in the
   Accept-Language field.

      Note: This use of a prefix matching rule does not imply that
      language tags are assigned to languages in such a way that it is
      always true that if a user understands a language with a certain
      tag, then this user will also understand all languages with tags
      for which this tag is a prefix. The prefix rule simply allows the
      use of prefix tags if this is the case.

   The language quality factor assigned to a language-tag by the
   Accept-Language field is the quality value of the longest language-
   range in the field that matches the language-tag. If no language-
   range in the field matches the tag, the language quality factor
   assigned is 0. If no Accept-Language header is present in the
   request, the server
   SHOULD assume that all languages are equally acceptable. If an
   Accept-Language header is present, then all languages which are
   assigned a quality factor greater than 0 are acceptable.

   It might be contrary to the privacy expectations of the user to send
   an Accept-Language header with the complete linguistic preferences of
   the user in every request. For a discussion of this issue, see
   section 15.1.4.

   As intelligibility is highly dependent on the individual user, it is
   recommended that client applications make the choice of linguistic
   preference available to the user. If the choice is not made
   available, then the Accept-Language header field MUST NOT be given in
   the request.

      Note: When making the choice of linguistic preference available to
      the user, we remind implementors of  the fact that users are not
      familiar with the details of language matching as described above,
      and should provide appropriate guidance. As an example, users
      might assume that on selecting "en-gb", they will be served any
      kind of English document if British English is not available. A
      user agent might suggest in such a case to add "en" to get the
      best matching behavior.
------------------------------------------------------

Other items to consider:

1. The values for this should be easily set depending on
   your default language at install time. (Cf. the notion of the
   default language in the UI proposal above.) This should not have
   to involve L10n modification.

Comment 1

[Frank Tang]

This was in the 4.x netlib. We can do the UI work but Necko folks need to take
the value from pref and send out in the "Accept-Charset" HTTP header. As I
understand, LDAP gateway is depend on this.

momoi, please tell warran in the bug report which pref contains the value.
Thanks.

Reassign this to warran.

Comment 2

[Warren Harris]

-> Gagan

Comment 3

[Katsuhiko Momoi]

The 4.x Accept-Language prefs.js uses the following format:

user_pref("intl.accept_languages", "ja,en");

Comment 4

[Katsuhiko Momoi]

Please note that this is Accept-Language header. (HTTP_ACCEPT_LANGUAGE)
A bug for the Accept-Charset header will be filed separately.

Comment 5

[Frank Tang]

sorry, I make some mistake in my early comment. The LDAP server is
depend on Accept-Charset header, not Accept-Language header.

Comment 6

[leger]

Moving Assignee from gagan to warren since he is away.

Comment 7

[Warren Harris]

Moving what's not done for M12 to M13.

Comment 8

[Warren Harris]

Back to Gagan for M13.

Comment 9

[Warren Harris]

Back to Gagan for M13.

Comment 10

[Gagan]

Fixed and about to be checked in ...

Comment 11

[Gagan]

...and done.

Comment 12

[Katsuhiko Momoi]

Hi, did you implement the Q-value algorithm? 
We don't have a Pref UI working yet, but if it did,
it needs to add some pre-determined q value as the user
adds more than 1 languages to this list.

The following is an example of what happens to Q values 
when multiple entries are made to IE5's Accept-Laguage list via
a Pref UI. The UI itself does not allow you to set this
and so these values are set by some formula which assigns 
a smaller value than 1 to an additional entry, and somewhat
smaller value to the 2nd addition, etc.

IE 5 implementation example.

ja,en;q=0.5
ja,en;q=0.7,fr;q=0.3
ja,en;q=0.8,fr;q=0.5,ru;q=0.3

Comment 13

[Teruko Kobayashi]

I verified this in 2000021408 build.

Comment 14

[Robin Lionheart]

In Mozilla/5.0 (Windows; U; Win98; en-US; m16) Gecko/20000529, the Accept-
Language header is at variance with the Language preferences. I edited my
Language Prefs, removing English [en] entirely and adding Esperanto [eo] as the
only language. HTTP requests continued to be sent out with the incorrect header
"Accept-Language: en".

Comment 15

[Gagan]

Its wrong to reopen this bug. I have filed a separate one for the problem 
mentioned in bug 41046

Comment 16

[Teruko Kobayashi]

I verified this in 2000-06-26-08 build.