583793 - application interface language set to LANG, ignores LANGUAGE

Status: RESOLVED DUPLICATE of bug 566590

(Core :: Internationalization, defect)

Description

[David Planella]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; ca; rv:1.9.2.8) Gecko/20100723 Ubuntu/10.10 (maverick) Firefox/3.6.8
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; ca; rv:1.9.2.8) Gecko/20100723 Ubuntu/10.10 (maverick) Firefox/3.6.8

(Forwarded from the downstream bug report at https://bugs.launchpad.net/ubuntu-translations/+bug/550222)

Firefox appears to misinterpret the locale environment variables set by the Language Support tool. It uses LANG instead of LANGUAGE.

Use case:
I am a software engineer, so I prefer to have my user interface language set to English ("en_GB" if possible, fall back to "en"). I'm Dutch and live in the Netherlands, so all measurements, paper sizes, and currency localization are set to "nl_NL". This works for all GTK+ applications as expected. Firefox however sets its interface language to Dutch instead of English.

As far as I can tell, the Language Support tool in Lucid Lynx sets LANGUAGE and LANG:

LANG=nl_NL.utf8
LANGUAGE=en_GB:en

Reproducible: Always

Steps to Reproduce:
1. Set the LANGUAGE environment variable to LANGUAGE=en_GB:en
2. Set the LANG environment variable to LANG=nl_NL.utf8
3. Start Firefox
Actual Results:  
Firefox starts up in Dutch, as per the value in LANG

Expected Results:  
Firefox should start in English, as per the value in LANGUAGE, which seems to be ignored.

According to [1], "The LANGUAGE environment variable is examined first to determine the message catalogs to be used.". This is the behaviour for all GTK applications in a Linux-based OS.

[1] http://www.openi18n.org/docs/html/LI18NUX-2000-amd4.htm

Possibly related downstream bugs:
* https://bugs.launchpad.net/bugs/539761 (LC_MESSAGES misinterpreted)

Comment 1

[Axel Hecht [:Pike]]

http://mxr.mozilla.org/mozilla-central/source/intl/locale/src/nsLocaleService.cpp#186 checks for LANG.

Moving to core intl for triage, as I know jack about the specs there.

Comment 2

[Simon Montagu :smontagu]

CC-ing Wolfgang, who is working on this code in bug 568459

Comment 3

[Wolfgang Rosenauer [:wolfiR]]

LANGUAGE should be supported I'd say. It's a gettext feature and therefore petty much standard on Linux systems.
With current nsLocale etc. it's a bit of a pain to implement it apparently though.
For example LANGUAGE can be a list of colon seperated locale names and nsLocale has no data structure for LANGUAGE. I'm currently a bit into the code and will try if I can come up with a patch.

Comment 4

[Axel Hecht [:Pike]]

I would like to add one thing: I read David's env use as a hack to actually get a en_NL. We're *very* unlikely to support that, IMHO.

Comment 5

[David Planella]

Axel,

The bug is not about adding any kind of hack, and in no way I've (well, the original downstream poster) suggested creating or supporting a en_NL locale.

The bug is about supporting the LANGUAGE gettext environment variable in Linux, a well established standard in all distributions [1] [2].

I'll be happy to reword the bug description to make it clearer, if necessary.

Thanks.

[1] http://www.gnu.org/software/hello/manual/gettext/Locale-Environment-Variables.html
[2] http://www.openi18n.org/docs/html/LI18NUX-2000-amd4.htm

Comment 6

[Axel Hecht [:Pike]]

My comment is about filing more bugs to get non-language pieces of our localized content to actually follow the LANG instead of the LANGUAGE setting. Say, time formats, or maybe at some point search engines.

Comment 7

[Wolfgang Rosenauer [:wolfiR]]

I've tested a minimalistic approach to support LANGUAGE. The biggest issue with that is that multiple definitions are not really supported. The other thing is that I reuse/override LC_MESSAGES internally but I understand that this is actually the result anyway.


(In reply to comment #6)
> My comment is about filing more bugs to get non-language pieces of our
> localized content to actually follow the LANG instead of the LANGUAGE setting.
> Say, time formats, or maybe at some point search engines.

This is sensible but not primary topic of this bug as as I understand.

Comment 8

[David Planella]

(In reply to comment #7)
> I've tested a minimalistic approach to support LANGUAGE.

That's great, thanks for working on that!

> The biggest issue with that is that multiple definitions are not
> really supported.

Are you referring here to the behaviour of supporting multiple language fallbacks (LANGUAGE=de:es)?

It would be nice to implement this, but having support for the main locale but be a good start already. 

> The other thing is that I reuse/override LC_MESSAGES internally but I
> understand that this is actually the result anyway.

If I understand correctly that you are referring to LANGUAGE overriding not only LANG, but also the LC_xxx categories, that's fine. According to the gettext documentation, the order of preference (unless LC_ALL or LANG is specified as "C") is:

1. LANGUAGE
2. LC_ALL
3. LC_xxx, according to selected locale category: LC_CTYPE, LC_NUMERIC, LC_TIME, LC_COLLATE, LC_MONETARY, LC_MESSAGES, ...
4. LANG

Comment 9

[Wolfgang Rosenauer [:wolfiR]]

(In reply to comment #8)
> > The biggest issue with that is that multiple definitions are not
> > really supported.
> 
> Are you referring here to the behaviour of supporting multiple language
> fallbacks (LANGUAGE=de:es)?

Yes.
 
> It would be nice to implement this, but having support for the main locale but
> be a good start already. 

Mozilla has no real idea which locales are supported. AFAIK at some point when a locale is supposed to be loaded and it fails it just falls back to en_US.
But I agree that it would be nice to support it.

> If I understand correctly that you are referring to LANGUAGE overriding not
> only LANG, but also the LC_xxx categories, that's fine. According to the
> gettext documentation, the order of preference (unless LC_ALL or LANG is
> specified as "C") is:
> 
> 1. LANGUAGE
> 2. LC_ALL
> 3. LC_xxx, according to selected locale category: LC_CTYPE, LC_NUMERIC,
> LC_TIME, LC_COLLATE, LC_MONETARY, LC_MESSAGES, ...
> 4. LANG

- LC_* are authoritative. Every LC_* which is not set manually will be set to 
  the value of LANG. 
- LC_ALL overrides all other LC_* settings.
- LANGUAGE exists besides LC_MESSAGES to override it for UI language

That fits your above description with the exception that LANGUAGE is only overriding LC_MESSAGES effectively.

Comment 10

[Jeroen Hoek]

(I am the downstream reporter)

Is there any kind of time-frame for resolving this bug? It has been a problem for quite a while now. The upcoming round of distro releases probably won't ship a fix either.

Comment 8:
Full LANGUAGE support would be great, but as you point out, even just supporting the first locale mentioned would help a lot.

Comment 11

[Wolfgang Rosenauer [:wolfiR]]

Status from my POV:
I most likely don't have the time to implement full LANGUAGE support for a list.

I'm not sure if the fast partial support is enough for Mozilla to accept it.
I will attach my current patch nevertheless.

Comment 12

[Wolfgang Rosenauer [:wolfiR]]

Attachment: simple patch

Patch only supporting one item in the $LANGUAGE environment.

Comment 14

[Jo Hermans]

Changing title, since it also affects other applications like Thunderbird.

Comment 15

[John Vandenberg]

It also affects other Linux distros.