730625 - Important languages are missing from the preferred language dialog

Status: RESOLVED FIXED

(Firefox :: Settings UI, defect)

Description

[Amir Aharoni]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:12.0a2) Gecko/20120216 Firefox/12.0a2
Build ID: 20120216042740

Steps to reproduce:

I went to Preferences -> Content -> Languages to add the Uzbek language to the list of my preferred languages (accept-language).


Actual results:

Uzbek was not in the list.


Expected results:

Uzbek (uz; 18 mln speakers) should have been in the list. So should:
* Chuvash (cv; 1.6 mln speakers)
* Bashkir (ba; 1.4 mln speakers)

There are many more that i'd like to see added, but for now i limited myself to ones with which i'm working and which have over 1 mln speakers.

See also Bug 730624.

Comment 1

[Amir Aharoni]

Oh, and also Uyghur (ug; 8 mln speakers).

Comment 2

[Toni Hermoso Pulido]

Hi, I think this should be added in:

http://mxr.mozilla.org/mozilla-central/source/toolkit/locales/en-US/chrome/global/languageNames.properties
Amir, you can file a patch against that file. Let me know if you need a hand.

Note: BCP-47 is going to be integrated in the future:
https://wiki.mozilla.org/User:GPHemsley/BCP_47

Comment 3

[Amir Aharoni]

I see Uzbek (uz) in that file, but not in the preferences dialog.

I also see Afar (aa) in the file, but not in the preferences dialog.

Comment 4

[Axel Hecht [:Pike]]

That's controlled in intl/locale/src/language.properties.

I'm not convinced that some arbitrary numbers are the right criteria for which locales should be in there.

Comment 5

[Amir Aharoni]

(In reply to Axel Hecht [:Pike] from comment #4)
> That's controlled in intl/locale/src/language.properties.
> 
> I'm not convinced that some arbitrary numbers are the right criteria for
> which locales should be in there.

Honestly, i'm not sure what should be the right criteria. Complete clever support for BCP-47 would be a good idea, of course, and resolving bug 730624 will make it possible until then.

In any case, Uzbek, an official language of a rather large independent country, should definitely be there.

Comment 6

[Gordon P. Hemsley [:GPHemsley]]

All of these languages are already in toolkit/locales/en-US/chrome/global/languageNames.properties, and have been since the move to Mercurial. The problem is, most are not turned on in intl/locale/src/language.properties [2]. ('cv' is, and does appear in the Language dialog.)

I don't really understand the need for having to opt-in to language support, and it is my understanding that that requirement would go away with full BCP 47 support.

In the meantime, the patch to turn them on should be fairly simple, assuming there isn't any legitimate reason they're turned off in the first place. It's just a matter of changing 'false' to 'true' for the given languages.

As for bug 730624, you can already manually set intl.accept_languages, so adding an interface that just lets you input additional arbitrary language tags wouldn't be very helpful. However, it is desired to improve the Language preferences dialog altogether, and there's some documentation about that here: [3].

[1] http://hg.mozilla.org/mozilla-central/annotate/390d0ca57270/toolkit/locales/en-US/chrome/global/languageNames.properties
[2] http://hg.mozilla.org/mozilla-central/annotate/580c63190d26/intl/locale/src/language.properties
[3] https://wiki.mozilla.org/User:GPHemsley/BCP_47#Language_preferences_UI

Comment 7

[Simon Montagu :smontagu]

I think (and have thought for some time) that the true/false in language.properties isn't expressing any meaningful distinction. In the mean time before we rework the language selection UI, we might as well just set everything to true. It won't make the dialog significantly less wieldy than it is now -- currently there are 223 lines with "true" and 42 with "false"

Comment 8

[Simon Montagu :smontagu]

Attachment: Set all languages to true

This is the quickest and dirtiest fix: just set every language to "true" in the properties file.

A slightly more intelligent approach would be to change the parser in browser/components/preferences/languages.js and make it just ignore the value.

Comment 9

[Gordon P. Hemsley [:GPHemsley]]

(In reply to Simon Montagu from comment #7)
> I think (and have thought for some time) that the true/false in
> language.properties isn't expressing any meaningful distinction. In the mean
> time before we rework the language selection UI, we might as well just set
> everything to true. It won't make the dialog significantly less wieldy than
> it is now -- currently there are 223 lines with "true" and 42 with "false"

Fine by me. Though I should note that I was considering an intermediate step between the way things are now and the way things would be with the refactored UI. Bug 716321 seeks to bring languageNames.properties up-to-date with commonly used language subtags, for a new total of 322. So I was envisioning an intermediate step where we get rid of language.properties and just use languageNames.properties directly.

(Bug 666662 seeks to differentiate between the shorter lists of localized subtags and the long list of all registered subtags. It's only the full list that would become too unwieldy for the current UI, particularly with regard to the extremely large number of possible combinations.)

Comment 10

[Gordon P. Hemsley [:GPHemsley]]

(In reply to Simon Montagu from comment #8)
> Created attachment 609509 [details] [diff] [review]
> Set all languages to true
> 
> This is the quickest and dirtiest fix: just set every language to "true" in
> the properties file.

In that case, we may want to consider dealing with bugs like bug 573320 at the same time.

Comment 11

[:Gavin Sharp [email: [e-mail]]]

Can we just remove language.properties instead?

Comment 12

[Gordon P. Hemsley [:GPHemsley]]

(In reply to Gavin Sharp (use [e-mail] for email) from comment #11)
> Can we just remove language.properties instead?

I was going to say yes, but then I realized: The whole point of language.properties is for things like bug 573320. That file is needed to be able to suggest subtag combinations (like 'en-US' or 'en-GB' or 'en-IN').

However, I see no reason we need to keep the accept business. Since we're OK with just hacking everything to true, why not just turn it into a simple list of language tag suggestions? (If it's in the list, it's "accepted" and shown in the Language pref UI; if it's not in the list, it's not.)

Comment 13

[Simon Montagu :smontagu]

Apart from the problem with the subtag combinations, changing the format of language.properties will require changing the parsing in both firefox and seamonkey, which I would like to avoid if possible (FWIW, bug 730624 doesn't apply to seamonkey, so missing languages are a less serious problem there)

Comment 14

[Jonathan Kew [:jfkthame]]

Comment on attachment 609509 [details] [diff] [review]
Set all languages to true

Review of attachment 609509 [details] [diff] [review]:
-----------------------------------------------------------------

Let's do this for now, to address the immediate problem of some "important" languages that are missing from the UI for no apparent reason. We definitely want to re-work this more extensively as part of proper BCP47 support....

Comment 15

[Gordon P. Hemsley [:GPHemsley]]

Attachment: List of affected language tags

For the record, here is the list of records for the affected tags.

Note that this list includes a number of macrolanguage/collection subtags. I'd suggest that we think a little harder about those, but I know there are already locale teams in the works using some of them, so I'm not sure what to do.

Note also that these changes exclude 3-char language subtags, as well as the aforementioned en-IN. I know that this patch was only intended to flip existing bits, but I thought I should note for the record that we're still missing a bunch of "important" language tags.

Comment 16

[Gordon P. Hemsley [:GPHemsley]]

One other thing: Some of these may have originally been turned off for political reasons. (The one that is probably the most political is Macedonian; since there is a conflict about which region the name Macedonia belongs to, it is like that 'mk-MK' was added to the list without plain 'mk' in an attempt to appease the conflict.)

Comment 17

[Simon Montagu :smontagu]

https://hg.mozilla.org/integration/mozilla-inbound/rev/c8e6770642a0

Comment 18

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/c8e6770642a0