866351 - Contribute locale data for Mozilla locales to CLDR

Status: RESOLVED WONTFIX

(Core :: Internationalization: Localization, task)

Description

[Norbert Lindenberg]

Through the ECMAScript Internationalization API, which in SpiderMonkey is implemented using ICU, which uses CLDR, Mozilla products now depend on CLDR for part of their localization.

CLDR is missing data for some languages/locales that Mozilla locales into. Comparing http://www.mozilla.org/en-US/firefox/all/ as of 2013-03-16 with the ICU locales as of version 50.1.2 (the one currently used by SpiderMonkey), I found the following languages missing from ICU:
- Acholi
- Asturian ("unconfirmed" in CLDR)
- Frisian
- Gaelic ("unconfirmed" in CLDR)
- Kashubian
- Ligurian
- Maithili
... and the languages under development:
- Kurdish
- Northern Sotho ("unconfirmed" in CLDR)
- Songhai

It would help if the Mozilla localization team could contribute data for these locales to CLDR, especially in the areas of collation, number formatting, and date and time formatting.

Comment 1

[Jeff Walden [:Waldo]]

Axel, do you still handle localization stuff, and could you point the right people at this?  (I am woefully out-of-date in my knowledge of product localization these days, apologies if this has switched to someone else and I never noticed.)

Comment 2

[Axel Hecht [:Pike]]

This sounds like stuff on my lap. (Jeff, still doing l10n, but not doing it alone, so I have the priviledge to punt things to the next person.)

There's two things here, IMHO:

We should adjust the data in our copy of cldr in icu to work for us.

We should try to upstream the changes we want. I don't know how good the process works today, though. Community members have done that in the past, with mixed results.

I suspect that we need to maintain our own fork of cldr, and import that into the different file formats that icu uses?

Comment 3

[Norbert Lindenberg]

I have no say in Mozilla localization (or CLDR), but personally I would do things in a different order:

1) Try to work with the CLDR team to integrate changes that Mozilla needs into CLDR. When we first discussed this by email in October 2012, Dwayne and Friedel reported problems encountered with contributions to CLDR. It would probably help if a Mozilla rep collected a list of such problems and met with the CLDR team to discuss them. Even better, Mozilla could join the Unicode Consortium and the CLDR TC, which would give you a bigger say and votes in CLDR.

2) If 1) fails, then create a fork of CLDR, maintain it within Mozilla, and use it to generate the locale data sources in the Mozilla copy of ICU. I'm afraid the maintenance of the data and the tools to maintain the data will become a big long-term burden, hence 1) first.

Comment 4

[Axel Hecht [:Pike]]

I don't see us having a way around keeping our own fork of CLDR.

At the point at which we intend to add a locale of Firefox, we need CLDR data on aurora in a matter of a couple of weeks.

Going through a CLDR process to get into a CLDR release, and then get that release into ICU, and then get that ICU into mozilla, and that version of mozilla onto aurora .... just way too much time.

Comment 5

[Jeff Walden [:Waldo]]

Would that "way too much time" be a one-time only thing, until CLDR/ICU produce a release with info for the Mozilla locales?  This stuff is all way too domain-specific for me to have much faith in our ability to competently fork it while waiting for any contributions we make to be upstreamed.  I could live with it for a few releases til our changes filtered back up to an ICU release.  But in the long run this particular bit of imported code/data seems especially unwise to ever touch.  (And let's set aside distros that want to use a system ICU, which obviously won't have anything we fold into ICU via forks.)

Comment 6

[Axel Hecht [:Pike]]

We won't be able to anser the outcome of us trying to upstream change requests until we did that a bunch of times.

There's obviously a need to downstream the changes from cldr into our fork.

And yes, this requires project management that we don't have resourced at this point.

Comment 7

[Zibi Braniecki [:zbraniecki][:gandalf]]

Flod - Flagging you to place it on your radar. It may be useful as a tracking bug for all our efforts to upstream and sync our data with CLDR?

Comment 8

[Francesco Lodolo [:flod]]

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #7)
> Flod - Flagging you to place it on your radar. It may be useful as a
> tracking bug for all our efforts to upstream and sync our data with CLDR?

Not sure yet. Taking so it doesn't fall out of radar.

Comment 9

[Francesco Lodolo [:flod]]

Unassigning, since this is not something I have resources to work on.

Comment 10

[Francesco Lodolo [:flod]]

We still don't have resources to work on this, and we might end up using a completely different road to get data in CLDR.