1321298 - migration lacks cldr plural forms for various Firefox localizations

Status: RESOLVED FIXED

(Localization Infrastructure and Tools :: Fluent Migration, defect)

Description

[Axel Hecht [:Pike]]

In order to be able to successfully run migrations on all our languages, we'll need to pad languages that cldr doesn't have.

We probably also need such a bug for our runtime js API, gandalf?

Missing are:

ach, an, cak, csb, gn, hto, kok, lij, ltg, mai, oc, pbb, qvi, rw, sat, son, trs, tsz

That's way more than I expected.

Comment 1

[Zibi Braniecki [:zbraniecki][:gandalf]]

Porting those CLDR plural forms to our JS API is not going to be very helpful since we're about to switch to Intl.PluralRules.

We should:

 * write a patch against Gecko's CLDR
 * upstream it
 * use it to patch our JS API

If you point me to where we take plural forms from now, I'll take this bug.

Comment 2

[Axel Hecht [:Pike]]

So we use "something" in gecko, https://dxr.mozilla.org/l10n-mozilla-aurora/search?q=path%3Aintl.properties+pluralRule%3D&redirect=false. I'm not sure to which extent those are vetted, and they also don't make any statement about the floats behavior.

Comment 3

[Zibi Braniecki [:zbraniecki][:gandalf]]

ach, an, cak, gn, hto, kok, lij, mai, oc, sat, son, tsz - pluralRule: 1
csb - pluralRule: 9
ltg - pluralRule: 3

pbb, qvi, rw, trs - ?

so, we'd need to add csb and ltg to cldr. Not sure about the other 4 since I don't see them on this list.

Comment 4

[Staś Małolepszy :stas]

There are two tasks here which I'd like to split up:

  a) supporting these locales in migrations so that we don't lose good translations which are currently stored in legacy files,
  b) supporting these locales in the runtime so that we can show theses good migrated translations to the users.

I would like to make this bug specifically about a).  We can take care of b) at a later time. It's not related to the migration module and it has a) as a dependency.

To fix a) we could patch our copy of the CLDR data in https://github.com/projectfluent/python-fluent/tree/master/fluent/migrate/cldr_data. Would that be enough?

Comment 5

[Staś Małolepszy :stas]

When we fix this, we should also fix the typo in https://github.com/projectfluent/python-fluent/blob/479af5dad7691a357763da0d67873819b90b5f67/fluent/migrate/context.py#L63; this code should read:

    self.plural_categories = get_plural_categories('en')

Comment 6

[Staś Małolepszy :stas]

(In reply to Axel Hecht [:Pike] from comment #0)

> ach, an, cak, csb, gn, hto, kok, lij, ltg, mai, oc, pbb, qvi, rw, sat, son,
> trs, tsz

I ran the migration from bug 1411012 on all locales I cloned using the following all-locales file:

    https://dxr.mozilla.org/mozilla-central/rev/1624b88874765bf57e9feba176d30149c748d9d2/browser/locales/all-locales

Only later did I realize that some locales are missing from it, namely:

    csb, hto, kok, pbb, qvi, rw, sat

I found one more for which our copy of CLDR doesn't have plural categories:

    ia

Lastly, the following two errored out during the annotation phase (bug 1426665); I don't know if plurals would work for them:

    ne-NP, si

Comment 7

[Staś Małolepszy :stas]

Bug 1432163 added missing plurals to compare-locales and bug 1415844 switched python-fluent's migration to only use the data stored in compare-locales rather than a copy of the CLDR data. Marking as fixed.