815966 - [build] Integrate IMs and dictionaries with l10n string configuration

Status: RESOLVED DUPLICATE of bug 884752

(Firefox OS Graveyard :: GonkIntegration, defect)

Description

[David Flanagan [:djf]]

We need a mechanism to be able to specify a set of supported languages at build time.  That mechanism must then get the right set of localized strings onto the phone. I suspect someone is working on that already.

But the mechanism really ought to integrate the keyboard, its input methods, and the input method's dictionaries as well. If a build includes Spanish strings, for example, it should also include a Spanish dictionary for word suggestions.  And if a build includes Japanese strings, it should include the Kanji input method and its dictionary.

I don't know enough about localization or the Gaia build process to even know where to begin addressing this, but here's what I know about the keyboard, ims, and keyboard-related dictionaries.

The keyboard is in apps/keyboard/.  The input methods are in apps/keyboard/js/imes/. The keyboard app is the largest one in Gaia because of the dictionary files. We can't just install all dictionaries or we'll run out of space on the device.

The Asian input methods each have a single dictionary in the directory.  These IMs and their dictionaries must always go together. I don't know if we want our build process to be able to copy the entire IM directory into the keyboard app depending on whether we are supporting the language or just copy the dictionary file.

The Latin input method is different: it can work with any latin-based script.  Currently there is a set of about 5 dictionaries included in apps/keyboard/js/imes/latin/, but there is no configuration. If a dictionary is checked in to the gaia tree, it goes into the build. There is no way to alter this dynamically at build time without creating a new git branch. This (latin IM dictionaries) should probably be the first target of an improved build system: we need a way to copy dictionaries into this latin IM directory depending on the set of supported languages.

The js/imes/latin/ directory includes a dict/ subdirectory of dictionary files that also includes a Makefile and python script for building those dictionary files.  They are built from raw XML files that are found in the toplevel dictionaries/ directory of Gaia.  (That toplevel directory and the Makefile and python script that uses it should probably all be under build/, I think.)

Comment 1

[David Flanagan [:djf]]

Cc'ing Stas and Kaze because they know all about localization. Also cc'ing Rudy because he knows the keyboard and worked with me on bug #808849, and this is a follow-up to that bug.  And cc'ing cjones because he and I discussed the need for a configurable build process for keyboard dictionaries today.

Stas or Kaze: does this duplicate any l10n or build bugs you already have on file?

Do any of you think this should be blocking? We obviously wouldn't hold the release for it, but it would be easier to create that release if we had this fixed...

Comment 2

[Tim Guan-tin Chien [:timdream] (please needinfo)]

David,

If this can wait, the ideal solution would be moving these IMEs to different keyboard apps. Once our build config supports specifying which app to build, and Gaia supports 
3rd (multiple) keyboard app (bug 816869), separate these IMEs per app will work with that Gaia.

I am not convinced to some up with some build file magic that could touch app files (yes, I don't like the way our shared script work right now), especially for this kind of specific magic that only deal with a specific keyboard app...

Comment 3

[John Ford [:jhford] CET/CEST Berlin Time]

(In reply to Tim Guan-tin Chien [:timdream] (MoCo-TPE) from comment #2)
> David,
> 
> If this can wait, the ideal solution would be moving these IMEs to different
> keyboard apps. Once our build config supports specifying which app to build,
> and Gaia supports 
> 3rd (multiple) keyboard app (bug 816869), separate these IMEs per app will
> work with that Gaia.
> 
> I am not convinced to some up with some build file magic that could touch
> app files (yes, I don't like the way our shared script work right now),
> especially for this kind of specific magic that only deal with a specific
> keyboard app...

Did this happen?  Can we close this bug?

Comment 4

[Tim Guan-tin Chien [:timdream] (please needinfo)]

3rd-party keyboard is an v1.2 feature being actively developed.
locale files is being built by build locale scripts, something that has already been running.

For the feature specifically required in this bug:

1) Enable/disable keyboard layout based on the select locale on the first run
2) Remove a given spellcheck directory/IME based on # of locales included in the build

I think (1) is done to some extend, but may be buggy. and for (2) I think we should be conservative because people usually expect the same set of features on a certain named OS.

Anyhow, Rudy and Evelyn are the best people to keep track of this feature.

Comment 5

[David Flanagan [:djf]]

I'd forgotten all about this bug.

Meanwhile, I've got bug 884752 assigned to myself. That one is a scaled down version that just aims to make the set of auto-correct dictionaries for the latin keyboard configurable at build time.

Its fine with me if we close this bug in favor of the more up-to-date 884752.

Comment 6

[Rudy Lu [:rudyl] (inactive after 2015/7/3)]

1) Enable/disable keyboard layout based on the select locale on the first run
2) Remove a given spellcheck directory/IME based on # of locales included in the build

Please be informed we have Bug 863719 created to track the changes needed to handle "lang." -> "keyboard layout" mapping in,
  1. build time
  2. settings app
  3. FTU

I guess task 1) would be addressed in Bug 863719 and we need to handle 2) in this bug or Bug 884752.

Comment 7

[Evelyn Hung]

Based on the discussion above, I think we can close this issue as a dup of bug 884752.