1179054 - Include Japanese IME in the Nightly build

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::Keyboard, defect)

Description

[Tim Guan-tin Chien [:timdream] (please needinfo)]

To make Japanese IME included in the build, we would need to do the following:

1) Generate metadata in the build process to get Japanese IME classified as downloadable, and with correct id/name etc. to construct URLs to CDN and have data gettable from InputMethodGlue#getData

https://github.com/mozilla-b2g/gaia/blob/b8b329a79ed81ee25681ae391db167a6ceff3cff/apps/keyboard/build/layout_configurator.js#L115
https://github.com/mozilla-b2g/gaia/blob/b8b329a79ed81ee25681ae391db167a6ceff3cff/apps/keyboard/js/keyboard/input_method_database_loader.js
https://github.com/mozilla-b2g/gaia/blob/b8b329a79ed81ee25681ae391db167a6ceff3cff/apps/keyboard/js/keyboard/input_method_manager.js#L284-L286

2) Patch jskanji.js so that it would load dict.json by asking for the data from InputMethodGlue#getData, and not by itself with XHR.

https://github.com/mozilla-b2g/gaia/blob/b8b329a79ed81ee25681ae391db167a6ceff3cff/apps/keyboard/js/imes/jskanji/jskanji.js#L1103

This part is a bit tricky. It turned out jskanji takes json as data and it try to cache it by loading it in IndexedDB. This is the old paradigm which try save memory by loading things into IndexedDB (and only query each bit when needed), but at the expense of consume a huge chunk of memory when loading the initial JSON at first place. The bottom line is we can't have the JSON object consistently live in memory.. so the data format has to change to custom design Typed Array.

I don't know if we want to do that first before making data downloadable. If so what's implemented in JSZunyin IME might help: https://github.com/timdream/jszhuyin/blob/5f9606ba1b2258f26137227cf8568326693e5b61/lib/storage.js#L3

Comment 1

[Brian Birtles (:birtles, Feb 26~ parental leave)]

I see this bug is about including the existing Japanese IME in the build, but after discussing with some of the Mozilla Japan engineers we think it might be better to re-use an existing open source IME.

The reason is that Japanese IMEs are quite involved. They require predictive text (予測変換) which requires a very large dictionary and context-sensitive matching (連文節変換) which involves many clever tricks for determining the important words and their common pairings.

Some open source options include:

* OpenWnn
  C API - can use Emscript.
  Includes a dictionary (that compresses well).
  Is used in AOSP.
  Supports predictive suggestions.

* OpenPobox
  This is the IME that specialized in predictive suggestions.
  It is old (development finished in the first half of the 2000s)
  The engine has a C API so we can use Emscripten.

* Mozc (OSS version, Google Japanese IME)
  Does not include the predictive dictionary.
  It appears difficult to extract just the engine component.

Makoto will work on a prototype based on OpenWnn.

Comment 2

[BMO Automation]

Firefox OS is not being worked on