libchewing + emscripten as a javascript IME engine

RESOLVED WONTFIX

Status

RESOLVED WONTFIX
6 years ago
6 months ago

People

(Reporter: laszio.bugzilla, Assigned: laszio.bugzilla)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment, 2 obsolete attachments)

(Assignee)

Description

6 years ago
This bug is created to track the status of translating libchewing into javascript by emscripten. Currently libchewing can be compiled and run on PC with emcripten's virtual file system. To work properly on Firefox OS, it's needed to:

1. Take care of IO. Emscripten's virtual fs loads all files into memory and eats a lot of memory.
2. Besides vfs's overhead, make sure it'll fit in FirefoxOS's limited amount of memory.
3. Make sure the performance is acceptable.
(Assignee)

Comment 1

6 years ago
Created attachment 727543 [details]
WIP -- Steps and files to compile and integrate libchewing into gaia.

Attached are steps and files to compile and integrate libchewing into gaia.

Currently, it takes about 3s to bootstrap and requires about 17.xMB memory on unagi. Refinements, especially to memory usages, are going on.

A more comprehensive document will be released later.
What is the target memory usage of this module?
(Assignee)

Comment 3

6 years ago
(In reply to Justin Lebar [:jlebar] from comment #2)
> What is the target memory usage of this module?

I tried to run a native version of libchewing and monitor the memory usage. It looks like that, without exposing the capability of MMU to javascript, what we can do best is a little more than 9452KB: 

VmPeak:     9452 kB
VmHWM:      1212 kB
VmStk:       148 kB

I tunned the emulated native heap to 10MB (it's 16MB by default) and the emulated stack to 256KB (5MB by default, within the heap). Now it requires about 11.10MB which is quite close to the lower bound.

I'm trying to measure other javascript based IMEs and will show the results later.
OOC what are the RSS numbers there?
(Assignee)

Comment 5

6 years ago
It's VmHWM, the peak of RSS.
I see.  I think this is a pretty compelling case for exposing decommit semantics to emscripten, then.
(Assignee)

Comment 7

6 years ago
Created attachment 729420 [details]
Steps and files to compile and integrate libchewing into gaia.

* Rename emchewing.sh to example.sh.
* Fine tune stack and total memory size.
* Added testjs.sh for running built-in tests in libchewing.
Attachment #727543 - Attachment is obsolete: true
(Assignee)

Comment 8

6 years ago
Here is the memory usage (right after gc) of another IME, Zhuyin:

28.25 MB (37.53%) -- window(app://keyboard.gaiamobile.org/index.html#show=312)

and it spent 4s on initialization.

Zhuyin is manually rewritten from its Objective-C version, McBopomofo. The databases are translated into JSON beforehand and loaded into JavaScript arrays at runtime.
(In reply to Ting-Yuan Huang from comment #8)
> Zhuyin is manually rewritten from its Objective-C version, McBopomofo. The
> databases are translated into JSON beforehand and loaded into JavaScript
> arrays at runtime.

The detail about this statement is incorrect -- JSZhuyin is manually written by me, with a lousy algorithm unrelated to McBopomofo. I just copied the database from that because there are no other phrases database available under compatible license.

(libchewing & libchewing-data are licensed under LGPL, incompatible with Gaia)
(Assignee)

Comment 10

5 years ago
Created attachment 771266 [details]
Steps and files to compile and integrate libchewing into gaia.

The patch to emscripten in the first step in the example is no longer needed; It's already landed.
(Assignee)

Updated

5 years ago
Attachment #729420 - Attachment is obsolete: true

Comment 11

6 months ago
Firefox OS is not being worked on
Status: NEW → RESOLVED
Last Resolved: 6 months ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.