1139255 - Extend keyboard prediction/auto correction functionality for agglutinative languages

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::Keyboard, defect)

Description

[John Lu [:mnjul][Please NI or I'll miss]]

As said in bug 1119426 comment 3, our current Latin IME prediction/auto correction mechanism is not suitable for agglutinative languages: Naively brute-force concatenating dictionary words to form all possible agglutinated words does not deliver quality results. We need to think about how to extend the prediction engine to deal with this kind of languages.

This might become a meta bug as we'll probably need quite some engineering inputs plus localization inputs to finalize the feature.

Comment 1

[John Lu [:mnjul][Please NI or I'll miss]]

Howie/Tim, it appears that I can't nominate feature-b2g. Could any of you help nominate this as a 3.0 feature? Thanks!

Comment 2

[Tim Guan-tin Chien [:timdream] (please needinfo)]

Done. Would you mark the dependencies as well please?

Comment 3

[John Lu [:mnjul][Please NI or I'll miss]]

I did a quick sweep on bug 1112707's engineering bugs. Marked bugs here are either stated in comments as having agglutinative property, or are said as such in http://en.wikipedia.org/wiki/Category:Agglutinative_languages .

Comment 4

[Dwayne Bailey]

(In reply to John Lu [:mnjul] [MoCoTPE] from comment #3)
> I did a quick sweep on bug 1112707's engineering bugs. Marked bugs here are
> either stated in comments as having agglutinative property, or are said as
> such in http://en.wikipedia.org/wiki/Category:Agglutinative_languages .

Ah glad to see this has been added as a blocker for various languages.

We pretty much can't do anything for Bantu languages (Xhosa, Zulu, Swahili, etc) without this support, some of these are highly agglutinative which might impact what we design here.

Just flagging this so that we're aware that any prediction for African languages for partners is potentially impacted by this bug.

Comment 5

[Kevin Scannell]

Thanks John for filing this.  There are probably more than 250 *million* speakers of a Bantu language in Africa; it's incredibly important that Mozilla get this right - I've said to many people that I think Firefox OS wins on language more than anything else.   

I'd be happy to talk about technical solutions with the engineers that end up working on this.

Comment 6

[Bruce Huang [:bhuang] (inactive)]

I think this will probably be a long running feature considering the testing we'll need.  I suggest we fork off the current Latin autocorrect engine and use the updated engine in this bug for the affected languages first.
Is it possible to add some examples on to the bug so that we can begin to put together some patterns to cover this?

Comment 7

[Kevin Scannell]

Hi Bruce, I'm discussing possible solutions with Friedel Wolff at the moment - we'll follow up here when we have more to say.  I agree this will be a long-running feature, and indeed may require a whole new engine that gets us away from a word-list model.

Comment 8

[Dwayne Bailey]

Kevin, is the lead on this aspect now I would say.  Can't add much more.

Comment 9

[Delphine Lebédel [:delphine - Use needinfo]]

Hey Kevin, any update as per your comment 7?

Comment 10

[Kevin Scannell]

We'll make a start on this at least during the upcoming work week.

Comment 11

[BMO Automation]

Firefox OS is not being worked on