Closed
Bug 1011777
Opened 11 years ago
Closed 7 years ago
[B2G][Keyboard] Arabic: 'Auto correction' and 'Word suggestion' are not working with the Arabic keyboard
Categories
(Firefox OS Graveyard :: Gaia::Keyboard, defect, P1)
Tracking
(feature-b2g:3.0?, tracking-b2g:+, b2g-v1.3 affected, b2g-v1.4 affected, b2g-v2.0 affected, b2g-v2.2 affected, b2g-master affected)
RESOLVED
WONTFIX
People
(Reporter: demerick, Unassigned)
References
()
Details
(Whiteboard: LocRun1.4)
Attachments
(1 file)
418.76 KB,
application/zip
|
Details |
Description: With the Arabic keyboard selected neither 'Auto correction' or 'Word suggestion' are working. Pre-req: Use 'GAIA_KEYBOARD_LAYOUTS=ar make reset-gaia' to add the Arabic keyboard to the build. Repro Steps: 1) Update a Buri to BuildID: 20140516000201 2) From the homescreen tap 'Settings' then 'Keyboards' 3) Tap the 'gear' next to 'Built-in keyboard' and enable 'Auto correction' and 'Word suggestion' 4) Tap on the arrow in the top left to return to 'Keyboards' and tap 'Selected keyboards' 5) Tap 'Add more keyboards' and select 'Arabic' check box 6) Return to the home screen and launch the 'messaging' app 7) Tap the 'globe' icon to switch to the Arabic keyboard 8) Start typing in Arabic Actual: There is no 'Auto correction' or 'Word suggestion' for Arabic Expected: 'Auto correction' and 'Word suggestion' appear above the keyboard 1.4 Environmental Variables: Device: Buri 1.4 MOZ BuildID: 20140516000201 Gaia: 32fca83da31b9a0f9a5a88f96c913a25accdc14b Gecko: a1e455367fa6 Version: 30.0 Firmware Version: v1.2-device.cfg Repro frequency: 100% Link to failed test case: https://moztrap.mozilla.org/manage/case/8449/ See attached: logcat.txt Video URL: https://www.youtube.com/watch?v=04YlGBTdPFI
1.3 Environmental Variables: Device: Buri 1.3 MOZ BuildID: 20140514024003 Gaia: 96e3fa769a436a2182e6d54088fb41386eb2b5b5 Gecko: 685cf1d0dedb Version: 28.0 Firmware Version: v1.2-device.cfg Auto-Correction and Word Suggestion are not working for Arabic in the Buri 1.3 MOZ RIL.
Updated•10 years ago
|
Blocks: keyboard-rtl
Updated•10 years ago
|
Blocks: WordPrediction
Comment 3•10 years ago
|
||
Although RTL is not officially in scope for 2.2, partners are interested to ship 2.2 in Arabic. Would be nice to have this in, if possible. Flagging Kevin Scannell and Rami to see if they can help move this forward before next week. Thanks!
Flags: needinfo?(rami223)
Flags: needinfo?(kscanne)
Comment 4•10 years ago
|
||
Delphine, Currently I am in holidays and I will not be able to work on my flame until end of next week. Is that ok?
Flags: needinfo?(rami223)
Comment 5•10 years ago
|
||
That's fine Rami! Will keep the needinfo on you for this one so it stays on your radar :) thanks!
Flags: needinfo?(rami223)
Comment 6•10 years ago
|
||
Hi Rami, any update about this please? Do you have an ETA maybe? Maybe Kevin can help out on this as well if needed? thanks!
Comment 7•10 years ago
|
||
This is a partner request and is very urgent. Since I have not heard back from anyone, asking Kevin Scannell to help here if possible? Is there any ETA for this? Would be good to have it in the next week or so if possible thanks
Comment 9•10 years ago
|
||
It's unfortunate that this bug was not updated. RTL is very much in scope for 2.2. This needs to be fixed. Nominating to block. Arabic keyboard layout was refactored in bug #1039186 if that is helpful.
blocking-b2g: --- → 2.2?
Priority: -- → P1
Comment 11•10 years ago
|
||
Delphine, you should already know we can't provide Arabic spell check due to engine issue. This cannot possibly be in 2.2 scope as part of RTL. See https://wiki.mozilla.org/L10n:B2G/Fonts
Comment 12•10 years ago
|
||
How would I be supposed to know that? I don't work on fonts, and this issue was not discussed in this bug. I just get these bugs on file and make sure that they are being followed up on. How come no one ever mentioned this before here? This bug was filed in May 2014, and Arabic has been in scope for months now. Flagging TAMs/PMs so they know about this. thanks
Flags: needinfo?(wmathanaraj)
Flags: needinfo?(wehuang)
Flags: needinfo?(aappleton)
Comment 13•10 years ago
|
||
Thank you, Delphine. When we agreed that Arabic was back in scope in late January, this bug was also back in scope.
Updated•10 years ago
|
feature-b2g: 3.0? → 2.2?
Updated•10 years ago
|
Flags: needinfo?(aappleton)
Comment 14•10 years ago
|
||
Rami, Kevin: would you mind looking into this please? From past comments I think there might have been offline discussions going on between you about this. RTL is in scope for 2.2 and we're getting late in the cycle, so any progress would be super helpful. thanks a bunch!
Flags: needinfo?(kscanne)
Comment 15•10 years ago
|
||
(taking ni off Wilfred and Wesly since they are aware of this bug)
Flags: needinfo?(wmathanaraj)
Flags: needinfo?(wehuang)
Comment 16•10 years ago
|
||
Delphine, I have to do some research regarding the dictionary. I don't have that much experience in keyboards and how they are working. I will try to do some research today.
Flags: needinfo?(rami223)
Comment 17•10 years ago
|
||
(In reply to Delphine Lebédel [:delphine - use need info] from comment #12) > How would I be supposed to know that? I don't work on fonts, and this issue > was not discussed in this bug. I just get these bugs on file and make sure > that they are being followed up on. How come no one ever mentioned this > before here? This bug was filed in May 2014, and Arabic has been in scope > for months now. > Flagging TAMs/PMs so they know about this. thanks The agreement was, for late feature request (post feature planning phase), engineering would decide what can be determined to be in-scope. We have previously communicated (maybe not a l10n team-wide communication, apologies) we cannot possibly support a language with late request when our implemented engines does not work with it, even with the presence of the dictionary. Arabic falls into exactly that case. Bruce, could you make sure we have the correct shared understanding here? I really can't comment on every bug for every language about this.
Flags: needinfo?(bhuang)
Comment 18•10 years ago
|
||
While the bug was filed early, the scope of Arabic support (autocorrect or no) has moved in and out of releases since then. Let's move that discussion off this bug, and just focus here on what it will take to implement this regardless of which version it lands. The current engine status is at the link provided in comment 11: https://wiki.mozilla.org/L10n:B2G/Fonts. Let's make sure this is kept up to date.
Flags: needinfo?(bhuang)
Comment 19•10 years ago
|
||
Please communicate better - it's no fun sinking time into these dictionaries if they're not going to see the light of day. Anyway, I don't have much to say about the engine in this case, but I can create a frequency dictionary if this is ever back in scope, just hit me up with a needinfo.
Flags: needinfo?(kscanne)
Comment 20•10 years ago
|
||
(In reply to Tim Guan-tin Chien [:timdream] (slow response; please ni? to queue) from comment #11) > Delphine, you should already know we can't provide Arabic spell check due to > engine issue. This cannot possibly be in 2.2 scope as part of RTL. See > > https://wiki.mozilla.org/L10n:B2G/Fonts Time, could you please explain what are the engine issues you're talking about? Frankly (and ideally) I can't imagine a situation where Arabic breaks it. I think that's the first step before trying to implement anything yet; understanding what's the exact issue here. Thanks!
Comment 21•10 years ago
|
||
Since I'd already put some time into this, and Ahmed seems interested in testing this out with the existing engine, I'm attaching a draft dictionary for Arabic. Built from a not-very-big not-very-well-sampled corpus of web texts, about 10 million words before cleaning. Enjoy :)
Comment 22•10 years ago
|
||
Tim, could you please as well see comment 20? Thanks so much!
Flags: needinfo?(timdream)
Comment 23•10 years ago
|
||
The latin.js Engine was designed with a set of assumptions so it's not entirely language-agnostic. I do not know the detail of the design -- John did some research for the past few month so he should be able to comment. Other than try to understand what's under the hood, we could blindly apply the dictionary in comment 21 and see how it works, but we would have to ask you for that Ahmed since none of us understand Arabic.
Flags: needinfo?(timdream) → needinfo?(jlu)
Comment 24•10 years ago
|
||
Well.. I don't speak Arabic, so please excuse me if I make silly assumptions or ask silly questions. Directly dropping in the dictionary (|python xml2dict.py ar_wordlist.xml -o ar.dict|) and amending the missing attributes in the layout file (|autoCorrectLanguage: 'ar'| in ar.js) will probably result in things to seem to work, but we're likely to encounter sub-optimal prediction/correction results if the following issues aren't investigated/evaluated/addressed. And depending on how sub-optimal the results are, this naive approach may just can't have shipping quality. The point I'm making is that even if this naive approach seems to "work" (i.e. you get predictions when typing) but predictions results are like only 70% accurate, then I personally don't think I can call it working and shippable. The issues are: (A) Unicode combining characters. This is like you input one Arabic diacritic [1] after you input an ordinary Arabic character [2], to produce a single-entity-looking character. Even though it looks like one character, it is actually two characters to JS codes. And our prediction engine has no knowledge whatsoever regarding such combining characters, and can fail to predict words containing such characters, be they appearing in user input or in dictionary. So, does your dictionary contain such characters? Even if the answer is no, if the dictionary has words that the users expect they can enter either with or without the use of combining the characters, the "with" case will fail to generate ideal predictions. [3] (B) Variant/accented forms. In the prediction engine, we apply a few penalty multipliers whenever the user input does not exactly match a dictionary word. Say we're trying to loosely match a user input character in base form (like "C") against a dictionary character in accented form (like "Ç"). Such loose matching is less penalized than if we matched the "C" to something completely different and irrelevant, like "P". Does such idea as variant/accented forms exist in Arabic? If so, do we have, or can we generate an exhaustive list like [4], for better prediction accuracy? (C) Wrong assumption on punctuations in the original prediction engine logics. Any user input character not defined in [4] is regarded as a punctuation mark and is specially treated when we apply a heuristic to insert it when loosely matching inputs, in [5]. This logics or the [4]-table needs to be changed such that all those non-punctuation-mark Arabic characters are not handled by the logics, again for better accuracy. (D) General penalization/scoring heuristics: All those penalization multipliers [6] are currently based on their performance with Latin-based languages. For Arabic, maybe some penalizations don't make sense at all, or maybe some multiplier values need significant tuning, or maybe we need extra penalization techniques. Do you think of any? [1] like https://github.com/mozilla-b2g/gaia/blob/master/apps/keyboard/js/layouts/ar.js#L38 [2] like the third char in this row https://github.com/mozilla-b2g/gaia/blob/master/apps/keyboard/js/layouts/ar.js#L21 [3] Analogy in Latin characters: Inputting an A and then a U+030A ("ring above") will result in Å, but it's two characters and is different from a single character U+00C5, Å, for the prediction engine's logics. [4] https://github.com/mozilla-b2g/gaia/blob/master/apps/keyboard/js/imes/latin/predictions.js#L150-L177 [5] https://github.com/mozilla-b2g/gaia/blob/master/apps/keyboard/js/imes/latin/predictions.js#L736-L743 [6] https://wiki.mozilla.org/Gaia/System/Keyboard/IME/Latin/Prediction_%26_Auto_Correction#List_of_GrowOperations . Please read "GrowOperation" there as "penalization" for this comment's purposes. Again, in all, I'm not saying you can't or shouldn't do this --- as an engineer I'm more happy than everyone else to see that something originally envisioned for a specific task can later be applied to something more general without significant extra efforts, and I believe all the Keyboard folks feel the same way. However, keyboard text predictions are something that users demand very good accuracy or they just don't have real value, so we'd want to think about the issues and do some rounds of trials-and-errors (and keep in mind the maintainability and possibility for even further expansion in the future) before we're confident to say we're satisfied with the results.
Flags: needinfo?(jlu)
Updated•10 years ago
|
feature-b2g: 2.2? → 3.0?
Comment 25•9 years ago
|
||
[Blocking Requested - why for this release]: Arabic is in scope of 2.5, can we get this nominated please? thanks!
blocking-b2g: --- → 2.5?
Updated•9 years ago
|
Comment 28•9 years ago
|
||
Wilfred, Can you please see, if this blocks 2.5? Thanks
Flags: needinfo?(wmathanaraj)
Updated•9 years ago
|
blocking-b2g: 2.5? → ---
Comment 29•9 years ago
|
||
its not a must have for 2.5 - should plan ahead for next release.
Flags: needinfo?(wmathanaraj)
Comment 30•9 years ago
|
||
Unblocking 1181935 as this is an Arabic-specific defect/feature request, not RTL in general.
No longer blocks: 1181935
Comment 31•8 years ago
|
||
Guys Please can someone make it easy ? I dont know how to add the arabic keyboard ??
Comment 32•8 years ago
|
||
(In reply to m.qashllan from comment #31) > Guys Please can someone make it easy ? I dont know how to add the arabic > keyboard ?? Mozilla no longer develops B2G under the Firefox OS brand. Please refer to existing documents on how to add the feature and flash a community build.
Comment 35•8 years ago
|
||
(In reply to m.qashllan from comment #34) > Please guys All what i need is to add arabic keyboard < I love this phone What's the version of FxOS on your phone? You should have the keyboard already. It would be better if you could email to the mailing list at https://lists.mozilla.org/listinfo/dev-fxos ... there will be more response there.
Comment 36•8 years ago
|
||
Sir, Thank you for replying me. I have Lg Fx0 v2.1. It has Arabic language but there is no keyboard They told me to enable it and I dont know how. Even I dont know where I need to put these command. Pre-req: Use 'GAIA_KEYBOARD_LAYOUTS=ar make reset-gaia' to add the Arabic keyboard to the build. I need to know where to use this command and how to use it . :( Plz Help me
Comment 37•8 years ago
|
||
There are some instructions on how to do this here: https://public.etherpad-mozilla.org/p/l10n.gaia-multilocale Not sure how helpful this will come to be though. As :timdream mentions, posting on the dev-fxos mailing list might help too. thanks
Comment 38•7 years ago
|
||
Firefox OS is not being worked on
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → WONTFIX
You need to log in
before you can comment on or make changes to this bug.
Description
•