1124459 - (Yoruba-WordPrediction) Add Yoruba (yo) wordlist/dictionary

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::Keyboard, defect)

Description

[Delphine Lebédel [:delphine - Use needinfo]]

Yoruba word prediction needed from 2.0 and onwards

Comment 1

[Delphine Lebédel [:delphine - Use needinfo]]

Adding Kevin and localizers who might be able to help with word prediction input. thanks!

Comment 2

[Delphine Lebédel [:delphine - Use needinfo]]

As no response from localizer, asking Devon and Ian from Rubric to pitch in here.
thanks!

Comment 3

[ian.henderson]

(In reply to Delphine Lebédel [:delphine - use need info] from comment #2)
> As no response from localizer, asking Devon and Ian from Rubric to pitch in
> here.
> thanks!

Once we have the FFOS translated, we can supply the translated po files to Kevin as per Bug 1121734.

Comment 4

[Delphine Lebédel [:delphine - Use needinfo]]

Comment offline from Dwayne: "The other issue is languages that use tone marking (Youruba) i.e. lots of diacritics or where the wordlist is very dirty i.e. usually a bad input where people use ASCII equivalents for missing letter .  I think it would be good to query Kevin and see if anything is needed in code.  Kevin's home grown scripts might be enough.  But I think it would be helpful if we're thinking on how to more easily involve the community in a process of iteratively improving these wordlists."
Flagging Kevin so he can speak to this, and what we might need to think on this before anything else?

Comment 5

[Kevin Scannell]

I have code for statistical diacritic restoration (http://sourceforge.net/projects/lingala/files/charlifter/) but Yoruba is exceptionally challenging (a) because it uses so many tone marks and (b) there is virtually no training data and no free word list with all of the correct marks.  So that approach is probably a dead end in this case (as would be any approach that relies on web crawling).  My best suggestion would be to partner with a language expert like Tunde Adegbola who's already worked on stuff like this to see if a good word list can be assembled manually.

Comment 6

[ian.henderson]

Hello Kevin,

Here is some corpus data for you to look at:

http://mozilla.locamotion.org/yo/mozilla_lang/main.lang.po	
http://mozilla.locamotion.org/yo/mozilla_lang/mozorg/home/index.lang.po		
http://mozilla.locamotion.org/yo/mozilla_lang/firefox/os/index.lang.po		
http://mozilla.locamotion.org/yo/mozilla_lang/firefox/os/devices.lang.po		
http://mozilla.locamotion.org/yo/mozilla_lang/firefox/os/faq.lang.po		
http://mozilla.locamotion.org/yo/mozilla_lang/firefox/partners/index.lang.po		
http://mozilla.locamotion.org/yo/mozilla_lang/firefoxos/firefoxos.lang.po		
http://mozilla.locamotion.org/yo/mozilla_lang/legal/index.lang.po		
http://mozilla.locamotion.org/yo/mozilla_lang/tabzilla/tabzilla.lang.po		

https://localize.mozilla.org/yo/masterfirefoxos/	
https://localize.mozilla.org/yo/sumo/

Comment 7

[ian.henderson]

Hello Kevin,

Have you looked at: http://yo.wikipedia.org/wiki/Oj%C3%BAew%C3%A9_%C3%80k%E1%BB%8D%CC%81k%E1%BB%8D%CC%81

You can use Wikipedia as a corpus source for many languages.