1403735 - libvoikko support for continued Finnish spell checking

Status: NEW

(Core :: Spelling checker, defect, P3)

Description

[Jan Henning [:JanH]]

So far implemented via an extension, but no longer possible (at least on non-Nightly/Dev Edition, or maybe not at all) after 57.

Some discussion as to potential solutions already happened on dev.platform (https://groups.google.com/forum/#!topic/mozilla.dev.platform/hu0QstgcQLs), although without a definitive result.

Comment 1

[Pasi Keinanen]

Finnish language is based on inflection the words. A static word file based dictionary spellcheck is not feasible for Finnish (accuracy of static checking, size/speed of parsing of the dict file, and the work required to generate such a file). The same applies to Saami language(s) and to some extent Icelandic. These languages require an access to a (third party) inflection engine that does the spellchecking, like Voikko for Finnish ( http://voikko.puimula.org ). 

Deprecating support for all third party proofing (incl. FOSS Voikko) is also an accessibility issue, affecting those with dyslexia and cognitive impairments.

Due to language size (Finnish + Saami + Icelandic, c. 6+ million people) this is probably a low priority issue for the developers.

Unfortunately it is a somewhat high priority issues for at least a big chuck of the the speakers of such languages.

Comment 2

[Jet Villegas (inactive)]

Henri: as the first poster for the thread noted in comment 1, I'm looking for some recommendations here. Thx!

Comment 3

[Henri Sivonen (:hsivonen)]

(In reply to Jan Henning [:JanH] from comment #0)
> Some discussion as to potential solutions already happened on dev.platform
> (https://groups.google.com/forum/#!topic/mozilla.dev.platform/hu0QstgcQLs),
> although without a definitive result.

There was also a thread of the Voikko development list: http://lists.puimula.org/pipermail/libvoikko/2017-March/000896.html

(In reply to Jet Villegas (:jet) from comment #2)
> Henri: as the first poster for the thread noted in comment 1, I'm looking
> for some recommendations here. Thx!

Ideally, we'd bake libvoikko into libxul like we do with Hunspell. The problems with this are:

1) There seems to be no one at Mozilla who could come up with a reasonable acceptable code size increment and commit Mozilla to accepting libvoikko if someone manages to slim libvoikko down to that size. 
2) Especially without a size target, there doesn't appear to be enthusiasm from the Voikko side to try to rip stuff out.

If we can't commit to baking libvoikko into libxul, the next question is whether Mozilla would be willing to use the mechanism created for OpenH264 to download a Mozilla-build copy of libvoikko and run it out of process to mitigate crashes. Again, the main problem is getting Mozilla to commit resources to such a thing.

So my primary recommendation is to figure out who at Mozilla can either commit to a libxul size increase or to putting resources into giving libvoikko the OpenH264 treatment.

If we can't figure out either commitment, then my recommendation is:
 1) Leave Windows 7 users without Finnish spell checking.
 2) Implement support for using system spell checking on Windows 8+. (Microsoft's Finnish spell checking is competent, but the Sami users would be out of luck unless someone creates a Voikko back end for the Windows spell checking framework.)
 3) Implement support for system spell checking on Mac. (And have users install Voikko as a system-wide spell checking service process-isolated by Apple; Apple's default Finnish spell checker is bad compared to Voikko.)
 4) Add the minimal code for dlopen()ing libvoikko into the Firefox chrome process on Linux and let distros that ship libvoikko enable this feature but don't enable it in Mozilla-official builds (since Mozilla wouldn't have the means to push libvoikko crash fixes).

Comment 4

[Henri Sivonen (:hsivonen)]

(I think we shouldn't go the wasm route, because enabling wasm-based spell checkers would probably be on par with giving libvoikko the OpenH264 treatment as far as Mozilla-side engineering commitments would go.)

Comment 5

[Pasi Keinanen]

So this is basically dead, right?

No more spell-checking for Finnish, Saami or any of the other inflected languages?

Both Chrome and Firefox have abandoned these language groups. This is sad feature regression.

Only Microsoft Explorer / Microsoft Edge work now for these languages.

Comment 6

[Henri Sivonen (:hsivonen)]

(In reply to Pasi Keinanen from comment #5)
> So this is basically dead, right?

No news after comment 3. It seems unrealistic to get an update before January. I've made a note for myself to ask about this again in January.

Comment 7

[Lasse Liehu]

Any news on this?

Since 2018, Microsoft Edge has switched to be based on Chromium and it continues to have spell checking for Finnish. Latest Chromium does not seem to support Finnish. It's sad that there seems to be no open source browser that supports Finnish spell checking.

Comment 8

[Jukka Laitinen [:fluks]]

I have made an experimental add-on for Finnish spell checking using libvoikko (https://addons.mozilla.org/en-US/firefox/addon/fin-spell/). You may want to try it, but it doesn't work on every page or input element, although you can open a spell checking side panel. And it probably can't work flawlessly until Firefox has an API for spell and grammar checking or some kind of general input API.

Comment 9

[Lasse Liehu]

(In reply to fluks from comment #8)

I have made an experimental add-on for Finnish spell checking using libvoikko (https://addons.mozilla.org/en-US/firefox/addon/fin-spell/). You may want to try it, but it doesn't work on every page or input element, although you can open a spell checking side panel. And it probably can't work flawlessly until Firefox has an API for spell and grammar checking or some kind of general input API.

That’s great news! Tried it out and it was usable when it didn't fail to function on some input elements. I remember how you mentioned this problem earlier also in bug 1343551 and how you mentioned there that also Grammarly has struggled with the same problem.

Comment 10

[Henri Sivonen (:hsivonen)]

The next step is probably seeing where bug 1653659 goes and, if it succeeds, assessing whether the same approach could be used with libvoikko such that the wasm bits are downloaded on-demand.

Comment 11

[Henri Sivonen (:hsivonen)]

I misunderstood what rlbox does, sorry. It does AOT compilation and doesn't do on-demand .wasm loading.