1240277 - Add hyphenation patterns for Indian languages

Status: RESOLVED FIXED

(Core :: Internationalization, enhancement)

Description

[Santhosh Thottingal]

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36

Steps to reproduce:

In Bug 672320,  while adding hyphenation patterns for many locales, there was an unresolved issue about Indian languages. The license was LGPL and not compatible to add.

Licensing should not be an issue now since I relicensed the packages to permissive license MIT.
Patterns are available at https://github.com/santhoshtr/hyphenation

Please add them to firefox.

Comment 1

[Richard Ishida]

Is it possible to move this forward? It seems that Santhosh has done much of the work.

See also https://w3c.github.io/iip/gap-analysis/taml-gap#issue79_hyphenation

Tamil is a language that really needs hyphenation support, because it has long words, and there are others that are similar, such as Malayalam (see https://r12a.github.io/scripts/malayalam/#linebreak).

Comment 2

[Zibi Braniecki [:zbraniecki][:gandalf]]

Jonathan, I think the question is for you :)

Comment 3

[Jonathan Kew [:jfkthame]]

Yes, I think we could try adding these patterns. I'll put up a patch.

It's unclear to me what degree of review & testing these have actually had among the relevant communities; they're extremely simple rule-based patterns -- apparently derived (without attribution) from a simple proof-of-concept that I originally posted to the XeTeX mailing list back in 2004 -- that may not fully handle cases that go beyond the simple "orthographic cluster" structure of these scripts, but they should be sufficient as a starting point.

Comment 4

[Jonathan Kew [:jfkthame]]

Attachment: Bug 1240277 - Add hyphenation patterns for Indic languages. r=#platform-i18n-reviewers

Using hyphenation patterns from https://github.com/santhoshtr/hyphenation.

The tests here are implemented as Mozilla reftests rather than added to WPT because I don't think
we can reasonably have such tests in WPT. The specific set of languages for which the UA supports
auto-hyphenation is not a normative requirement, and nor is the particular dictionary or algorithm
that will be used for any specific language. As such, the exact results are not defined by the
spec. (They may also change over time, if the hyphenation rules we use are updated, in which case
the tests will have to change accordingly.)

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2a669757a3cd
Add hyphenation patterns for Indic languages. r=platform-i18n-reviewers,dminor

Comment 6

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/2a669757a3cd

Comment 7

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/2a669757a3cd