1022322 - Change the default font for malayalam

Status: VERIFIED FIXED

(Firefox OS Graveyard :: General, defect)

Description

[Anivar Aravind]

Attachment: Lohit font with rendering issue.

User Agent: Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.152 Safari/537.36

Steps to reproduce:

Switch to malayalam locale or Open Any malayalam website which doesn't have @fontface webfont embedding enabled. eg: 


Actual results:

The default font for malayalam in FFOS is Lohit Malayalam, which is not a community preferred  font because of its unpleasant aesthetics and orthographic differences from popular usage. apart from this there are rendering issues with lohit font. see the attached image for this (Rendering issue is highlighted by underlining in red color)


Expected results:

The community prefers Meera, which is actively maintained and widely used.
It is the default font for most of the mainstream malayalam websites,  news portals   and Gnu/Linux distributions like fedora, debian, ubuntu etc.  It is licensed under GPL+Font Exception. And the latest version is available at http://download.savannah.gnu.org/releases/smc/fonts/malayalam-fonts-6.0/Meera/

Comment 1

[Axel Hecht [:Pike]]

Not sure what the licensing requirements are for https://github.com/mozilla-b2g/moztt.

Jonathan? (this time spelled right, sorry)

Comment 2

[Michael Wu [:mwu]]

(In reply to Axel Hecht [:Pike] from comment #1)
> Not sure what the licensing requirements are for
> https://github.com/mozilla-b2g/moztt.
> 

OFL is preferred, Apache 2 is ok, and for everything else we ask gerv.

Comment 3

[anish]

Attachment: Rendering issue with itta, Malayalam

Comment 4

[anish]

Added Screenshot for renderring issue for itta (Malayalam).

Comment 5

[Jonathan Kew [:jfkthame]]

Another possibility here would be to switch from Lohit to Noto, as we have already done for a couple of other Indian languages.

A question here - which may require broader input - is whether the default Malayalam font in FxOS should use the "traditional" or "reformed" Malayalam orthography.[1,2] I notice that the Meera font implements the traditional orthography, whereas both Lohit and Noto implement reformed.

From my (limited!) knowledge of the issues, I'm inclined to think we should probably stick with the reformed version as the default in FxOS. Several factors seem to point in this direction: (a) it's an official standard (even if not universally adopted); (b) the reduction in complex conjuncts and stacked forms, resulting in generally simpler glyph shapes, will tend to be more legible on a small screen; and (c) the default fonts on both Windows and Mac provide the reformed orthography, and authors assume this is what they'll get unless they explicitly choose a traditional font.

(For a demonstration of point (c), see [2], where the traditional examples are displayed as images, while the new (reformed) ones use plain text. The author clearly assumes the plain text will be displayed in reformed style, otherwise the comparison would become meaningless.)

So although there are doubtless some users who would prefer a traditional-orthography font, the evidence I see suggests that reformed is the more appropriate default, and we should switch to the Noto font (which fixes the rendering bugs mentioned in comments 1 and 3) rather than Meera.

[1] http://www.malayalamresourcecentre.org/Mrc/order.pdf
[2] http://en.wikipedia.org/wiki/Malayalam_alphabet#Orthography_reform

Comment 6

[Gervase Markham [:gerv]]

Is there still a question for me here?

Gerv

Comment 7

[Anivar Aravind]

Hi Jonathan Kew 

let me clarify the concerns 

(a) This is a popular Misconception . It is not an official standard for Computer or Mobile use . On pre-unicode age reformed orthography is prepared for reducing the no. of keys for typewritter Layouts. It was a intended for Printing . But even the print industry does not following this now . They increased the no of characters that can include in ASCII 256 Character limit in fonts first . The News Papers Like Kerala Kaumudi which switched to unicode last year adopted Traditional Orthography printing using the possibility to include more glyphs via Opentype tables. 

As of Now Almost all the news portals which publish in Malayalam uses same Meera font for their websites (as @font-face)  and provide same in Downloads . 
Most of the newsportals of Mainstream dailys uses Traditional Orthography Fonts (Mostly Meera) . To name a few check Newsportals of Dailys like  1) http://mathrubhumi.com/ 2) http://www.madhyamam.com/ 3) http://www.chandrikadaily.com/ 4) http://www.sirajlive.com/ 5) http://varthamanam.com 6) http://www.janmabhumidaily.comOr News channel portals like 1) http://www.indiavisiontv.com/ 2) http://www.asianetnews.tv/ 3) http://mediaonetv.in 4) http://www.reporterlive.com/ 

Apart from these most of the popular Webportals Uses Traditional Lipi fonts alone and meera became most popular font. 

If you need examples of Govts suggestion to use of Meera Font (Traditional Orthography ) Check Their Malayalam Computing promotion website http://malayalam.kerala.gov.in/index.php/Fonts (links an old version )
 and The download suggestion of recently launched District Administration automation project https://edistrict.kerala.gov.in/ (need to validate certificate ) Meera is the default font in all Linux Distros and in the Linux distro used by State IT@school Project . 

In short the users and Userbase prefers Meera in First Place 

b) I Know Complex stacked glyph is a problem for readability in mobile devices .   We have plans to develop a traditional lipi Equal Hight Font suitable for reading in mobile devices but that need resource mobilization first . So at this point meera is the best option available. Meera is actively maintained and Latest git version reduced sizes in a major way https://github.com/smc/fonts/tree/master/Meera

c) GNU/Linux is very popular in This part of the world and all the school students study in IT@school platform . I know mozilla may be missing your metrics due to that . Meera is the default font in all Linux Distros and in the Linux distro used by State IT@school Project . If it is unicode , People assumes ti get a traditional lipi font at this part of the world . Check font installation suggestions in Wikipedia or just google search on "How to install a Malayalam Unicode font" to get a feeling . apart from that Karthika in windows is a buggy font and contains many rendering mistakes. 


Apart from this Meera and Rachana are most actively maintained fonts and it is a community project . 
They also compatible to Opentype Malayalam v2 (MLM2) font tables and perfectly renders with Harfbuzz

Comment 8

[Anivar Aravind]

@gerv , I think pike want license compatibility clarification about including a font which is GPL+Font Exception

Comment 9

[Gervase Markham [:gerv]]

GPL + font exception is OK for Firefox OS.

Gerv

Comment 10

[Jonathan Kew [:jfkthame]]

(In reply to Anivar Aravind from comment #7)

> c) GNU/Linux is very popular in This part of the world and all the school
> students study in IT@school platform . I know mozilla may be missing your
> metrics due to that . Meera is the default font in all Linux Distros and in
> the Linux distro used by State IT@school Project . If it is unicode , People
> assumes ti get a traditional lipi font at this part of the world . Check
> font installation suggestions in Wikipedia or just google search on "How to
> install a Malayalam Unicode font" to get a feeling . apart from that
> Karthika in windows is a buggy font and contains many rendering mistakes. 

Thanks for your comments, that's helpful information. In particular, if many popular sites are using Meera via @font-face, I think this tilts the balance in favor of adopting it. I'm still a bit concerned about the potential readability: but perhaps the best thing to do is try making the switch and then let people see how it looks in context on the devices.


(I do still wonder about the relative use of Linux vs Windows in India. While the statistics may not be entirely accurate, the StatCounter figures show that Windows remains overwhelmingly more common:

  http://gs.statcounter.com/#desktop-os-IN-monthly-201305-201405

...which implies that an awful lot of users (and authors) will most likely be seeing a Reformed font by default on their desktops; similarly, on mobile devices I believe Android dominates the market and most commonly ships with a Reformed font. But on web sites that use @font-face, the OS default font is no longer the determining factor.)

Comment 11

[Hrishikesh K B]

(In reply to Jonathan Kew (:jfkthame) from comment #10)
> (I do still wonder about the relative use of Linux vs Windows in India.
> While the statistics may not be entirely accurate, the StatCounter figures
> show that Windows remains overwhelmingly more common:
> 
>   http://gs.statcounter.com/#desktop-os-IN-monthly-201305-201405
> 
> ...which implies that an awful lot of users (and authors) will most likely
> be seeing a Reformed font by default on their desktops; similarly, on mobile
> devices I believe Android dominates the market and most commonly ships with
> a Reformed font. But on web sites that use @font-face, the OS default font
> is no longer the determining factor.)

The default malayalam font shipped with windows is karthika, which is buggy. 
and as you can see in this page ( http://wiki.smc.org.in/Fonts ), most of the commonly 
used fonts are in traditional script.  
It is true that there are people who prefers reformed script, but AFIK the most uptodate, 
well maintained fonts are meera and rachana. and since meera has the better look in screens we prefer this. 
You might be knowing that noto is maintained by Google and meera is maintained by the community. 
and we prefer a community maintained font.

Comment 12

[Jonathan Kew [:jfkthame]]

(In reply to Hrishikesh K B from comment #11)
> The default malayalam font shipped with windows is karthika, which is buggy.

Are there details about this available somewhere? Just saying "it is buggy" is not very informative.

Comment 13

[Hrishikesh K B]

(In reply to Jonathan Kew (:jfkthame) from comment #12)
> Are there details about this available somewhere? Just saying "it is buggy"
> is not very informative.

Hope you know that Kathika is a proprietary font. tracking the bugs in a proprietary font is not at all my priority. 
AFAIK there are rendering issues with characters like 'Nta'. and there are many other issues. Even the help page of malayalam wikipedia is advising to get rid of karthika[1]   

[1] - https://ml.wikipedia.org/wiki/%E0%B4%B8%E0%B4%B9%E0%B4%BE%E0%B4%AF%E0%B4%82:To_Read_in_Malayalam#Managing_Chillu_Characters

Comment 14

[Jonathan Kew [:jfkthame]]

(In reply to Hrishikesh K B from comment #13)
> (In reply to Jonathan Kew (:jfkthame) from comment #12)
> > Are there details about this available somewhere? Just saying "it is buggy"
> > is not very informative.
> 
> Hope you know that Kathika is a proprietary font. tracking the bugs in a
> proprietary font is not at all my priority. 

I understand that. But if you're going to advise people to use other fonts because the default font included with Windows is buggy, you need to clearly explain the nature of those bugs. What specific Unicode sequences does it render incorrectly? Without such details, how can we evaluate the relevance and quality of that advice?

> Even the help page of malayalam wikipedia is advising to
> get rid of karthika[1]   
> 
> [1] -
> https://ml.wikipedia.org/wiki/%E0%B4%B8%E0%B4%B9%E0%B4%BE%E0%B4%AF%E0%B4%82:
> To_Read_in_Malayalam#Managing_Chillu_Characters

That advice appears to be obsolete (although it may still be relevant for users running old versions of Windows); the current version of Kartika does support the Malayalam chillu characters.

Comment 15

[Jonathan Kew [:jfkthame]]

Attachment: link to github PR to replace the Malayalam font

Based on comments above, I suggest we try this and see how people like it in actual builds.

Comment 16

[Anivar Aravind]

I hope we can discuss Karthika Bugs in a Microsoft bugzilla (If microsoft Provides one and they license it under a FOSS license) . The reason for people moving away from karthika is anyway not mu concern. For me it is an aesthetically bad, linguistically incorrect (cant handle ന്റ  and ന്‍റ correctly when I am last checked) proprietary font.  

Here the point is the need of changing Default font for Malayalam . As per we are informed in Mozilla L10n Sprint Pune, the choice of best font selected by L10n team on a compatible free license  will be included. Also mozilla staff present there made it clear that they cant assure OEM inclusion of same font and it is upto device vendor . 

As of now we have a bug free font , widely popular in Language community and even used and promoted by Govt . 

So please include meera and close this bug. 

If we need to discuss Microsoft karthika bugs, Email /lists are the best medium for that .


Just one word on statistics. 
Malayalam is a language in state of Kerala and it is contains less than 3% of population in India (33.3 million). So your counters for country level status may not be accurate to address that now. 
On Android , The redering of Indian languages became perfect only after Harfbuzz-ng integration in Android 4.3. Before Noto Fonts arrival the font included in Android phones was Mostly vendor specific (samsung had their samsungmalayalam.ttf which is still have many encoding bugs) or AnjaliNewLipi.ttf  which is maintained by a google employee which contained many encoding bugs in MLM2 tables . Noto arrived after that. In the cathedral model of Android development , there is no scope for language speakers and communities to demand which font they want . But we think Mozilla is different and respect community decisions in this regard.

Comment 17

[Anivar Aravind]

Wrote above comment before seeing the commit . Thanks jfkthame

Comment 18

[Michael Wu [:mwu]]

Comment on attachment 8438541 [details] [review]
link to github PR to replace the Malayalam font

Interesting that it versions itself 06 internally, but the ftp site uses 6.0. Oh well - at least we're consistent with the font itself.

Comment 19

[Ryan VanderMeulen [:RyanVM]]

Master: https://github.com/mozilla-b2g/moztt/commit/08b7f6a625f23e48a9cb0c8f63ee131009754f7c

Comment 20

[Lancy(Leave from Mozilla)]

Attachment: Verify_1022322.png

Hi Anivar,

Acorrding to Comment 3, I have got the screenshot in Stopwatch(See the attachment:Verify_1022322.png), but I don't know Malayalam, could you help me confirm that problem has been fixed?

Comment 21

[Hrishikesh K B]

(In reply to Lancy from comment #20)

> Acorrding to Comment 3, I have got the screenshot in Stopwatch(See the
> attachment:Verify_1022322.png), but I don't know Malayalam, could you help
> me confirm that problem has been fixed?

Hello Lancy, 

Yes the rendering in the attached image is correct. The issue can be marked as solved :)

Comment 22

[Anivar Aravind]

Screenshots shows meera and bug is resolved

BTW meera Upstream is moved to  
https://gitlab.com/smc/meera please take a note of this for future updates .

Comment 23

[Delphine Lebédel [:delphine - OOO]]

Since fix was verified, marking bug as verified fixed