843592 - Indic fonts are not available in B2G

Status: RESOLVED FIXED

(Firefox OS Graveyard :: General, defect)

Description

[Aniruddha Adhikary [:tuxboy]]

User Agent: Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.70 Safari/537.17

Steps to reproduce:

Tried to display text written in UTF-8 Bangla (Indic language).


Actual results:

UTF-8 Indic text did not render properly.


Expected results:

UTF-8 text should have rendered properly.

Comment 1

[Axel Hecht [:Pike]]

The repo where we store our fonts is at https://github.com/mozilla-b2g/moztt, fyi.

Comment 2

[Aniruddha Adhikary [:tuxboy]]

Sent the first ever pull request to the moztt repo! (:D)

https://github.com/mozilla-b2g/moztt/pull/1

Comment 3

[Jonathan Kew [:jfkthame]]

Excellent, that's just what I was going to suggest we should do. :)

BTW, the Lohit fonts are distributed under the Open Font License. Do we need to go through some kind of license review/approval process in order to include these?

See also https://fedorahosted.org/lohit/.

Comment 4

[Aniruddha Adhikary [:tuxboy]]

No, it is Okay to continue according to Wikipedia.

Comment 5

[Michael Wu [:mwu]]

Patryk, what do you think?

Comment 6

[Jonathan Kew [:jfkthame]]

(In reply to Aniruddha Adhikary from comment #4)
> No, it is Okay to continue according to Wikipedia.

I don't think Wikipedia is the final authority on Mozilla licensing policy. :)

http://www.mozilla.org/MPL/license-policy.html

"If you are planning to import new Third Party Code into a Mozilla repository or a Mozilla product, always consult [e-mail] first."

Comment 7

[Aniruddha Adhikary [:tuxboy]]

Should I send a mail there? I think you guys should check it!

Comment 8

[Gervase Markham [:gerv]]

OFL is OK - we have some OFLed fonts already. We need to make sure this code gets covered by the scan which generates the licensing page, so a copy of its OFL gets in. (The OFL text is font-specific <sigh>).

Gerv

Comment 9

[Jonathan Kew [:jfkthame]]

Thanks, Gerv - yes, I didn't expect OFL to be a problem as such, but wasn't sure what mechanisms might be involved to make sure any needed documentation is handled properly.

Comment 10

[Aniruddha Adhikary [:tuxboy]]

I have included the license text in OFL.txt in my latest commit to the pull request. (Also included the COPYRIGHT file attached with the tar archive)

Comment 11

[Aniruddha Adhikary [:tuxboy]]

Attachment: Link to proposed patch

Comment 12

[Patryk Adamczyk [:patryk] UX]

(In reply to Michael Wu [:mwu] from comment #5)
> Patryk, what do you think?

Sorry for the late reply. But yes at this point since we're not release to these markets please us open fonts as substitutes.

Comment 13

[Aniruddha Adhikary [:tuxboy]]

Guys, according to TechChrunch (http://techcrunch.com/2013/02/24/firefox-os-smartphones-marketplace/) , B2G phones are coming to Bangladesh pretty soon via Telenor (known as GrameenPhone locally), shouldn't we add such support soon?

Comment 14

[Aniruddha Adhikary [:tuxboy]]

I am still alive! Since these are all open fonts, I think they can be used in B2G. Waiting for the bug to progress.

Comment 15

[Michael Wu [:mwu]]

FWIW we actually have Indic fonts enabled for a number of devices coming up, but it's Android's set of Indic fonts, which isn't the full set being proposed here. See https://android.googlesource.com/platform/external/lohit-fonts/+/ics-plus-aosp for that set.

Comment 16

[Axel Hecht [:Pike]]

I recall coming across a bug that asked to rip out the fonts we picked up from android, but I had no idea how to find that right now. Do you know if that's still a topic?

Also, does "number of devices coming up" tie to a version number of fx os, or is that some independent switch that partners may flip in one direction or the other?

Comment 17

[Michael Wu [:mwu]]

(In reply to Axel Hecht [:Pike] from comment #16)
> I recall coming across a bug that asked to rip out the fonts we picked up
> from android, but I had no idea how to find that right now. Do you know if
> that's still a topic?
> 

https://bugzilla.mozilla.org/show_bug.cgi?id=834244

Not all Android fonts were removed. The work for that is complete, though more fonts will be up for being removed once more things are supported in Feura Sans.

> Also, does "number of devices coming up" tie to a version number of fx os,
> or is that some independent switch that partners may flip in one direction
> or the other?

It related to whether our partners include the lohit font repo in their manifest. All 1.0 and 1.1 devices I know of will have them turned on.

Comment 18

[Enzam]

Any update on this? I can't see bangla on keon with latest update

Comment 19

[Soumya Deb [:Debloper]]

(In reply to Patryk Adamczyk [:patryk] UX from comment #12)
> ... But yes at this point since we're not release to
> these markets please us open fonts as substitutes.

I can vaguely understand the statement, and then find it wrong.

Any particular blocker for this patch to be committed?
Practical, theoritical, philosophical?

Comment 20

[Rudy Lu [:rudyl] (inactive after 2015/7/3)]

I cannot see most of the characters when using Unagi (m-c)
http://en.wikipedia.org/wiki/Hindi#Sample_text

It that is the case, Michael, can you help on this?

--
I was using the build from (this is for Mozilla internal only)
https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/nightly/mozilla-central-unagi-eng/

Comment 21

[arky [:arky]]

(In reply to Rudy Lu [:rudyl] from comment #20)
> I cannot see most of the characters when using Unagi (m-c)
> http://en.wikipedia.org/wiki/Hindi#Sample_text
> 
> It that is the case, Michael, can you help on this?
> 
> --
> I was using the build from (this is for Mozilla internal only)
> https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/nightly/mozilla-central-
> unagi-eng/


Download the noto hinted fonts and compress them: 
https://code.google.com/p/noto/downloads/detail?name=NotoFonts-hinted-2013-04-11.zip&can=2&q=

Connect your b2g device to the PC.

Mount the partition
adb shell mount -o remount,rw /system 
 
Copy the font files 
adb push ../hinted/ /system/fonts/ 

Reboot!

Comment 22

[arky [:arky]]

Sorry, Typo in my last comment. I mean to say un-compress/unzip the fonts

Comment 23

[arky [:arky]]

Hindi Sample text on Peak with noto hinted fonts
https://twitter.com/playingwithsid/status/377239544443047936/photo/1

Comment 24

[Michael Wu [:mwu]]

(In reply to Rudy Lu [:rudyl] from comment #20)
> I cannot see most of the characters when using Unagi (m-c)
> http://en.wikipedia.org/wiki/Hindi#Sample_text
> 
> It that is the case, Michael, can you help on this?
> 

Unagi doesn't count. You'll have to check a commercial build.

Comment 25

[Michael Wu [:mwu]]

Attachment: Add noto-fonts to base-jb.xml manifest

This enables noto fonts on our jb gonk builds. Commercial builds will likely use this regardless of whether we enable it in our builds or not, unless we explicitly decide to do something different and ask our partners to change the defaults.

Comment 26

[Michael Wu [:mwu]]

Here is the noto fonts repo for reference - https://android.googlesource.com/platform/external/noto-fonts/+/master

Comment 27

[Jonathan Kew [:jfkthame]]

(In reply to Michael Wu [:mwu] from comment #26)
> Here is the noto fonts repo for reference -
> https://android.googlesource.com/platform/external/noto-fonts/+/master

This Noto collection seems very incomplete; it only appears to include support for Bengali, Kannada, Malayalam and Telugu at this point, even though the Noto font list at https://code.google.com/p/noto/wiki/FontList also has Hindi and Tamil (as well as various non-Indic scripts...) What's up with this?

If it means many of the scripts in Noto aren't really ready for prime time, maybe we should be bundling the more mature Lohit collection instead. These would also cover other (very major) Indic scripts such as Gujarati, Gurmukhi (Punjabi) and Oriya.

Comment 28

[Michael Wu [:mwu]]

I think Android has always limited its Indic script support to about three - see their Lohit font repo: https://android.googlesource.com/platform/external/lohit-fonts/+/master .

This is the default set of Indic fonts that gets shipped - we'll have to work with our partners if we want to add or switch fonts.

Comment 29

[arky [:arky]]

@jfkthame Noto[1] is supposed to provide high quality fonts. It doesn't offer extensive Indic language coverage as Lohit family. I believe that Noto is supposed to super-cede Lohit in Android.

Suggest we use a noto font when available and otherwise fall back to Lohit.

@mwu Thanks for the patch :)

[1] http://code.google.com/p/noto/wiki/FontList

Comment 30

[Axel Hecht [:Pike]]

I just love to link-drop http://www.thehindubusinessline.com/industry-and-economy/info-tech/samsung-phones-to-support-9-indian-languages/article5018907.ece. There are actually extensive changes on top of Android to make India work, it seems.

Comment 31

[Jonathan Kew [:jfkthame]]

Comment on attachment 808956 [details] [diff] [review]
Add noto-fonts to base-jb.xml manifest

Review of attachment 808956 [details] [diff] [review]:
-----------------------------------------------------------------

Looking again at the external/noto-fonts directory referenced here, I really don't think we should be using it as the basis for what we include in b2g. Instead, we should pull fonts from the canonical repository on code.google.com (e.g. https://code.google.com/p/noto/source/browse/#svn%2Ftrunk%2Fpackages), and then choose which ones to include in the b2g platform. Maybe the best way to do this is to copy the latest font files into the moztt repo, if we can't directly reference the Google Code sources here.

Comment 32

[Michael Wu [:mwu]]

Yeah, if you want the latest noto fonts, we'll have to do it ourselves in moztt, and make sure people aren't using the noto repo.

Comment 33

[Axel Hecht [:Pike]]

MAK says to ask Jamil about the noto fonts.

Comment 34

[Jonathan Kew [:jfkthame]]

Attachment: Lohit fonts pull request

Axel tells me a number of Indian users have spoken in favor of using the Lohit fonts. I think we should go ahead and get these into the build, so that people can begin to try them out; we may also want to add Noto fonts, and make some test builds for comparison and review purposes, but at this point the Lohit set provides more complete coverage for the major Indian languages.

Comment 35

[Michael Wu [:mwu]]

This looks ok, but now we'll have to ask people to drop Android's Lohit and Noto font sets.

m1, are you ok with dropping lohit-fonts from ICS manifests and noto-fonts from JB manifests?

Comment 36

[Michael Wu [:mwu]]

Alternately, the product makefiles can be modified to not pull in those particular fonts, but dropping repos from the manifest seems simpler to me if it's a valid option.

Comment 37

[Michael Wu [:mwu]]

Comment on attachment 814015 [details] [review]
Lohit fonts pull request

This looks fine to me, but we'll need to coordinate this with m1 before landing.

Comment 38

[Michael Vines [:m1] [:evilmachines]]

Weird, I swear I commented on this bug days ago.   

> m1, are you ok with dropping lohit-fonts from ICS manifests and noto-fonts from JB manifests?

Yep.

Comment 39

[Axel Hecht [:Pike]]

Requesting koi as this blocks Bangla, which is on our list for 1.2.

Also, it looks like this patch is ready to land?

Comment 40

[Ryan VanderMeulen [:RyanVM]]

Master: https://github.com/mozilla-b2g/moztt/commit/3cb006b0299e456865d70312549eb5b3c885c509

Comment 41

[Preeti Raghunath(:Preeti)]

Just to confirm, this bug is not needed in 1.2. Please confirm.

Comment 42

[Axel Hecht [:Pike]]

(In reply to Axel Hecht [:Pike] from comment #39)
> Requesting koi as this blocks Bangla, which is on our list for 1.2.
> 
> Also, it looks like this patch is ready to land?

Why do you ask again, preeti?

Comment 43

[Ryan VanderMeulen [:RyanVM]]

John, can you assist with this uplift please? :)

Comment 44

[John Ford [:jhford] CET/CEST Berlin Time]

v1.2: 7b195cfef5c7eb2bfae401d5b8f892129be8cb9e