1080739 - Firefox doesn't render Devanagari web font glyphs properly on OSX

Status: RESOLVED FIXED

(Core :: Layout: Text and Fonts, defect)

Description

[Rod]

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.101 Safari/537.36

Steps to reproduce:

Load http://jsbin.com/miqoqu/quiet (uses Devanagari characters from Noto Sans) on Firefox 32.0.3 running on OS X 10.9.5.


Actual results:

Text rendered using system font. 


Expected results:

Text rendered using Noto Sans

Comment 1

[Rod]

Note that it seems to render just fine using Firefox on Windows and Ubuntu; only OS X fails.

Comment 2

[Rod]

Attachment: Expected rendering (captured on Firefox 32.0.3 on Ubuntu)

Comment 3

[Rod]

Sorry, url should be http://jsbin.com/jezakajopoga/quiet. Sadly I can't seem to edit description to correct.

Comment 4

[Rod]

Attachment: Wrong rendering (used system font) on OS X

Comment 5

[Jonathan Kew [:jfkthame]]

Try adding "%e0%a4%bf" to the end of the text string you're passing to the Google webfont API for subsetting purposes.

Comment 6

[Rod]

That does seem to help, ty. I confess I'm a little unclear on why this would be necessary only for Firefox Mac and not other environments though.

Comment 7

[Jonathan Kew [:jfkthame]]

Yes, that's far from obvious!

On OS X (and on Android, IIRC), Firefox will not use a font for text in a "complex-shaping" script (Arabic, Indic, etc) if the font lacks the necessary OpenType tables to support proper shaping. In your original testcase here, the subsetted font that Google Fonts serves appears to have GSUB/GPOS tables that are completely gutted -- no 'deva' or 'dev2' support at all. Apparently, including the short-I matra in the subset of characters prompts the subsetting process to include some OpenType layout support.

Comment 8

[Behdad Esfahbod]

Jonathan.  That's unfortunate.  Can you tell me what condition *exactly* you are using so I adjust the subsetter?

Comment 9

[Behdad Esfahbod]

And is it time to revisit those conditions?

Comment 10

[John Daggett (:jtd)]

(In reply to Behdad Esfahbod from comment #8)
> Jonathan.  That's unfortunate.  Can you tell me what condition *exactly* you
> are using so I adjust the subsetter?

http://mxr.mozilla.org/mozilla-central/source/gfx/thebes/gfxMacPlatformFontList.mm#192

Comment 11

[John Daggett (:jtd)]

(In reply to Behdad Esfahbod from comment #9)
> And is it time to revisit those conditions?

We need to fallback to fonts that support a given script which means including some level of shaping support. That check isn't particularly elegant but it does help us avoid fonts that have been poorly subsetted or has glyphs simply copied in from other fonts. Apple still ships a number of Chinese fonts that lack shaping info for Arabic for example.

Comment 12

[Behdad Esfahbod]

(In reply to John Daggett (:jtd) from comment #11)
> (In reply to Behdad Esfahbod from comment #9)
> > And is it time to revisit those conditions?
> 
> We need to fallback to fonts that support a given script which means
> including some level of shaping support. That check isn't particularly
> elegant but it does help us avoid fonts that have been poorly subsetted or
> has glyphs simply copied in from other fonts. Apple still ships a number of
> Chinese fonts that lack shaping info for Arabic for example.

I understand.  But perhaps reconsider it for webfonts?

Comment 13

[John Daggett (:jtd)]

(In reply to Behdad Esfahbod from comment #12)

> I understand.  But perhaps reconsider it for webfonts?

If anything there's a greater chance that a font is malformed with webfonts. Everyone seems to think font subsetting is easy peasy so there's a lot of font mangling that webdevs do. Just yesterday the webdev folks in my office were discussing how "simple" it would be to subset Adobe's Source Han Sans. Yikes!

Comment 14

[Behdad Esfahbod]

(In reply to John Daggett (:jtd) from comment #13)
> (In reply to Behdad Esfahbod from comment #12)
> 
> > I understand.  But perhaps reconsider it for webfonts?
> 
> If anything there's a greater chance that a font is malformed with webfonts.

But if it's broken, it won't work, and the designer will see it!  I think your logic is completely wrong.  Rejecting the font without any error log whatsoever is even worse.


> Everyone seems to think font subsetting is easy peasy so there's a lot of
> font mangling that webdevs do. Just yesterday the webdev folks in my office
> were discussing how "simple" it would be to subset Adobe's Source Han Sans.
> Yikes!

It *is* easy peasy using pyftsubset.  Many foundries are using that already.

Comment 15

[Jonathan Kew [:jfkthame]]

(In reply to Behdad Esfahbod from comment #8)
> Jonathan.  That's unfortunate.  Can you tell me what condition *exactly* you
> are using so I adjust the subsetter?

John pointed to the code in comment #10; basically, for "complex-shaping" script ranges, we check for the appropriate script tag in the GSUB table. If there's no GSUB, or if hb_ot_layout_table_choose_script gives us back 'DFLT', then we consider the font unusable for that script.

Presumably, if the subsetter ends up removing all the lookups for a given feature (because all the glyphs they involve have been removed), you remove that feature altogether; and if all features in a langSystem have been removed, you remove that langSystem from its script; and if all langSystems have been removed from a script, you remove the script, too. (Though apparently not the actual GSUB or GPOS table, even if it ends up empty.)

If you were to stop short of removing script and language records - just leave them with empty feature lists, but still present - then the font would still in effect be claiming to "support" the same script and language combinations as originally, and it should work as expected (for the subset of glyphs that remain).

Actually, perhaps even empty feature records (with no lookups) should be left, too, so that subsetting doesn't affect the font's claims regarding the features it implements. Otherwise, we could end up applying "synthetic" fallbacks for features that have been stripped, even though the fact that they're stripped is because none of the remaining glyphs actually need them. Consider a subsetted Arabic font with none of the actual shaping letters left in it: if you leave the (empty) 'init','medi','fina' features, then we won't be tempted to synthesize fallback shaping tables. Or a Latin font that originally supported 'smcp', but all the lowercase letters have been removed; leaving the empty 'smcp' feature will avoid the extra work of applying a fake-smallcaps codepath to text that doesn't actually require it.

Comment 16

[Paul Oiegas [:poiegas], PM for (Ecosystem QA, Services QA, WebCompat QA, Hubs QA teams)]

Hi guys, 

This still reproduces on latest Firefox release (43.0.3) and latest Nightly build (46.0a1-20151228030213). Do we still consider this an issue or it's an OS limitation (if I understood well from previous comments)?

Thanks,
Paul

Comment 17

[Jonathan Kew [:jfkthame]]

I think this should be addressed (along the lines of comment 15) in the font subsetter code. If it didn't strip entire OpenType script/language/feature records, then Firefox would still believe the font to be valid for rendering Devanagari characters.

Comment 18

[Paul Oiegas [:poiegas], PM for (Ecosystem QA, Services QA, WebCompat QA, Hubs QA teams)]

Considering Jonathan's comment I am marking this as NEW.

Comment 19

[John Daggett (:jtd)]

(In reply to Paul Oiegas [:pauloiegas] from comment #18)
> Considering Jonathan's comment I am marking this as NEW.

The font subsetter means the code in whatever font utility the author is using, not in Firefox. I think Firefox is code as it stands, we should reject fonts without the required shaping info for complex scripts.

I suggest this be resolved as WONTFIX.

Comment 20

[Behdad Esfahbod]

Fixed in FontTools subsetter to retain GSUB scripts:
https://bugzilla.mozilla.org/show_bug.cgi?id=1080739#c15

Comment 21

[Behdad Esfahbod]

Err, meant to paste this:
https://github.com/behdad/fonttools/commit/74e629e6004c71a9eeebbb94189b0b910dcac098