534284 - fonts with "symbol"-encoded names fail to load via @font-face on Windows

Status: RESOLVED DUPLICATE of bug 1228799

(Core :: Layout: Text and Fonts, defect)

Description

[Jonathan Kew [:jfkthame]]

When the Windows-platform names in a TrueType/OpenType font have encoding ID 0 (symbol) rather than 1 (Unicode), our font-loading code rejects the font as invalid (in gfxFontUtils::MakeEOTHeader) and the downloaded font is discarded.

This is a problem for some "hacked-encoding" Indic fonts, e.g. eenadu.ttf used on http://eenadu.net/. The font works in Firefox if installed locally, and it works via @font-face on non-Windows platforms, but fails on Windows because our code only accepts names tagged as Unicode.

Comment 1

[Jonathan Kew [:jfkthame]]

It looked to me like there should be two possible approaches to this:

(1) modify gfxFontUtils::MakeEOTHeader to accept names with platform 3, encoding 0 in addition to those with platform 3, encoding 1.

(2) modify FontEntry::LoadFont so that when gfxFontUtils::MakeEOTHeader fails (because of the name encoding), we fall back to the alternate code path using AddFontMemResourceEx rather than just returning null.

However, I've tried both of these things, and font activation (whether through TTLoadEmbeddedFont in case 1 or AddFontMemResourceEx in case 2) still fails.

Microsoft's Font Validator reports a bunch of errors and warnings for the Eenadu font, suggesting that it was created with a pretty old and flaky tool of some kind. However, given that it does work if installed directly in the Windows Fonts folder, the problems don't appear to be fatal, but I have not yet figured out why the activation APIs are still failing.

Comment 2

[Jonathan Kew [:jfkthame]]

Attachment: work around problems loading eenadu.ttf (symbol-encoded) font

So there are two basic problems to overcome here. First, our code rejects the font outright because it doesn't find Windows/Unicode names - the names are tagged as Windows/Symbol. Second, even once our code is updated to accept the names, I get errors from the Windows font activation APIs we're using. This seems to also be related to the encoding tags, but in this case it's the cmap subtable encoding that causes the problem. Changing the encoding value in the downloaded font data allows the font to load correctly.

This is an ugly hack, I admit, but it does seem to resolve the problems, and should not affect "normal" (proper Unicode) @font-face usage.

Tryserver build at http://build.mozilla.org/tryserver-builds/[e-mail]-try-3277a7d48b18/

Comment 3

[Arun Ranganathan]

I verified that the tryserver build referenced in Comment 2 works on Windows XP Professional Version 2002 Service Pack 3.

Comment 4

[Arun Ranganathan]

A test case that you can use is my own version of the eenadu site:

http://arunranga.com/examples/eenadu/EenaduHeadlines.html

Of course, it won't work in IE8+ because I suspect that the EOT font may have root strings, but the Tryserver build does work (as per Comment 3).

Comment 5

[Jonathan Kew [:jfkthame]]

(In reply to comment #4)
> A test case that you can use is my own version of the eenadu site:
> 
> http://arunranga.com/examples/eenadu/EenaduHeadlines.html
> 
> Of course, it won't work in IE8+ because I suspect that the EOT font may have
> root strings, but the Tryserver build does work (as per Comment 3).

Just FYI, I looked at the EOT font and it does indeed have rootstrings for the eenadu.net domain, so that explains the IE8 failure on your version of the site.

Comment 6

[John Daggett (:jtd)]

Comment on attachment 417357 [details] [diff] [review]
work around problems loading eenadu.ttf (symbol-encoded) font

+    // Hack for eenadu.ttf (and other non-standard fonts using SYMBOL-tagged cmap):
+    // modify the downloaded font data to tag the cmap table as UNICODE instead,
+    // otherwise activation fails.
+    // This will invalidate the cmap table and overall font checksums in such fonts;
+    // however, these are not checked by the Windows APIs, so no real harm is done.
+    // (In fact, the eenadu.ttf font has incorrect checksums anyway.)
+    if (foundCmap && cmapLen >= sizeof(CmapHeader)) {
+        const CmapHeader *cmap = reinterpret_cast<const CmapHeader*>(aFontData + cmapOffset);
+        const CmapEncodingRecord *encRec =
+            reinterpret_cast<const CmapEncodingRecord*>(cmap + 1);
+        PRInt32 symbolSubtable = -1;
+        for (PRUint32 i = 0; i < (PRUint16)cmap->numTables; ++i) {
+            if ((i + 1) * sizeof(CmapEncodingRecord) + sizeof(CmapHeader) > cmapLen)
+                break; // looks like an error, table is too small
+            if (PRUint16(encRec[i].platformID) != PLATFORM_ID_MICROSOFT)
+                continue; // only interested in MS platform
+            if (PRUint16(encRec[i].encodingID) == ENCODING_ID_MICROSOFT_UNICODEBMP) {
+                // we've got a Microsoft/Unicode table, so don't interfere
+                symbolSubtable = -1;
+                break;
+            }
+            if (PRUint16(encRec[i].encodingID) == ENCODING_ID_MICROSOFT_SYMBOL) {
+                symbolSubtable = i;
+            }
+        }
+        if (symbolSubtable != -1) {
+            // We found a windows/symbol cmap table, and no windows/unicode one;
+            // change the encoding ID so that AddFontMemResourceEx will accept it
+            const_cast<CmapEncodingRecord*>(encRec)[symbolSubtable].encodingID =
+                ENCODING_ID_MICROSOFT_UNICODEBMP;
+        }
+    }

I don't think this is a good idea at all, this code is making the assumption that a cmap SYMBOL encoding really implies a UNICODE BMP encoding when that's basically just a good guess.  We could ignore the platform encoding but that doesn't seem like such a good idea either.

This font was clearly built with a tool that never explicitly sets the platform encoding field for either the cmap or name tables:

<cmap>
  <tableVersion version="0"/>
  <cmap_format_4 platformID="0" platEncID="0" language="0">
  <cmap_format_0 platformID="1" platEncID="0" language="0">
  <cmap_format_4 platformID="3" platEncID="0" language="0">
</cmap>
  
<name>
  <namerecord nameID="0" platformID="0" platEncID="0" langID="0x0">Any other use of this font other than for reading or printing Eenadu Web edition is a copy right violation.</namerecord>
  <namerecord nameID="1" platformID="0" platEncID="0" langID="0x0">Eenadu</namerecord>
  <namerecord nameID="2" platformID="0" platEncID="0" langID="0x0">Regular</namerecord>
  <namerecord nameID="3" platformID="0" platEncID="0" langID="0x0">Eenadu Web Font</namerecord>
  <namerecord nameID="4" platformID="0" platEncID="0" langID="0x0">Eenadu</namerecord>
  <namerecord nameID="5" platformID="0" platEncID="0" langID="0x0">10/10/98</namerecord>
  <namerecord nameID="6" platformID="0" platEncID="0" langID="0x0">Eenadu</namerecord>
  <namerecord nameID="0" platformID="1" platEncID="0" langID="0x0">Any other use of this font other than for reading or printing Eenadu Web edition is a copy right violation.</namerecord>
  <namerecord nameID="1" platformID="1" platEncID="0" langID="0x0">Eenadu</namerecord>
  <namerecord nameID="2" platformID="1" platEncID="0" langID="0x0">Regular</namerecord>
  <namerecord nameID="3" platformID="1" platEncID="0" langID="0x0">Eenadu Web Font</namerecord>
  <namerecord nameID="4" platformID="1" platEncID="0" langID="0x0">Eenadu</namerecord>
  <namerecord nameID="5" platformID="1" platEncID="0" langID="0x0">10/10/98</namerecord>
  <namerecord nameID="6" platformID="1" platEncID="0" langID="0x0">Eenadu</namerecord>
  <namerecord nameID="0" platformID="3" platEncID="0" langID="0x409">Any other use of this font other than for reading or printing Eenadu Web edition is a copy right violation.</namerecord>
  <namerecord nameID="1" platformID="3" platEncID="0" langID="0x409">Eenadu</namerecord>
  <namerecord nameID="2" platformID="3" platEncID="0" langID="0x409">Regular</namerecord>
  <namerecord nameID="3" platformID="3" platEncID="0" langID="0x409">Eenadu Web Font</namerecord>
  <namerecord nameID="4" platformID="3" platEncID="0" langID="0x409">Eenadu</namerecord>
  <namerecord nameID="5" platformID="3" platEncID="0" langID="0x409">10/10/98</namerecord>
  <namerecord nameID="6" platformID="3" platEncID="0" langID="0x409">Eenadu</namerecord>
</name>

Unless this is logic that always holds true for a large number of fonts and is absolutely necessary, I think we should avoid burdening our code with what is basically fixup code for bad fonts.

Comment 7

[Jonathan Kew [:jfkthame]]

Outside of the Indic-hacks world, there probably isn't widespread use of symbol-encoded fonts on the web, as Unicode has pretty well supplanted the old way of doing these things. It used to be that symbol-encoded fonts such as SIL Galatia (Greek), SIL Ezra (Hebrew), and the SIL IPA93 collection (phonetics) were de facto standards for their respective types of material, but I suspect they are rarely used any more.

As an experiment, however, I created a test page using one of these fonts via @font-face, to represent what would happen if someone tries to put a legacy document of this kind online:
   http://people.mozilla.org/~jkew/silipa93/index.html
This page renders as intended in Safari, and in Firefox 3.5 on the Mac, but fails on FF 3.5/Win. However, with this patch, it works on Minefield/Win as well.

(This also serves to demonstrate that the cmap "hack" for the Eenadu font does not break the use of other symbol-encoded fonts.)

Comment 8

[John Daggett (:jtd)]

Attachment: fixed version of font

Arun Ranganathan wrote:

> > Are there a large number of fonts you're seeing with this problem
> > (i.e. fonts constructed using tools that don't set the platform
> > encoding fields in the name or cmap tables)?
> 
> Sadly, yes. 
> 
> It would seem that:
> 
> 1. We support this genre of TTF font  -- notably, when it is downloaded 
> as a system font, it works.  And, as a font to accompany @font-face, it 
> works on Mac OS X, but NOT on Windows.  So the behavior is inconsistent.
> 2. The fonts in question are "bad" TTF fonts.
> 
> Eenadu has a huge circulation.  But there are other large circulation 
> culpable sites as well, that also have "bad" TTF fonts as in 2. above.  
> The goal was to allow sites like eenadu to coast off of their TTF 
> support but point out the benefits of Unicode.  Until this bug is fixed, 
> we don't have a good story for these sites.

Just to be clear here, the font here is being refused by a Windows font loading API, that's why your testpage doesn't render in other Windows browsers either (Opera 10, Chrome 4).  This means it probably won't load if embedded in a Word document either.  The inconsistency is in the way Windows font code handles dynamically loaded fonts versus platform fonts.  But to be fair, there will always be some level of inconsistency because a downloaded or embedded font requires stricter validation, it's not the same as a font that came installed or was explicitly installed by a user.

The solution is simply to fix the font, then the font will work across browser implementations and platforms, without the need for every user agent to add special fixup code like this.

To fix this font:

Fix the name table records:

Original:
<namerecord nameID="1" platformID="3" platEncID="0" langID="0x409">

Fixed:
<namerecord nameID="1" platformID="3" platEncID="1" langID="0x409">

Fix cmap headers:

Original:
    <cmap_format_4 platformID="3" platEncID="0" language="0">

Fixed:
    <cmap_format_4 platformID="3" platEncID="1" language="0">

I used ttx/fonttools to fix the font.

http://sourceforge.net/projects/fonttools/

Attached is a version of the font that works with FF3.5/Windows and Google Chrome (Opera still doesn't render with font, not sure why that is).

Comment 9

[Robert O'Callahan (:roc) (email my personal email if necessary)]

I think we should try getting sites to use fixed fonts, starting with this one.

If that doesn't work, I think there is a reasonable case for taking a patch like this, given that it makes our Windows behaviour consistent with our Mac behaviour. (Someone should check GTK.) Such consistency is actually quite important.

Comment 10

[Masatoshi Kimura [:emk]]

Fixed by bug 1228799.