969814 - enable FreeType support for color bitmap glyphs in 'sfnt' fonts

Status: RESOLVED FIXED

(Core :: Graphics: Text, defect)

Description

[Jonathan Kew [:jfkthame]]

FreeType has added support for color bitmaps embedded in an OpenType (sfnt) font container.

In Android 4.4.2, Google has gutted the old monochrome outline AndroidEmoji.ttf font, leaving only a handful of glyphs, and is instead shipping the new NotoColorEmoji.ttf, which uses their newly-proposed color bitmap format (and has no outline glyphs).

We currently build FreeType without support for PNG-format bitmaps (note the --without-png option at [1]), and therefore this font fails to render - and may cause other text to fail too, probably due to cairo getting into an error state.

E.g. see http://www.emojipedia.org/ (compare rendering on OS X or Win8.1 to see how it's supposed to look).

Another example - a random twitter status that includes an emoji character: https://twitter.com/TanAGardner/status/432033135623032833. Viewed in Nightly on Android 4.4.2, almost the entire text is lost. (Compare it on desktop.)

Unfortunately, -only- enabling PNG support in FreeType is not sufficient; the color emoji font still fails. We probably have other code in thebes and/or cairo that is assuming the presence of outline glyphs, and leading to errors on bitmap-only fonts.

An alternative to building in-tree FreeType without the --without-png option might be to use system freetype on Android, which (on new systems) presumably includes this support. But I don't know whether this option is possible while maintaining backward compatibility with the oldest Android versions we support.


[1] http://mxr.mozilla.org/mozilla-central/source/configure.in#9026

Comment 1

[Jonathan Kew [:jfkthame]]

(In reply to Jonathan Kew (:jfkthame) from comment #0)

> Unfortunately, -only- enabling PNG support in FreeType is not sufficient;
> the color emoji font still fails.

This may or may not be accurate: there were other problems with my build where I was trying to test this, so I don't know whether it would in fact work if I could make it build properly.

Comment 2

[James Willcox (:snorp) ([e-mail]) (he/him)]

I don't think we're going to want to try to use the system freetype. It's not exposed in the NDK, and there are no ABI guarantees whatsoever, etc, so I think we need to try to get this going in our own.

Comment 3

[Jonathan Kew [:jfkthame]]

So there are several issues here:

(1) The presence of the Noto Color Emoji font is causing large chunks of *other* text to disappear (see comment 0). Filed bug 973380 on that.

(2) We need to build FreeType with PNG support. I have a patch in progress for that; will attach here after some further testing.

Once that is done, I'm seeing at least three further problems; these should probably be spun off into followup bugs (that block bug 939280) once we have the PNG support enabled.

(3) The emoji get rendered as grayscale rather than color glyphs. At a minimum, we need to ensure the new FT_LOAD_COLOR flag is passed to FreeType, but I suspect other work may also be needed within moz2d or cairo or skia or something...

(4) The emoji font doesn't scale properly, and metrics don't match the rendered glyphs; it looks like we render the glyphs at one of a few fixed sizes, but scale the metrics according to the CSS font-size.

(5) When scrolling a page that uses emoji, we seem to often render the glyphs *vastly* oversized when they're outside the visible viewport, and then once the area becomes visible, we re-render at a more reasonable size.

Comment 4

[Jonathan Kew [:jfkthame]]

Attachment: build freetype with support for color bitmap (png) glyphs within opentype fonts.

Here's what I hacked together in order to get PNG support into our in-tree FreeType. The idea here is to point the FT build at our in-tree libpng; that's what the MOZ_PNG_CFLAGS setting in configure.in is for. (But is that a reasonable way to do it, or will it be fragile?) Then I also needed to tweak a few things in the libpng configuration, to adjust what libpng features get built and name-mangled/exported. This appears to build fine for both Android and B2G: https://tbpl.mozilla.org/?tree=Try&rev=aa20d3f7baa9.

Comment 5

[(not currently active) Ted Mielczarek]

Comment on attachment 8376978 [details] [diff] [review]
build freetype with support for color bitmap (png) glyphs within opentype fonts.

Review of attachment 8376978 [details] [diff] [review]:
-----------------------------------------------------------------

Sorry for the delay!

Comment 6

[Jonathan Kew [:jfkthame]]

Attachment: build freetype with support for color bitmap (png) glyphs within opentype fonts.

Rebased patch following the landing of bug 952505 (libpng 1.6.9); carrying forward r=ted. Jeff, given that this slightly tweaks the configuration of libpng that we build, flagging you for r? as well.

Comment 7

[Jonathan Kew [:jfkthame]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/6e8140ae4961

Comment 8

[Phil Ringnalda (:philor)]

Backed out in http://hg.mozilla.org/integration/mozilla-inbound/rev/dd9cf2bef693 for ASan bustage like https://tbpl.mozilla.org/php/getParsedLog.php?id=35580012&tree=Mozilla-Inbound&full=1#error0

Comment 9

[Jonathan Kew [:jfkthame]]

Re-landed this, as it was actually bug 974575 that triggered the ASan failure there.
https://hg.mozilla.org/integration/mozilla-inbound/rev/f56c157fa5ee

Comment 10

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/6e8140ae4961

Comment 11

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/f56c157fa5ee

Comment 12

[Glenn Randers-Pehrson]

(In reply to Jonathan Kew (:jfkthame) from comment #4)
> I also needed to
> tweak a few things in the libpng configuration, to adjust what libpng
> features get built and name-mangled/exported. This appears to build fine for
> both Android and B2G: https://tbpl.mozilla.org/?tree=Try&rev=aa20d3f7baa9.

That libpng tweaking increased the size of libpng (Unified_c_media_libpng0.o)
by about 20 percent, on all platforms, whether or not the build includes
Freetype.

Comment 13

[Jonathan Kew [:jfkthame]]

(In reply to Glenn Randers-Pehrson from comment #12)
> (In reply to Jonathan Kew (:jfkthame) from comment #4)
> > I also needed to
> > tweak a few things in the libpng configuration, to adjust what libpng
> > features get built and name-mangled/exported. This appears to build fine for
> > both Android and B2G: https://tbpl.mozilla.org/?tree=Try&rev=aa20d3f7baa9.
> 
> That libpng tweaking increased the size of libpng (Unified_c_media_libpng0.o)
> by about 20 percent, on all platforms, whether or not the build includes
> Freetype.

This happened because PNG_ERROR_TEXT_SUPPORTED and PNG_WARNINGS_SUPPORTED were made unconditional, rather than only enabled when PR_LOGGING is defined.

Fortunately, it looks like your libpng-update patch in bug 980488 handles this better, and restores the conditional around these features, so that ought to undo the size increase here.

Comment 14

[Glenn Randers-Pehrson]

Right, your patch enabled those and also the macro redefine of png_error.  Only the latter was really required, so I've fixed that in the libpng patch and was therefore able to put the others back inside a PR_LOGGING conditional.  Thanks for rechecking.