1454152 - Noto Color Emoji doesn't show in Firefox for Linux if fontconfig antialias enabled

Status: VERIFIED FIXED

(Core :: Graphics: Text, defect, P5)

Description

[Guo Yunhe]

Attachment: Screenshot_20180414_111713.png

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0
Build ID: 20180323154952

Steps to reproduce:

1. Install Noto Color Emoji fonts (Linux).
2. Open Firefox.
3. Visit http://www.fileformat.info/info/emoji/browsertest.htm
4. Check if any colored emoji shows


Actual results:

No colored emoji shows


Expected results:

Colored emoji shows in most places (Attachment is a screenshot comparison with Chromium, in the same system)

Comment 1

[Andre Klapper]

> 1. Install Noto Color Emoji fonts (Linux).

How? From where? This needs clearer steps to reproduce.

Using Firefox 59 and having google-noto-emoji-color-fonts-20180307-1.fc27.noarch installed, I get color emojis displayed like in your Chromium screenshot. 
I cannot reproduce the problem.

Comment 2

[Andre Klapper]

Which Linux distribution / operating system and version is this about?
Is this an upstream Firefox version that you downloaded? Or is this a Firefox version provided by your distribution?

Comment 3

[Guo Yunhe]

Opensuse tumbleweed 20180408

I tested both Firefox from distribution and Firefox nightly from Mozilla website.

Comment 4

[Guo Yunhe]

I am using the font package from distribution.

Comment 5

[Guo Yunhe]

New steps to reproduce:

1. Open Firefox and visit https://guoyunhe.me/demo/noto-color-emoji/index.html
2. Open Chromium and visit https://guoyunhe.me/demo/noto-color-emoji/index.html
3. Compare rendered content.

The web page contains some emoji faces and load latest Noto Color Emoji from the server with CSS @fontface feature.

In openSUSE Tumbleweed, Firefox has nothing to render.

In Ubuntu 17.10, Firefox uses its built in Mozilla EmojiOne font.

Chromium always renders the fonts in correct way.

Comment 6

[Guo Yunhe]

Attachment: Remote font comparison in openSUSE Tumbleweed

Comment 7

[Guo Yunhe]

Attachment: Remote font comparison in Ubuntu 17.10

Comment 8

[Guo Yunhe]

Attachment: Remote font comparison in openSUSE Tumbleweed, Firefox Nightly

Comment 9

[Guo Yunhe]

Attachment: Remote font comparison in Ubuntu 17.10, Firefox Nightly

Comment 10

[Daniel_A[:daschilean] [Please contact - [e-mail]]]

I verified this issue on Ubuntu X86 with Firefox 61.0a1(2018-04-19) and I cannot reproduce it.

If you still have the issue please create a new profile, you have the steps here:https://support.mozilla.org/en-US/kb/profile-manager-create-and-remove-firefox-profiles?redirectlocale=en-US&redirectslug=Managing-profiles#w_starting-the-profile-manager

Please test if the issue is reproducible in safe mode, here is a link that can help you:
https://support.mozilla.org/t5/Procedures-to-diagnose-and-fix/Troubleshoot-Firefox-issues-using-Safe-Mode/ta-p/1687#w_how-to-start-firefox-in-safe-mode

Comment 11

[Guo Yunhe]

(In reply to Daniel_A from comment #10)
> I verified this issue on Ubuntu X86 with Firefox 61.0a1(2018-04-19) and I
> cannot reproduce it.
> 
> If you still have the issue please create a new profile, you have the steps
> here:https://support.mozilla.org/en-US/kb/profile-manager-create-and-remove-
> firefox-profiles?redirectlocale=en-US&redirectslug=Managing-
> profiles#w_starting-the-profile-manager
> 
> Please test if the issue is reproducible in safe mode, here is a link that
> can help you:
> https://support.mozilla.org/t5/Procedures-to-diagnose-and-fix/Troubleshoot-
> Firefox-issues-using-Safe-Mode/ta-p/1687#w_how-to-start-firefox-in-safe-mode

Can you provide a screenshot of https://guoyunhe.me/demo/noto-color-emoji/index.html ?

Comment 12

[Daniel_A[:daschilean] [Please contact - [e-mail]]]

Attachment: New Remote font comparison in Ubuntu .png

I verified this issue on Ubuntu X86 with Firefox Nightly 61.0a1(2018-04-22).
Please check my results of the website: https://guoyunhe.me/demo/noto-color-emoji/index.html for both Firefox and Chrome browsers. I cannot reproduce the issues displayed on the following attachments: Screenshot_20180414_111713.png, Remote font comparison in openSUSE Tumbleweed,  Remote font comparison in openSUSE Tumbleweed.

Comment 13

[Guo Yunhe]

(In reply to Daniel_A from comment #12)
> Created attachment 8970144 [details]
> New Remote font comparison in Ubuntu .png
> 
> I verified this issue on Ubuntu X86 with Firefox Nightly 61.0a1(2018-04-22).
> Please check my results of the website:
> https://guoyunhe.me/demo/noto-color-emoji/index.html for both Firefox and
> Chrome browsers. I cannot reproduce the issues displayed on the following
> attachments: Screenshot_20180414_111713.png, Remote font comparison in
> openSUSE Tumbleweed,  Remote font comparison in openSUSE Tumbleweed.

The following screenshot attachments are from the same system. I think it might have some messed up system configuration ( fontconfig ).

Screenshot_20180414_111713.png, Remote font comparison in openSUSE
Tumbleweed

I made a new screenshot on another machine (openSUSE Tumbleweed on an iMac).

Comment 14

[Guo Yunhe]

Attachment: Remote font comparison in openSUSE Tumbleweed, Firefox Nightly

Comment 15

[Guo Yunhe]

Attachment: Remote font comparison in openSUSE Tumbleweed

Comment 16

[Alice0775 White]

Web Console and Browser Console:

downloadable font: no supported glyph shapes table(s) present (font-family: "My Noto Color Emoji" style:normal weight:normal stretch:normal src index:0) source: https://guoyunhe.me/demo/noto-color-emoji/NotoColorEmoji.ttf  unknown:8:19

downloadable font: rejected by sanitizer (font-family: "My Noto Color Emoji" style:normal weight:normal stretch:normal src index:0) source: https://guoyunhe.me/demo/noto-color-emoji/NotoColorEmoji.ttf  unknown:8:19

(You should enable |CSS| filter to reprort these error on Web Console and Browser Console)

Comment 17

[Jonathan Kew [:jfkthame]]

It's not expected to work as a webfont unless you enable gfx.downloadable_fonts.keep_color_bitmaps (default is false).

Comment 18

[Daniel_A[:daschilean] [Please contact - [e-mail]]]

Attachment: emoji.png

Comment 19

[Daniel_A[:daschilean] [Please contact - [e-mail]]]

I manage to reproduce this issue on Mac 10.12 with Firefox Nightly 61.0a1(2018-04-24) (64bit) . Please see the attached file with my results.

Comment 20

[Guo Yunhe]

(In reply to Daniel_A from comment #19)
> I manage to reproduce this issue on Mac 10.12 with Firefox Nightly
> 61.0a1(2018-04-24) (64bit) . Please see the attached file with my results.

According to Google, Noto Color Emoji uses a technology that only supported by Linux (GNU/Linux, Android, Chromebook). So Mac won't render it at all.

https://www.google.com/get/noto/help/emoji/

Comment 21

[Guo Yunhe]

Attachment: Screenshot of CodePen example https://codepen.io/guoyunhebrave/pen/ReaPgd

Comment 22

[Guo Yunhe]

After talking with several friends that are using Firefox for Linux, I found I missed an important information: ~/.config/fontconfig/fonts.conf

By default, this configuration doesn't exist or is just as simple as:


<?xml version='1.0'?>
<!DOCTYPE fontconfig SYSTEM 'fonts.dtd'>
<fontconfig/>

This should works fine. Noto Color Emoji can show. (I will upload screenshot later)

But I usually enable hinting on my system:

<?xml version='1.0'?>
<!DOCTYPE fontconfig SYSTEM 'fonts.dtd'>
<fontconfig>
 <match target="font">
  <edit mode="assign" name="hinting">
   <bool>true</bool>
  </edit>
 </match>
 <match target="font">
  <edit mode="assign" name="hintstyle">
   <const>hintfull</const>
  </edit>
 </match>
 <match target="font">
  <edit mode="assign" name="antialias">
   <bool>true</bool>
  </edit>
 </match>
</fontconfig>

After these hinting rules enabled, Noto Color Emoji disappeared.

Comment 23

[Guo Yunhe]

Attachment: Screenshot of CodePen example (after remove fontconfig antialias and hinting rules) https://codepen.io/guoyunhebrave/pen/ReaPgd

Comment 24

[ben.alkov+github]

Absolutely true on Fedora 30, FF 69.0.1 (64-bit), NotoColorEmoji.ttf installed.

No "~/.config/fontconfig/fonts.conf" - all emoji render using the Noto emoji font (the differences between the glyphs in Noto and other emoji fonts are not subtle; it's instantly obvious whether it's being used or not).

Antialiasing turned on in "~/.config/fontconfig/fonts.conf" - no emoji at all on some sites (internal company GitLab instance), some sites show some missing.

Comment 25

[ZaWertun]

Same issue on Fedora 32 with Firefox 77.0.1.

Comment 26

[Johannes Pfrang [:johnp]]

Workaround via e.g. ~/.config/fontconfig/conf.d/89-color-emoji-no-antialias.conf:

<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM 'fonts.dtd'>
<fontconfig>
  <description>Disable anti-aliasing for Noto Color Emoji so it works in Firefox (bug 1454152).</description>
  <match target="scan">
    <test name="family" compare="eq" ignore-blanks="true">
      <string>Noto Color Emoji</string>
    </test>
    <edit name="antialias" mode="assign">
      <bool>false</bool>
    </edit>
    <edit name="hinting" mode="assign">
      <bool>false</bool>
    </edit>
  </match>
</fontconfig>

Comment 27

[sowrensen]

It seems that it only occurs when the antialias section is explicitly written in the ~/.config/fontconfig/fonts.conf file and hinting is disabled:

<match target="font">
    <edit name="hinting" mode="assign">
        <bool>false</bool>
    </edit>
</match>

<match target="font">
    <edit name="antialias" mode="assign">
        <bool>true</bool>
    </edit>
</match>

Now if I remove the antialias section from the file, the emojis just render fine.

Comment 28

[artdevjs]

(In reply to sowrensen from comment #27)

It seems that it only occurs when the antialias section is explicitly written in the ~/.config/fontconfig/fonts.conf file and hinting is disabled:

Now if I remove the antialias section from the file, the emojis just render fine.

10 months later it still works, thanks for the workaround. Finally fixed)
Firefox 89.0 Manjaro KDE

Comment 29

[Adam Fontenot]

Can confirm I see this exact issue with hinting=false and antialias=true, with Noto Color Emoji.

Fortunately the workaround is a decent one. On most recent Linux systems, fontconfig should come with antialiasing enabled by default. See here for Arch Linux, for example. So removing the explicit enable in the fontconfig should be a no-op, but it apparently does have this affect on Firefox.

I'm doing a reqinfo to the triage owner to try to get this bug marked confirmed. It's still set to new even though it's easily replicated and affects multiple users.

Comment 30

[Lee Salzman [:lsalzman]]

Unfortunately, this is specific intentional behavior in Cairo. See here: https://searchfox.org/mozilla-central/source/gfx/cairo/cairo/src/cairo-ft-font.c#1941

If antialiasing is enabled (which is the default anyway), and if hinting is disabled, then it will explicitly not allow bitmaps. I guess this could be relevant in the case that a font is a scalable outline font, but with some bitmaps none the less, that the intention is to go with the outline?

I can see how on a programmatic level that might make sense in that embedded bitmaps in a font must be hinted, just by virtue of how rendering from a bitmap glyph masks. But, on the flip side, I am not sure the intention makes sense from a configuration stand-point - it seems like a valid configuration that a user just may want to disable hinting...

I am not sure there's a great answer about how to handle this. Do we essentially want to detect if an FT font is not a scalable outline font, and just always allow bitmaps to proceed here? Jonathan, Jeff, thoughts?

Comment 31

[Jonathan Kew [:jfkthame]]

Do we essentially want to detect if an FT font is not a scalable outline font, and just always allow bitmaps to proceed here?

I think so. If the font isn't supposed to be used at all (because it isn't a scalable/outline format), then it should've been excluded at the fontconfig level. Given that the font is exposed as usable, we should go ahead and render it -- so if bitmaps are the only kind of glyphs it contains, we need to use them.

My guess is that this wouldn't affect legacy bitmap-only font formats (such as .pcf), where there is no alternative. I think the feature of excluding "embedded bitmaps" is specifically aimed at primarily-outline formats (OpenType/TrueType/Type1) that supported the provision of bitmaps for specific sizes in addition to the scalable outlines. In this model, the bitmaps are in effect "pre-hinted", fixed renderings for specific pixel sizes, so it makes sense to use them or not according to whether or not hinting is enabled.

What that code didn't anticipate (because it didn't exist at the time it was designed) was the use of the normally-scalable OpenType format to package a font that only contains bitmaps and doesn't include outline glyphs at all. This was a new approach used for color-emoji fonts, where the bitmap is not just a hand-tuned raster version of the primary outline glyph, but the only representation available. It doesn't make sense to ignore the bitmaps in such a font.

Comment 32

[Lee Salzman [:lsalzman]]

Attachment: Bug 1454152 - Allow bitmaps from emoji fonts that have no outlines with Fontconfig. r?jfkthame

The default Cairo-compatible behavior of ignoring all embedded bitmaps in a font
if hinting is disabled and antialiasing is enabled seems to not work with emoji
fonts like Noto Color Emoji which do not have any outlines at all, but do have
color bitmaps that present as scalable. Detect this corner case and allow bitmaps.

Comment 33

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3a0ccde3d684
Allow bitmaps from emoji fonts that have no outlines with Fontconfig. r=jfkthame

Comment 34

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/3a0ccde3d684

Comment 35

[Ciprian Georgiu, Desktop QA]

Hi Guo! Are you still experiencing this issue on latest Beta 97.0b9, can you please confirm?

Unfortunately, I was not able to reproduce the bug on an affected Nightly (2021-12-01) build, using Ubuntu 18.04 x64. I've tried to follow the steps from comment 0, and comment 22, but the fonts were always correctly rendered.

Comment 36

[Timea Zsoldos [:tzsoldos], Desktop QA]

I have reproduced this issue using Firefox 61.0a1 (2018.04.14) on Ubuntu 18 x64 with link: https://guoyunhe.me/demo/noto-color-emoji/.
I can confirm this issue is fixed, I verified using Firefox 98.0 on Ubuntu 18 x64, every emoji is colored.

Comment 37

[Adam Fontenot]

Clarification for anyone still seeing this issue:

I can confirm that the issue has been fixed, however if you have the following in your fonts.conf, it will now break in a similar way as before:

<match target="font">
  <edit mode="assign" name="embeddedbitmap">
    <bool>false</bool>
  </edit>
</match>

You might say "well duh, what did you expect? You're explicitly asking for bitmaps embedded in your font files to be disabled." Which is true, however, this is a pretty common way of avoiding the issue where fontconfig will fall back to ugly bitmaps, by simply disabling bitmap fonts entirely. This is what the Arch Wiki suggests: https://wiki.archlinux.org/title/Font_configuration#Disable_bitmap_fonts

Also, including this code doesn't seem to prevent emoji display in any program except Firefox. Firefox seems to interpret these lines as indicating that a selected OpenType font containing bitmaps should not display those bitmaps (i.e. display nothing). Native fontconfig appears to use the lines only in determining which fonts get made available for a given glyph. So if you have Noto Sans Emoji explicitly set as a fallback font for your other fonts (as I do), it continues to work as expected. This is similar to the issue described here, which required a workaround for the hintnone case. I'm not sure if a similar workaround would be appropriate for the case where the user has embeddedbitmap = false.

In any case, I'm going to try removing these lines from my fonts.conf entirely, and see if anything breaks. Maybe I don't have any bitmap glyphs in my fonts to act as ugly fallbacks. An alternative (note: not tested) would be to add the following override to your fonts.conf:

<match target="font">
  <test qual="any" name="family">
    <string>Noto Color Emoji</string>
  </test>
  <edit name="embeddedbitmap">
    <bool>true</bool>
  </edit>
</match>