1685434 - Emoji won’t be displayed with monochrome fonts even if explicitly requested

Status: VERIFIED FIXED

(Core :: Layout: Text and Fonts, defect)

Description

[Charlotte Eiffel Lilith Buff]

Attachment: Demonstration of the bug with (upper half) and without (lower half) a @font-face declaration.

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:84.0) Gecko/20100101 Firefox/84.0

Steps to reproduce:

I have created a small HTML document to demonstrate the bug: http://randomguy32.de/misc/firefox-emoji-font-bug.html

To see the effect, you will need to have the font Symbola (https://dn-works.com/ufas/) installed on your system. Alternatively, you can use any other non-colourful font that contains the characters U+1F620 and U+1F44D; you just have to edit the HTML file to substitute the new font name.

This bug has been present since two updates ago, so probably Firefox 83.

Actual results:

Firefox will always render emoji characters (those with the character property Emoji_Presentation=Yes) using multi-colour fonts such as Segoe UI Emoji or Twemoji Mozilla, even when other, black-and-white fonts are provided (be they local or downloaded) and explicitly requested via CSS.

This behaviour can only be overriden by defining the desired monochrome font in a CSS @font-face declaration. However, even then emoji that have U+FE0F VARIATION SELECTOR-16 (the emoji variation selector) applied will continue to prefer the polychrome fonts.

If you have the Symbola font installed and look at the example page I provided, you will see that the “😠” will be correctly displayed using Symbola, while the “👍️” will be displayed using a polychrome font (most likely Segoe UI Emoji if you are on Windows). This is shown in the upper portion of the attached screenshot.

If you download the HTML file, remove the @font-face declaration from the <style> tag, and view it again, both emoji will now be displayed using a polychrome font, even though the local Symbola font continues to be available.

It does not matter whether the @font-face declaration is used to actually download a font, only that it exists at all.

This disregard for font declarations also extends to user preferences. I have my font.name-list.emoji set to “Symbola, serif, sans-serif”, and yet Segoe UI Emoji and Twemoji Mozilla are always used for emoji where no fonts have been specified.

Firefox will only choose a monochrome font for emoji when no polychrome fonts containing the characters exist at all, as is the case for the new emoji introduced in Unicode 13.

Expected results:

Firefox should actually use the fonts specified by CSS or user settings instead of always prefering polychrome fonts for emoji characters.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Comment 2

[Jonathan Kew [:jfkthame]]

This sounds like it is by design: Firefox does its best to respect the presentation style (see http://unicode.org/reports/tr51/#Presentation_Style) suggested by the Unicode characters, so that characters with Emoji_Presentation=Yes will by default appear with an emoji-style glyph, unless a variation selector is used to explicitly request text-style rendering.

Comment 3

[Charlotte Eiffel Lilith Buff]

(In reply to Jonathan Kew (:jfkthame) from comment #2)

This sounds like it is by design: Firefox does its best to respect the presentation style (see http://unicode.org/reports/tr51/#Presentation_Style) suggested by the Unicode characters, so that characters with Emoji_Presentation=Yes will by default appear with an emoji-style glyph, unless a variation selector is used to explicitly request text-style rendering.

It is good that Firefox respects the Emoji_Presentation property in the absence of any font declarations, but when I explicitly tell Firefox what fonts to use for emoji style display, why should the browser get to override my preferences?

Comment 4

[Jonathan Kew [:jfkthame]]

There are a couple of aspects to this:

when I explicitly tell Firefox what fonts to use

could refer to specifying fonts via the CSS font-family property, or it could refer to the font.name-list.emoji setting.

We need to consider each of these separately; they're quite different cases, because one is controlled by the page author, and the other is in the hands of the Firefox user.

In general, it doesn't work well to apply the font-family from CSS to emoji characters, because authors usually use it to set their preferred fonts for text content, without regard for whether those fonts happen to support some emoji codepoints; when emoji then occur "randomly" in the text (perhaps in user-generated content), the results are often poor, with a mix of glyphs from different fonts depending on what character coverage they happen to have; and the results are unpredictable depending on whether the user has the same fonts as the author assumed, or is seeing fallbacks. Even when authors do think about emoji, they often set font-family to one or more emoji fonts that they happen to have, but if users don't have the same emoji fonts available, they won't see the intended result.

So in general I believe it works better -- more predictably and consistently -- for Firefox to respect the Emoji_Presentation property and give it higher precedence than CSS font-family. All characters whose properties (including the consideration of variation selectors) imply a request for an emoji presentation will choose a color font if possible; and characters whose properties point to a text presentation will prefer a non-color font.

I do think the case with font.name-list.emoji is different. While the default for font.name-list.emoji is (logically enough) a color-emoji font, if the user has configured this to a non-color font such as Symbola, I think it's reasonable to argue that the user -- who should be in control of their experience -- has explicitly chosen not to see emoji characters rendered with an emoji-style font, and the browser should respect that.

So what I propose is that while the presentation style requested by the Unicode characters/properties will continue to take precedence (for emoji codepoints) over the fonts specified by CSS, we can (and I think should) change the handling of the font preferences so that the font(s) specified in font.name-list.emoji will always be used regardless of their presentation style, as they represent a deliberate user choice.

Comment 5

[Charlotte Eiffel Lilith Buff]

In general, it doesn't work well to apply the font-family from CSS to emoji characters, because authors usually use it to set their preferred fonts for text content, without regard for whether those fonts happen to support some emoji codepoints; when emoji then occur "randomly" in the text (perhaps in user-generated content), the results are often poor, with a mix of glyphs from different fonts depending on what character coverage they happen to have; and the results are unpredictable depending on whether the user has the same fonts as the author assumed, or is seeing fallbacks. Even when authors do think about emoji, they often set font-family to one or more emoji fonts that they happen to have, but if users don't have the same emoji fonts available, they won't see the intended result.

I don’t agree with this assessment. Explicitly overriding default emoji fonts – especially with non-typical fonts like Symbola – is not something that designers do incidentally or on a whim. Nobody styles their website in such a way unless they feel that having monochrome emoji is absolutely necessary for their intended design. In fact, the reason I even became aware of the finer details of this quirk (such as the interaction with U+FE0F or the @font-face exception) is because I do have a page on my personal website where I want and need emoji to be displayed with a monochrome font I designed myself. That page is now broken in Firefox for the time being, but continues to work just fine in every other browser.

Emoji are text characters like any other, and font attributes should apply to them like they do to any other text characters. Firefox shouldn’t even have to concern itself at all with whether a given font contains monochrome or polychrome glyphs; it’s completely outside a web browser’s sphere of relevance.

Comment 6

[Jonathan Kew [:jfkthame]]

If "having monochrome emoji is absolutely necessary for their intended design", the author should be using the U+FE0E variation selector to indicate this, rather than just depending on a particular font (which may or may not be available/load successfully for any given user).

You can specify a particular monochrome font that supports the emoji codepoints in addition to using the variation selector to express a preference for text-style rendering; this should work in the widest range of situations.

Comment 7

[Charlotte Eiffel Lilith Buff]

If "having monochrome emoji is absolutely necessary for their intended design", the author should be using the U+FE0E variation selector to indicate this

That would be an effective workaround to force Firefox to exhibit the same behaviour as every other browser on the market, but I still think it is completely ludicrous that Firefox should be able to override the wishes of both the user and the designer like that.

If I have a font installed, and the CSS says to use that particular font, then Firefox shouldn’t get to say “Actually, what you reeeeally want is that completely unrelated font I have picked for you without your consent”.

Comment 8

[Jonathan Kew [:jfkthame]]

I do sympathize with your position, but I think there's an irreconcilable tension here between the two "sources of truth" for how the characters should appear: their Unicode properties vs CSS font properties. Simply applying fonts specified in CSS without regard to the presentation style implied by the Unicode characters themselves leads to problems of its own (e.g. bug 1658190 and other similar reports).

Emoji are text characters like any other, and font attributes should apply to them like they do to any other text characters

I don't think this is the whole picture. Unlike "normal" text characters, emoji characters have two distinct styles of possible presentation, and Unicode provides both guidance for which to use by default, and a mechanism for the author to express a specific preference. This is independent of any styling of the text, and it seems appropriate for the browser to handle it.

Comment 9

[Jonathan Kew [:jfkthame]]

(In reply to Charlotte Buff from comment #7)

That would be an effective workaround to force Firefox to exhibit the same behaviour as every other browser on the market, but I still think it is completely ludicrous that Firefox should be able to override the wishes of both the user and the designer like that.

The patch I'm proposing here will allow the user to choose a monochrome emoji font as their default, and have that wish respected. The designer cannot rely on specific fonts being available unless they use @font-face to provide them (and even then, fonts may fail to load for any number of reasons), and therefore if they regard monochrome emoji as an important aspect of their design, they should use the available Unicode mechanism to express this, independent of what fonts are present.

To look at things from the other side, if I'm writing content for a site and I expressly request a colorful emoji-style smiling face using <U+263A U+FE0F>, it would seem wrong to display this as a monochrome text-style smiley just because the normal text font named in font-family happens to include the smiley-face codepoint.

Comment 10

[Charlotte Eiffel Lilith Buff]

While testing some things I have discovered another dimension to this issue: It isn’t just U+FE0F that prevents the application of monochrome fonts, but the Fitzpatrick modifiers (U+1F3FB through U+1F3FF) as well. This is to be expected because the Fitzpatrick modifiers are also defined to induce emoji-style display of their respective base character, but this means that it is currently completely impossible to render emoji modifier sequences with a monochrome font in Firefox. And some monochrome fonts (like Segoe UI Symbol) do actively support them.

Comment 11

[Jonathan Kew [:jfkthame]]

If you load the desired monochrome font using @font-face then I think it should work, no?

Comment 12

[Charlotte Eiffel Lilith Buff]

I do sympathize with your position, but I think there's an irreconcilable tension here between the two "sources of truth" for how the characters should appear: their Unicode properties vs CSS font properties. Simply applying fonts specified in CSS without regard to the presentation style implied by the Unicode characters themselves leads to problems of its own (e.g. bug 1658190 and other similar reports).

I do get that. The keycap bases in particular are an unfortunate case due to their unification with ASCII characters. However, I am still not convinced that Firefox unilaterally deciding to throw stylesheets out the window just to preserve the text-style/emoji-style distinction as a blanket solution is the right way.

The reason I used U+FE0F on that personal page of mine I cited as an example despite desiring a monochrome look is because, semantically, it was the right choice. The emoji characters function as proper emoji on that page. If you copied the contents of that page and pasted them somewhere else where the original styles no longer apply, emoji-style display would be more appropriate for all of them. The monochrome font I intended for them only works if it is surrounded by the entire stylistic context of the rest of the page. In particular, the page is meant to look like it was hand-written on a piece of paper.

It is also a question of consistency. Not all emoji characters even allow U+FE0E and U+FE0F, so there is just no way to force text-style display for a lot of them. If my set of emoji contains some characters that support variation selectors and some that don’t, then skewing towards U+FE0F is the only way for them to have a uniform look. Otherwise, copy-pasting the content will result in a random mishmash of monochrome and polychrome glyphs.

Lastly – and I understand that this is getting increasingly niche – this behaviour makes it impossible to properly switch between monochrome and polychrome emoji fonts without also manipulating the underlying text content in some way.

I have this page on my website that simply lists every single emoji for easy copy/paste access: http://randomguy32.de/unicode/misc/emoji/

Because that is its main purpose, all the emoji (for which it is valid) dutifully include U+FE0F. By default, Segoe UI Emoji is the font of choice for that page, but by appending various query strings to the URL (such as ?font=symbola or ?font=unifont), other font stacks can be selected. However, U+FE0F completely blocks fonts like Symbola or Unifont, so the result is a mess of mostly monochrome glyphs with random blobs of colour strewn inbetween.

With your proposed solution, I could fix this issue on my personal machine by lying to Firefox about the nature of those fonts in about:config, but as soon as I want to copy/paste emoji on any other computer, I would be greeted by the same inconsistent mess again. (Also not discounting the possibility that other people might want to use the same site for themselves.)

And it’s not like I could circumvent this by simply embedding Symbola as a downloadable font either – even if its license allowed for that – because U+FE0F would still block the font unless I have direct access to the browser settings.

Comment 13

[Charlotte Eiffel Lilith Buff]

(In reply to Jonathan Kew (:jfkthame) from comment #11)

If you load the desired monochrome font using @font-face then I think it should work, no?

Not for characters with VS16 or modifier sequences, unfortunately. Those beat even @font-face.

Comment 14

[Jonathan Kew [:jfkthame]]

(In reply to Charlotte Buff from comment #13)

(In reply to Jonathan Kew (:jfkthame) from comment #11)

If you load the desired monochrome font using @font-face then I think it should work, no?

Not for characters with VS16 or modifier sequences, unfortunately. Those beat even @font-face.

Ah, right -- re-checking the code, I see that using @font-face can override default-emoji presentation, but if the emoji presentation is explicitly called for by the encoded text (i.e. using VS16 or a Fitzpatrick modifier) then we consider that the requested presentation should be honored.

Unfortunately I don't see how to reconcile the two points of view here. Previously, people complained that Firefox didn't reliably honor the emoji presentation request (e.g. see bug 1564193, where the reporter's "expected results" are the exact opposite of what you've asked for here: "All fully qualified emoji should be shown as emoji, no matter what font family was specified. Emoji variation sequence should have higher priority than font-family property.")

Comment 15

[Charlotte Eiffel Lilith Buff]

Well, I suppose I’ll just have to live with the fact that my specific needs are at odds with the way most people expect emoji to work. Once the issue with font.name-list.emoji is fixed, I’ll at least be able to return to the old behaviour for most general purposes. And for the more arcane stuff, I’ll need to write special code that dynamically replaces all variation selectors on the page with their respective counterparts everytime a font change occurs.

Comment 16

[Jonathan Kew [:jfkthame]]

I'm sorry I don't have a better answer for you at the moment, but I don't see how to "fix" your use-case without simultaneously "breaking" other people's expectations.

In principle maybe we could add an about:config setting to modify the behavior here, to give CSS font-family priority over Unicode emoji presentation controls, though I'd be a bit concerned about the performance implications of adding more options to the font-selection process, which is very performance-critical; and in any case, that would still only enable you to modify the behavior in your own browser, it wouldn't affect what other users see when they visit your page.

Comment 17

[Charlotte Eiffel Lilith Buff]

Restoring the functionality of font.name-list.emoji would be enough for most general purposes, because then I could simply tell Firefox that Symbola means emoji-style, even though the font’s glyphs aren’t actually polychrome. At the moment that setting seemingly doesn’t have any effect whatsoever because Firefox completely bypasses it do determine what fonts to use for emoji.

Comment 18

[Jonathan Kew [:jfkthame]]

Attachment: Bug 1685434 - Respect fonts specified in the font.name-list.emoji preference even if this overrides the Unicode Emoji_Presentation style. r=m_kato

This is a minor adjustment to the font-selection heuristics implemented in bug 1371386:
for emoji codepoints, we should accept the font(s) specified in the font.name-list.emoji
preference even if it's a monochrome font for characters that would normally be expected
to use a color rendering, as this is the user's choice.

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/176e40f7636b
Respect fonts specified in the font.name-list.emoji preference even if this overrides the Unicode Emoji_Presentation style. r=m_kato

Comment 20

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/176e40f7636b

Comment 21

[Ciprian Georgiu, Desktop QA]

Hi jfkthame, it seems that on the latest Beta build 86.0b8, the thumbs up sign is not displayed in monochrome, as I understand from comment 0. I installed on mac10.15, the Symbola font from https://dn-works.com/ufas/. FWIW, when checking on Chrome, the sign looks to be displayed correctly in monochrome. Please see the screenshot, here https://drive.google.com/file/d/1s38BnqoQ4Fde-P2RWSVoz3-Bz0Z5dRP3/view?usp=sharing.

Comment 22

[Jonathan Kew [:jfkthame]]

(In reply to Ciprian Georgiu [:ciprian_georgiu], Release Desktop QA from comment #21)

Hi jfkthame, it seems that on the latest Beta build 86.0b8, the thumbs up sign is not displayed in monochrome, as I understand from comment 0. I installed on mac10.15, the Symbola font from https://dn-works.com/ufas/. FWIW, when checking on Chrome, the sign looks to be displayed correctly in monochrome. Please see the screenshot, here https://drive.google.com/file/d/1s38BnqoQ4Fde-P2RWSVoz3-Bz0Z5dRP3/view?usp=sharing.

Did you include Symbola (ahead of Apple Color Emoji) in the font.name-list.emoji list? So, set it to "Symbola, Apple Color Emoji". That's required for it to be used in the case where the text has the U+FE0F variation selector, which otherwise requests emoji-style display.

With the default setting for the pref, the thumbs-up with U+FE0E should always display in color; with U+FE0E it would display monochrome.

Comment 23

[Ciprian Georgiu, Desktop QA]

Thanks Jonathan! I was able to reproduce the issue with your instructions.

The fix is verified on the latest Beta 86.0b9, across platforms: Win 10 x64, macOS 10.15 and Ubuntu 18.04 x64.