1794298 - Keycap emojis display incorrectly

Status: RESOLVED FIXED

(Core :: Graphics: Text, defect)

Description

[Teal Dulcet [:tdulcet] (TB Council)]

Attachment: image.png

On some Linux systems without an installed emoji font the keycap emojis display incorrectly even though the "Twemoji Mozilla" fallback font bundled with Firefox is capable of rendering them (see attached screenshot). Note that they do display as expected on webpages that manually set the CSS font-family to "Twemoji Mozilla". This could be related to Bug 543200.

Another issue is that on both Windows and Linux, the unqualified keycap emojis do not display at all when one sets the font to "Twemoji Mozilla". The user just gets a blank zero width character. None of the unqualified emojis display correctly with "Twemoji Mozilla", but for the other non-keycap ones it just show the individual component emojis separately.

Both of these issues can be seen in attachment 9297459 [details] from bug 1692791. Open that attachment in Firefox on Linux (it may take a few seconds to load), check all three checkboxes at top, search the page for "keycap" and then note the fourth and fifth columns which show the native emoji (without specifying a font) and when forcing the "Twemoji Mozilla" font respectively.

The first issue affects my Server Status add-on which supports using these keycap emojis for the browser action icon to display information to the user. This along with Bug 1692791 causes the emojis to get cutoff when generating the icons.

Comment 1

[Ashley Hale [:ahale]]

Attachment: Screenshot 2022-10-09 at 15-49-13 Measure Text Twemoji Mozilla.png

Looking at the linked https://bug1692791.bmoattachments.org/attachment.cgi?id=9297459 in Windows Nightly also behaves very weirdly, is this the same issue?

Comment 2

[Ashley Hale [:ahale]]

Attachment: Screenshot 2022-10-09 at 15-54-55 Measure Text Twemoji Mozilla.png

On Linux Nightly only keycap 10 appears in the attachment https://bug1692791.bmoattachments.org/attachment.cgi?id=9297459 and looks correct for me (?).

Comment 3

[Ashley Hale [:ahale]]

Attachment: Screenshot 2022-10-09 at 16-06-16 Measure Text Twemoji Mozilla.png

Comment 4

[Ashley Hale [:ahale]]

Attachment: Screenshot 2022-10-09 at 16-08-15 Measure Text Twemoji Mozilla.png

Checked all 3 boxes and got this on Windows Nightly:

Comment 5

[Ashley Hale [:ahale]]

So on Linux Dev on a pretty fresh Ubuntu install, I get https://bugzilla.mozilla.org/attachment.cgi?id=9297757 and on Windows Nightly I get https://bugzilla.mozilla.org/attachment.cgi?id=9297758

I don't see a link to a page that reproduces the image you attached in the original post, is there something I should be looking at for that?

It would be helpful to attach about:support as well, so we can distinguish which rendering features are being used.

Comment 6

[Ashley Hale [:ahale]]

Also, is this a regression or was it always broken? If you don't mind using mozregression to determine if it's a regression, it may help a lot in understanding the issue to know what change broke it.

Triage notes: Rating as S2 because this visual artifact seems likely to occur when using a fallback that ought to work, and may affect a majority of users (e.g. I was able to see issues on a regular Windows install, but was unable to repro on a fresh Ubuntu install), may revise rating depending on further details of impact.

Comment 7

[Teal Dulcet [:tdulcet] (TB Council)]

Attachment: image.png

Looking at the linked https://bug1692791.bmoattachments.org/attachment.cgi?id=9297459 in Windows Nightly also behaves very weirdly, is this the same issue?

Yes, those are the "unqualified" keycap emojis, so this would be the second issue. Note how column five is blank and it should at least be identical to the preceding column.

On Linux Nightly only keycap 10 appears in the attachment https://bug1692791.bmoattachments.org/attachment.cgi?id=9297459 and looks correct for me (?).

You would need to check all three checkboxes at top, otherwise not all the emojis may show (the checkboxes are related to bug 1692791, so they are irrelevant in this bug). Yes, the keycap 10 emoji displays as expected, likely because it does not involve a variation selector character like the others (see bottom of screenshot in comment 0).

So on Linux Dev on a pretty fresh Ubuntu install, I get https://bugzilla.mozilla.org/attachment.cgi?id=9297757

It looks like your Ubuntu system includes an emoji font, which is being used for the keycap emojis. What is strange is that it looks like this font is also being used when forcing "Twemoji Mozilla" (the fifth column), which I have not seen.

I attached a screenshot of what I see, which is on a minimal installation of Ubuntu 20.04 with the latest Firefox 105. Note how the emojis in the fourth column are displayed incorrectly. This is the first issue, as the "fully-qualified" ones (every other row) should be identical to the fifth column.

I don't see a link to a page that reproduces the image you attached in the original post, is there something I should be looking at for that?

No, the screenshot from comment 0 was just from Emojipedia. I thought attachment 9297459 [details] would be better to reproduce the issue, as it shows both the native emoji and when forcing the "Twemoji Mozilla" font side by side.

Also, is this a regression or was it always broken? If you don't mind using mozregression to determine if it's a regression, it may help a lot in understanding the issue to know what change broke it.

I do not believe this is a regression, at least not a recent regression. It may have been fixed at some point by Bug 1278614, but it is not clear (the link in that bug is dead). Mozilla should consider integrating the official Unicode Emoji test data into their test suite to prevent these types of bugs from occurring: https://unicode.org/Public/emoji/latest/emoji-test.txt

Comment 8

[Teal Dulcet [:tdulcet] (TB Council)]

Attachment: image.png

What is strange is that it looks like this font is also being used when forcing "Twemoji Mozilla" (the fifth column), which I have not seen.

It seems this is a third issue, as on many systems even when manually setting the font to "Twemoji Mozilla" with CSS, it is still using the native font for the keycap emojis. The attached screenshot is on Windows 11 with the latest Firefox 105, but I am able to reproduce it on Windows 10 and it is also shown in comment 3 and comment 4. For comparison, note how the keycap 10 emoji on the very bottom displays as expected.

Comment 9

[Bob Hood [:bhood]]

Jonathan, any insights you could provide here?

Comment 10

[Jonathan Kew [:jfkthame]]

So there are (at least) two distinct issues here. One is that when the VS16 variation selector is not included in the keycap-emoji sequence (for sequences based on default-text-style characters such as the ASCII digits), we determine that Twemoji Mozilla supports the character (it's present in the 'cmap'), but in fact it just renders blank glyphs for the digits, and for the COMBINING ENCLOSING KEYCAP character U+20E3. They only map to something visible (via ligation rules in the font) when the U+FE0F variation selector is present.

That's pretty much unavoidable from the Firefox side, I think; to fix it we'd need to modify the font so as to actually support the non-VS16 sequence. But that should render a text-style glyph, not the full-color emoji-style one, so it's not just a question of adding the VS-less ligature rule; we'd also need to add new, non-color glyphs to support this.

(Or, I suppose, we could try to detect when the glyphs are blank, and fall back to a different font; but I'm hesitant to go down that path both for performance reasons and because it seems wrong to second-guess what a font designer does. There's probably a valid -- if slightly weird -- use-case somewhere for a font that has blank glyphs for some/all characters....)

The other issue is that when the variation selector is present, we still fail to render the Twemoji Mozilla keycap glyph (at least that's the behavior I'm seeing on Windows), and fall back to the system emoji font (Segoe UI Emoji, in my case) instead.

This happens because we call gfxFontGroup::FindFontForChar for the first character of the sequence (e.g. an ASCII digit), also passing it the following VS16 for context. That prompts it to seek a font with a color glyph. But Twemoji Mozilla doesn't provide a color glyph for the digit alone (it only provides one for the full three-character ligature including U+20E3, but at this point we're not looking at entire sequences). So therefore we consider Twemoji Mozilla a poor choice in the CheckCandidate function, and fall back to Segoe UI Emoji instead.

Doing font selection on a per-cluster basis (rather than per-character with only limited help from context), which is bug 543200, would help there, but is a fairly complex matter. (Masatoshi started a patch there many years ago, but that was before the added complication of text- vs emoji-style presentation, so things have only got trickier!)

Meanwhile, I think we can probably improve our heuristics here so that at least these Twemoji Mozilla examples -- in the form with VS16 -- will render the expected glyphs. I'm thinking that if we see a case where VS16 has been explicitly used to request emoji-style rendering where the base character would otherwise have been text-style, and the font does support both the base codepoint and the VS16 codepoint, we should assume that it knows what it is doing with these codepoints and so return true from the CheckCandidate function.

Comment 11

[Jonathan Kew [:jfkthame]]

Attachment: Bug 1794298 - Improve font-selection heuristics so that Twemoji-Mozilla keycaps (in emoji style) work as expected. r=#layout-reviewers

Comment 12

[Jonathan Kew [:jfkthame]]

Attachment: Bug 1794298 - Add a reftest for Twemoji Mozilla keycap on Windows. r=#layout-reviewers

This depends on Windows emoji font availability, so is skipped on other
platforms. Without the fix here, the testcase will fail to use Twemoji
and so matches the (not-)reference file.

Comment 13

[Jonathan Kew [:jfkthame]]

Attachment: Bug 1794298 - Improve font-selection heuristics so that Twemoji-Mozilla keycaps (in emoji style) work as expected. r=#layout-reviewers

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/daaaf734e8d1
Improve font-selection heuristics so that Twemoji-Mozilla keycaps (in emoji style) work as expected. r=emilio
https://hg.mozilla.org/integration/autoland/rev/9c484601dceb
Add a reftest for Twemoji Mozilla keycap on Windows. r=emilio

Comment 15

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/daaaf734e8d1
https://hg.mozilla.org/mozilla-central/rev/9c484601dceb