1756720 - Firefox doesn't display WingDings font (and other symbol fonts).

Status: VERIFIED FIXED

(Core :: Layout: Text and Fonts, defect)

Description

[Rachel Martin]

Attachment: test.html

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Firefox/91.0

Steps to reproduce:

Open the enclosed HTML document in Firefox and Chrome on a Windows system with Wingdings installed.

Actual results:

FF displays: Following is our agenda.
Chrome displays a bunch of symbols.

Expected results:

FF should behave the same as Chrome.

Looks like nsIFontEnumerator.EnumerateAllFonts() doesn't return them. Thunderbird calls that function, but those symbol fonts are not offered of editing and are also not displayed in received messages.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Layout: Text and Fonts' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 2

[Rachel Martin]

Based on nsIFontEnumerator.idl, this should go to gfx.

Comment 3

[Robert Longson [:longsonr]]

See also bug 236854 and bug 31538 and various others.

Comment 4

[Rachel Martin]

Without reading through a lot of bug reports (bug 31538 alone has 13 duplicates, bug 193404 has 12), can you please summarise why the request is invalid. At least there is a web-compat issue here, since Chrome displays the example differently.

Bug 193404 comment #2 only says: "verified-this really shouldn't work, ...". Bug 31538 comment #12 also doesn't give a good reason.

People at Chrome/Chromium have a different opinion or are they just wrong?

Comment 5

[Robert Longson [:longsonr]]

bug 236854 comment 1 explains it. Chrome is just wrong therefore.

Comment 6

[Jonathan Kew [:jfkthame]]

To summarize, the glyphs in the Wingdings font are encoded in the Unicode private use area at codepoints U+F020 .. U+F0FF. So by using those codepoints, they can be displayed just fine in Firefox (or in other browsers).

I guess Chrome also supports rendering them using the (extended-)ASCII codepoints that were used in pre-Unicode software, where "symbol fonts" just re-used the limited range of 8-bit codepoints and assigned arbitrary glyphs to them. This is of course terrible for interoperability. In a world of Unicode-based software, it should be universal and unambiguous that U+0041 is always a LATIN LETTER CAPITAL A; it is never a hand with two fingers raised.

(There is actually a standardized Unicode codepoint for VICTORY HAND at U+270C. However, Wingdings doesn't encode its glyph there, so to display its VICTORY HAND symbol you'd need to use something like  -- which is pretty sad from an interoperability perspective.)

Comment 7

[Rachel Martin]

Thanks for all your explanations here. It's still an issue for e-mail rendering/composing as can be seen in bug 1751694.

Comment 8

[Jonathan Kew [:jfkthame]]

I'd actually like to re-open this for further discussion; somewhat reluctantly, I think we should consider making a change here.

The Wingdings font (and other similar "legacy" symbol/dingbat fonts) is in a peculiar position with respect to character encoding. Its cmap table contains two subtables, as shown (with most of the content elided) here:

  <cmap>
    <tableVersion version="0"/>
    <cmap_format_0 platformID="1" platEncID="0" language="0">
      ...
      <map code="0x20" name="uniF020"/>
      <map code="0x21" name="uniF021"/>
      <map code="0x22" name="uniF022"/>
      ...
      <map code="0xfe" name="uniF0FE"/>
      <map code="0xff" name="uniF0FF"/>
    </cmap_format_0>

    <cmap_format_4 platformID="3" platEncID="0" language="0">
      <map code="0xf020" name="uniF020"/><!-- ???? -->
      <map code="0xf021" name="uniF021"/><!-- ???? -->
      <map code="0xf022" name="uniF022"/><!-- ???? -->
      ...
      <map code="0xf0fe" name="uniF0FE"/><!-- ???? -->
      <map code="0xf0ff" name="uniF0FF"/><!-- ???? -->
    </cmap_format_4>

The second of these is the subtable Firefox uses; it is tagged with platform ID 3, encoding 0, which is the legacy MS Windows "Symbol" encoding.[1] This maps PUA codepoints beginning at U+F020 to the font's glyphs (as also indicated by the glyph names).

However, there's also the first subtable, platform ID 1, encoding 0; this is the legacy MacRoman code page. Note how this maps the 8-bit MacRoman codes 0x20..0xFF to the same Wingdings glyphs uniF020 etc. Now, 0x20 in MacRoman is simply SPACE, 0x21 is EXCLAMATION, etc.; so this illustrates how, in the legacy world, fonts like Wingdings "overlay" standard ASCII characters with arbitrary symbol glyphs. While the MS Symbol subtable assigns arbitrary PUA codepoints to the glyphs, the existence of the MacRoman one makes it clear that they can also be viewed as simple replacements for common Latin characters.

The way fonts like Wingdings are encoded is obsolete, and indeed the OpenType spec explicitly says that "[w]hen creating fonts for Windows, Unicode 'cmap' subtables [not the Symbol encoding] should always be used...". But Wingdings (and friends) isn't likely to go away any time soon, for reasons of backward compatibility, nor can we expect its encoding to be updated.

Given that other browsers[2] (and email software such as Outlook, see bug 1751694) do support applying such fonts to Latin characters, remapping the codepoints to access the (PUA-encoded) Wingdings glyphs, I think we should consider doing the same. Fundamentally, if content is explicitly styled with Wingdings, it's clear that the intention is to display the "corresponding" Wingdings glyphs.

I think we can quite readily do this, by simply mapping the range U+0020..00FF to the same glyphs as the PUA range at U+F020..F0FF for fonts that use the MS Symbol encoding. Essentially, the decision to call for such a font is implicitly also a call to override the Unicode identity of those character codes and interpret them as other symbols instead. This is unfortunate, in that such data loses the (huge) interoperability benefits of using a standardized encoding. But the fact that Wingdings is present on the overwhelming majority of the world's desktop computers, and "just works" in all other major browsers, means that I think compatibility overrides Unicode purity here.

[1] https://docs.microsoft.com/en-gb/typography/opentype/spec/cmap#windows-platform-platform-id--3
[2] Tested in Safari and Chrome on macOS, Edge and Chrome on Windows, Chromium on Linux: all display the test file as Wingdings glyphs.

Comment 9

[Jonathan Kew [:jfkthame]]

(Moved this to Layout instead of Graphics, as it's more about font selection than the actual rendering.)

Comment 10

[Jonathan Kew [:jfkthame]]

Attachment: Bug 1756720 - When using legacy MS Symbol fonts, map character codes from the U+00xx range to the PUA range U+F0xx found in the cmap subtables. r=#layout-reviewers

Comment 11

[Rachel Martin]

Thanks for addressing this issue.
nsIFontEnumerator.EnumerateAllFonts() doesn't return symbol fonts since they are filtered here:
https://searchfox.org/mozilla-central/rev/a17011de808c24f015ad03debe7a157c1b43b602/gfx/thebes/gfxGDIFontList.h#200
Do you plan to change that behaviour or is this going to be a display-only fix? The function is used in TB to offer the user a choice of fonts. If they edit text which is in a font that is not installed, TB shows "(not installed)" next to the font name. It would continue showing that despite Wingdings being installed and rendered.

Comment 12

[Jonathan Kew [:jfkthame]]

This is specifically about rendering: whether Gecko considers Wingdings (etc) as a valid candidate to render the text when it is requested via the font-family property, or falls back to the next font in the list (or a generic/default font).

I think the behavior of nsIFontEnumerator.EnumerateAllFonts() should be addressed separately, if at all -- though I think that it may become a non-issue in more recent versions anyhow, as it (currently, at least) doesn't apply the same filtering when gfx.e10s.font-list.shared is true. And arguably it's not such a bad thing if TB users can't explicitly choose to use "symbol-encoded" fonts: they're bad for compatibility/interoperability, and it's better to use the proper Unicode codepoints for whatever dingbats people want to represent.

So the change I'm proposing here is intended to help us display content created elsewhere in the intended way, not necessarily to facilitate the creation of more such content. I think EnumerateAllFonts deserves a separate bug, and I'm not sure offhand how I feel about it; the case for exposing symbol fonts there may be weaker.

Comment 13

[Rachel Martin]

Yes, with gfx.e10s.font-list.shared at true, all fonts on the system appear to appear in the TB menu. There is a TB add-on that displays a font sample (the font name in the font face, and a tooltip), and the following fonts show as symbols:

<span style="font-family:Bookshelf Symbol 7;">Bookshelf Symbol 7</span><br>
<span style="font-family:Marlett;">Marlett</span><br>
<span style="font-family:MS Reference Speciality;">MS Reference Speciality</span><br>
<span style="font-family:MT Extra;">MT Extra</span><br>
<span style="font-family:MT Symbol;">MT Symbol</span><br>
<span style="font-family:Symbol;">Symbol</span><br>
<span style="font-family:Tera Special;">Tera Special</span><br>
<span style="font-family:Webdings;">Webdings</span><br>
<span style="font-family:Wingdings;">Wingdings</span><br>
<span style="font-family:Wingdings 2;">Wingdings 2</span><br>
<span style="font-family:Wingdings 3;">Wingdings 3</span><br>

(Possibly there are typos in this.) Not all of them show as symbols in Chrome. You might want to compare if compatibility is the aim here.

Comment 14

[Rachel Martin]

Attachment: symbol-fonts.png

FF would display, for example "MS Reference Speciality", as symbols, when Chrome doesn't. Perhaps there are other properties of the font that can be used to determine how to render the font.

Comment 15

[Rachel Martin]

Hmm, when pasting the HTML from comment #13 into a TB compose window, it displays differently to what the menus shows (comment #14). It's the same as Chrome, apart from MT Symbol (which still shows as Greek characters as you can see in the picture). The code of the add-on for the menu is:

        for (let i = 7; i < nodes.length; ++i) {
          let n = nodes[i];
          n.setAttribute('style', 'font-family: "' + n.value + '" !important;');
          n.tooltipText = n.value;
        }

So somehow the menu is rendered differently.

Comment 16

[Rachel Martin]

Attachment: symbol-fonts-difference-on-menu.png

Wingdings 2/3 being rendered differently in message body and menu (hopefully no typos).

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/cd5ac5f6ff6e
When using legacy MS Symbol fonts, map character codes from the U+00xx range to the PUA range U+F0xx found in the cmap subtables. r=emilio

Comment 18

[Rachel Martin]

Re. comment #13 to comment #16:

• <span style="font-family:MT Symbol;">MT Symbol</span> displays MT Symbol in FF now (grabbed a build from autoland), but in TB in the message compose window and in the font menu it shows MS <some Greek> (see TB screenshots in comment #14 and comment #16). Why is that?
• Why are Windings 2/3 displayed differently in menu and compose window document (comment #16)?

Comment 19

[Jonathan Kew [:jfkthame]]

(In reply to Rachel Martin from comment #18)

Re. comment #13 to comment #16:

• <span style="font-family:MT Symbol;">MT Symbol</span> displays MT Symbol in FF now (grabbed a build from autoland), but in TB in the message compose window and in the font menu it shows MS <some Greek> (see TB screenshots in comment #14 and comment #16). Why is that?

I'll try and look at this tomorrow at my Windows machine, and see what it shows...

• Why are Windings 2/3 displayed differently in menu and compose window document (comment #16)?

In the document, is the content something like comment 13 shows?

<span style="font-family:Wingdings 2;">Wingdings 2</span><br>
<span style="font-family:Wingdings 3;">Wingdings 3</span><br>

That won't work because a font name that has a number as a separate "word" needs to be quoted in CSS (see https://drafts.csswg.org/css-fonts-4/#family-name-syntax). So try

<span style="font-family: 'Wingdings 2';">Wingdings 2</span><br>
<span style="font-family: 'Wingdings 3';">Wingdings 3</span><br>

and I'd expect you to see the dingbats for these as well.

Comment 20

[Rachel Martin]

OK, we corrected the spelling (of "Specialty") and added the missing quotes:

<span style="font-family:'Bookshelf Symbol 7';">Bookshelf Symbol 7</span><br>
<span style="font-family:Marlett;">Marlett</span><br>
<span style="font-family:'MS Reference Specialty';">MS Reference Specialty</span><br>
<span style="font-family:'MT Extra';">MT Extra</span><br>
<span style="font-family:'MT Symbol';">MT Symbol</span><br>
<span style="font-family:Symbol;">Symbol</span><br>
<span style="font-family:'Tera Special';">Tera Special</span><br>
<span style="font-family:Webdings;">Webdings</span><br>
<span style="font-family:Wingdings;">Wingdings</span><br>
<span style="font-family:'Wingdings 2';">Wingdings 2</span><br>
<span style="font-family:'Wingdings 3';">Wingdings 3</span><br>  </body>

now the add-on menu which sets the font with n.setAttribute('style', 'font-family: "' + n.value + '" !important;'); (yes, quotes weren't missing in the code) and message body where the shown HTML was inserted look 100% the same.

The only odd observation is that "MT Symbol" shows differently in FF/Chrome and TB. In TB it's "MS sigma upsilon mu beta omicron lambda (in Greek)" (seen in comment #16).

Comment 21

[Marian-Vasile Laza]

Backed out changeset cd5ac5f6ff6e (Bug 1756720) for causing reftest failures on 399636-standards-css.html.
Backout link
Push with failures - R2
Failure Log

Comment 22

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3b719cd12933
When using legacy MS Symbol fonts, map character codes from the U+00xx range to the PUA range U+F0xx found in the cmap subtables. r=emilio

Comment 23

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/3b719cd12933

Comment 25

[Monica Chiorean]

I was able to reproduce issue on Win10 using build 99.0a1 (20220223190530).

Verified as fixed on Win10 and Mac10.13 using builds 99.0b4(20220315185755) and 100.0a1(20220316214937).
But issue is still reproducing on Ubuntu 20.4 on the same builds 99.0b4(20220315185755) and 100.0a1(20220316214937), can this be fixed on Ubuntu also? On Ubuntu I still see the message 'Following is our agenda'.

Comment 26

[Robert Longson [:longsonr]]

Do you actually have the wingdings font installed on Ubuntu?

Comment 27

[Monica Chiorean]

(In reply to Robert Longson [:longsonr] from comment #26)

Do you actually have the wingdings font installed on Ubuntu?

No I did not know they were required. Thank you.

Comment 29

[Monica Chiorean]

Could not find Wingdings in the MS font package installation for Ubuntu20.4 (https://ostechnix.com/install-microsoft-windows-fonts-ubuntu-16-04/).