1854950 - [LINUX] Broken font rendering in Firefox 118.0 in private mode (with no "native" Linux fonts and only Windows fonts installed)

Status: RESOLVED FIXED

(Core :: Layout: Text and Fonts, defect)

Description

[Artem S. Tashkinov]

Attachment: fonts.webp

Steps to reproduce:

This is a critical issue.

Font rendering in private mode in Linux (Fedora 38 to be precise) in Firefox 118.0 is completely broken. Not a single character is rendered.

This is happening on all websites regardless whether they are using web fonts or not.

To make sure I got things right, I checked the issue in a brand new Firefox profile.

Actual results:

Expected results:

Comment 1

[Artem S. Tashkinov]

Firefox 118.0b1 is also (was already) affected. I've no idea how to check earlier releases.

Comment 2

[Artem S. Tashkinov]

I meant builds, not releases.

The issue does not affect Firefox 117.0.

Comment 3

[BugBot [:suhaib / :marco]]

The Bugbug bot thinks this bug should belong to the 'Core::Layout: Text and Fonts' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 4

[Tom Schuster]

Hi Artem. Thanks for you report. Can you verify if disabling privacy.fingerprintingProtection.pbmode in about:config fixes the issue for you?

Comment 5

[Artem S. Tashkinov]

(In reply to Tom Schuster (MoCo) from comment #4)

Hi Artem. Thanks for you report. Can you verify if disabling privacy.fingerprintingProtection.pbmode in about:config fixes the issue for you?

It does fix the issue, thanks a ton!

I think you really don't want to release Firefox 118 with such a glaring issue.

Comment 6

[BugBot [:suhaib / :marco]]

:timhuang, since you are the author of the regressor, bug 1849903, could you take a look? Also, could you set the severity field?

For more information, please visit BugBot documentation.

Comment 7

[Tim Huang[:timhuang]]

Hi Jonathan, this issue happens for the Fedora system. So, I wonder, do we accidentally block all fonts in Fedora because of font visibility restrictions?

Comment 8

[Jonathan Kew [:jfkthame]]

It kinda looks like it, but I don't know why that would be.... can we confirm whether this affects all Fedora systems, or is it something about the reporter's specific configuration?

I'll see if I can get a Fedora VM set up, and try to repro.....

Comment 9

[Tom Ritter [:tjr]]

We are wondering if it might be the localized name issue being fixed in Bug 1850672.

Artem if you're able to help us more with investigating - could you download this build of Firefox and tell us if it works for you in PBM with privacy.fingerprintingProtection.pbmode enabled?

Also, is your Fedora localized to any particular locale or language? From about:support you could copy/paste the 'Internationalization & Localization' section.

Comment 10

[Artem S. Tashkinov]

(In reply to Tom Ritter [:tjr] from comment #9)

We are wondering if it might be the localized name issue being fixed in Bug 1850672.

Artem if you're able to help us more with investigating - could you download this build of Firefox and tell us if it works for you in PBM with privacy.fingerprintingProtection.pbmode enabled?

This build works fine with this variable set to true.

Also, is your Fedora localized to any particular locale or language? From about:support you could copy/paste the 'Internationalization & Localization' section.

Internationalization & Localization
Application Settings
Requested Locales 	["en-US"]
Available Locales 	["en-US"]
App Locales 	["en-US"]
Regional Preferences 	["en-US"]
Default Locale 	"en-US"
Operating System
System Locales 	["en-US"]
Regional Preferences 	["en-US"]

Comment 11

[Artem S. Tashkinov]

Here's what I've found out.

To trigger the bug you need to delete all default Fedora fonts and install your own fonts (I'm using core Microsoft fonts). Otherwise there's no issue.

I can't imagine too many Linux users having the same configuration, so this bug doesn't seem critical after all.

Comment 12

[Jonathan Kew [:jfkthame]]

Ah! That does make a big difference, yes. Still, perhaps we should try to detect such a situation and disable the anti-font-fingerprinting setting, as it's not going to work out well in such a case.

Comment 13

[Artem S. Tashkinov]

(In reply to Jonathan Kew [:jfkthame] from comment #12)

Ah! That does make a big difference, yes. Still, perhaps we should try to detect such a situation and disable the anti-font-fingerprinting setting, as it's not going to work out well in such a case.

And contrary to what I said in the bug report, web fonts are rendered properly.

Weirdly, the font set I have installed is quite normal albeit for Windows, not Linux.

Comment 14

[Jonathan Kew [:jfkthame]]

(In reply to Artem S. Tashkinov from comment #13)

And contrary to what I said in the bug report, web fonts are rendered properly.

That makes sense, thanks for clarifying.

Weirdly, the font set I have installed is quite normal albeit for Windows, not Linux.

The restricted set of fonts used in anti-fingerprinting mode is different for Windows vs Linux; it's not expected that Linux users will have the Windows font collection. (Which is more than just the MS Core fonts...)

One thing that seems odd to me is that the bug 1850672 build (linked from comment 9 here) apparently avoided the problem. I wonder if some other factor was also different there.

Comment 15

[Jonathan Kew [:jfkthame]]

I have just installed Fedora 38 in a new VM -- could you please confirm what packages I should install and delete in order to replicate your configuration of fonts? I'd like to understand better what's happening there, and whether we can make this more robust.

Comment 16

[Artem S. Tashkinov]

it's not expected that Linux users will have the Windows font collection.

Quite a lot of Linux users actually routinely install Windows fonts, not least because they want to preserve compatibility with MS Office and even Acrobat Reader documents.

Looks like anti-fingerprinting mode disables all fonts specific to Windows, thus the bug.

One thing that seems odd to me is that the bug 1850672 build (linked from comment 9 here) apparently avoided the problem. I wonder if some other factor was also different there.

That's far outside what I could comment upon :-)

Comment 17

[Artem S. Tashkinov]

(In reply to Jonathan Kew [:jfkthame] from comment #15)

I have just installed Fedora 38 in a new VM -- could you please confirm what packages I should install and delete in order to replicate your configuration of fonts? I'd like to understand better what's happening there, and whether we can make this more robust.

I've replied to you privately (check your SPAM folder just in case).

Comment 18

[Jonathan Kew [:jfkthame]]

(In reply to Artem S. Tashkinov from comment #16)

it's not expected that Linux users will have the Windows font collection.

Quite a lot of Linux users actually routinely install Windows fonts,

Some Windows fonts, yes -- I expect that's quite common. All the standard Windows fonts? That would be more surprising to me. And if you don't install the complete collection that ships with Windows, your particular subset becomes a potential fingerprinting signal.

But anyhow, if the user-installed MS fonts are disabled, I'd expect Firefox to fall back to the standard Fedora fonts, so the display still wouldn't be broken in the way your screenshot shows. That must be because you've removed standard fonts, and I'd like to try and handle that situation better.

(In reply to Artem S. Tashkinov from comment #17)

I've replied to you privately (check your SPAM folder just in case).

Thanks for this; received your message, and will look into it here.

Comment 19

[Artem S. Tashkinov]

Some Windows fonts, yes -- I expect that's quite common. All the standard Windows fonts? That would be more surprising to me. And if you don't install the complete collection that ships with Windows, your particular subset becomes a potential fingerprinting signal.

I don't have all the Windows fonts installed (besides they differ between different Windows releases), just a bizarre subset of them. In fact my configuration is a mishmash of certain Windows 98, 2000, 7/10/11 fonts and some are even from the MS Office 2003 distribution.

Would be great if you found a way of using fonts without exposing them to webpages (please forgive me if I'm talking nonsense - I've no idea how it all works).

But anyhow, if the user-installed MS fonts are disabled, I'd expect Firefox to fall back to the standard Fedora fonts, so the display still wouldn't be broken in the way your screenshot shows. That must be because you've removed standard fonts, and I'd like to try and handle that situation better.

The issue with Linux is that there's no such thing as "standard" fonts. Ubuntu, Debian, Fedora, Mint, etc. all offer different sets of fonts. What's more they change from version to version just like with Windows. There's nothing you can really rely on. People may use Firefox starting with such old distros as CentOS 7.0 and ending with something like Arch Linux or Gentoo.

Comment 20

[Frederik Braun [:freddy]]

(In reply to Artem S. Tashkinov from comment #19)

Would be great if you found a way of using fonts without exposing them to webpages (please forgive me if I'm talking nonsense - I've no idea how it all works).

Using and displaying fonts does expose them to web pages. That's the whole crux of the issue.

The issue with Linux is that there's no such thing as "standard" fonts. Ubuntu, Debian, Fedora, Mint, etc. all offer different sets of fonts. What's more they change from version to version just like with Windows. There's nothing you can really rely on. People may use Firefox starting with such old distros as CentOS 7.0 and ending with something like Arch Linux or Gentoo.

From our studies, most people use the default fonts for their distribution. We know that they differ but we our research shows that there's a reasonable set of system fonts that are non-unique per user and can thus prevent fingerprinting.

Lastly, I want to thank you for engaging to positively, debugging, testing, and helping us better understand the issue. It's of great importance to us that this protection does not do more harm than good.

Comment 21

[BugBot [:suhaib / :marco]]

The bug has a release status flag that shows some version of Firefox is affected, thus it will be considered confirmed.

Comment 22

[Artem S. Tashkinov]

(In reply to Frederik Braun [:freddy] from comment #20)

Using and displaying fonts does expose them to web pages. That's the whole crux of the issue.

I don't understand how exactly it all works. Are you saying there's special JS/CSS which can be used to deduce the installed fonts? Is it not possible to display characters with whatever fonts the end user computer has but never allow the web page to actually get to know which fonts are present?

Or are you talking about canvas fingerprinting? If it's only that, then the game is lost completely, as Linux distros use different font antialiasing techniques, DPI settings, etc. and other varying features which make uniquely identifying the user a lost cause.

This page https://browserleaks.com/canvas portrays quite a terrifying picture.

From our studies, most people use the default fonts for their distribution. We know that they differ but we our research shows that there's a reasonable set of system fonts that are non-unique per user and can thus prevent fingerprinting.

It feels to me you make use of some hardcoded fonts which means all the Linux distros must patch Firefox in order for this protection to work. And many users (like me) simply download binary Firefox releases from your servers because I don't want to browse the web knowing there are publicly released 0-day vulnerabilities since my $DISTRO_X is slow to push a Firefox update.

Here's how you can address this issue if what I said above makes no sense. You simply reveal certain fonts which must be standard and present for all Linux distros.

Strangely no such requirement exists for Windows/MacOS/Android systems and AFAIK Android ROMs often have very different font sets. Windows and MacOS users also often install additional fonts but at least Windows doesn't allow you to delete preinstalled system fonts (I need to check it).

I'm getting lost in how it all works and how you want to address the issue. Not that you owe me any explanation, so this comment may be dismissed as noise. I am sorry for interrupting you.

Comment 23

[Jonathan Kew [:jfkthame]]

(In reply to Artem S. Tashkinov from comment #22)

(In reply to Frederik Braun [:freddy] from comment #20)

Using and displaying fonts does expose them to web pages. That's the whole crux of the issue.

I don't understand how exactly it all works. Are you saying there's special JS/CSS which can be used to deduce the installed fonts?

Yes. Not features specifically designed for this purpose, but it's possible for sites to use a variety of existing APIs to achieve this.

Is it not possible to display characters with whatever fonts the end user computer has but never allow the web page to actually get to know which fonts are present?

No. For example, a page could create two <span>s, one using font-family: Times, monospace and the other just using font-family: monospace, containing the same string of text, and then compare their .offsetWidth. If they're different, the page can deduce you have Times available; if they're the same, you haven't. (This is a simplified version; in reality, pages can do various more sophisticated things.)

To completely prevent this, we'd have to block a lot of fundamental APIs that pages have been using for years, and many sites would break.

Comment 24

[BugBot [:suhaib / :marco]]

Set release status flags based on info from the regressing bug 1849903

Comment 25

[Daniel Holbert [:dholbert]]

(In reply to Artem S. Tashkinov from comment #11)

To trigger the bug you need to delete all default Fedora fonts and install your own fonts (I'm using core Microsoft fonts). Otherwise there's no issue.

I can't imagine too many Linux users having the same configuration, so this bug doesn't seem critical after all.

This seems edge-casey, but given the catastrophic outcome here, it seems like we should have some sort of mitigation for this situation.

I wonder if we can add some sort of heuristic to our privacy.fingerprintingProtection.pbmode implementation, to e.g. check if some minimal set of fonts are available, and relax our local-font-availaibility restriction if we can't find sufficient fonts for proper rendering of basic text? (Not sure how we want to define that heuristic, but a strawman proposal might be something like: "Can we find an allowed-for-use system font to render each of the monospace, serif, sans-serif, etc. fallback font-families")

jfkthame, what do you think?

Comment 26

[Jonathan Kew [:jfkthame]]

Yes, I was thinking along those lines. If we determine that enough standard fonts aren't available, we're in effect on an "unknown platform/distro" and we should just disable the restriction.

Comment 27

[Artem S. Tashkinov]

(In reply to Jonathan Kew [:jfkthame] from comment #26)

Yes, I was thinking along those lines. If we determine that enough standard fonts aren't available, we're in effect on an "unknown platform/distro" and we should just disable the restriction.

Firefox used to have notification bars rolling down from the address bar, I wonder if you could add a notification that:

"In Private Mode Firefox needs certain fonts to guard your from websites using font fingerprinting. Please refer to this article [URL] to get more information."

Sounds quite straightforward, easy to implement and is informative enough for the user to sort it all out.

Comment 28

[Jonathan Kew [:jfkthame]]

Attachment: Bug 1854950 - Disable font-fingerprinting protection if hardly any standard distro fonts are present, as it would totally break content rendering. r=#layout

Comment 29

[Daniel Holbert [:dholbert]]

(In reply to Artem S. Tashkinov from comment #27)

Firefox used to have notification bars rolling down from the address bar, I wonder if you could add a notification that:

That's a good idea, but we've got a high bar for showing that category of rolldown-bar / popup-permission-request UI. Users tend to ignore and click through those, and the more-of-those-we-show, the more users ignore/click-through them.

In this case, this is more of a "defense-in-depth" type of privacy protection anyway where the consequences of not-applying-the-protection aren't catastrophic. So it's not the sort of thing that we want to nag the user about, probably.

Comment 30

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/28d3f5afbfe4
Disable font-fingerprinting protection if hardly any standard distro fonts are present, as it would totally break content rendering. r=dholbert,emilio

Comment 31

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/28d3f5afbfe4

Comment 32

[BugBot [:suhaib / :marco]]

The patch landed in nightly and beta is affected.
:jfkthame, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
• If no, please set status-firefox119 to wontfix.

For more information, please visit BugBot documentation.

Comment 33

[Jonathan Kew [:jfkthame]]

It's a pretty obscure edge-case (not many users remove all the standard fonts that shipped with their system!), but given the drastic nature of the failure for anyone affected, and the simple/safe nature of the patch, I think we should take this.

Comment 34

[Jonathan Kew [:jfkthame]]

Comment on attachment 9355365 [details]
Bug 1854950 - Disable font-fingerprinting protection if hardly any standard distro fonts are present, as it would totally break content rendering. r=#layout

Beta/Release Uplift Approval Request

• User impact if declined: Potential broken rendering for users who have removed all standard fonts from their system
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Simple patch that just disables the font-fingerprinting restriction if no standard fonts are present
• String changes made/needed:
• Is Android affected?: No

Comment 35

[Jonathan Kew [:jfkthame]]

Artem, the latest build of Nightly (see https://nightly.mozilla.org/) should include this fix; if you could verify that it resolves the problem on your system, that would be great - thanks!

Comment 36

[Artem S. Tashkinov]

Confirming fixed in the latest nightly.

Comment 37

[Dianna Smith [:diannaS]]

Comment on attachment 9355365 [details]
Bug 1854950 - Disable font-fingerprinting protection if hardly any standard distro fonts are present, as it would totally break content rendering. r=#layout

Approved for 119.0b4

Comment 38

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-beta/rev/79a4e7879638