1946625 - Many Bramic scripts are not rendered properly in Firefox on Samsung S10 with Android 12, when font visibility protections are active

Status: UNCONFIRMED

(Core :: Layout: Text and Fonts, defect)

Description

[chentz]

Attachment: broken_scripts_in_page_4_to_6.PDF

Steps to reproduce:

Go to a webpage with Brahmic scripts, e.g. https://en.m.wiktionary.org/wiki/India#Translations_2 and unfold "the country". This one has the translation of India to many languages using Brahmic scripts.

Actual results:

Notably all major scripts in India like Devanagari, Tamil, Telugu can not be rendered. Boxes are shown instead. However, Odia and Sinhala scripts are shown fine. Outside India, Tibetan script cannot be rendered either. However, SEA scripts like Thai, Burmese are fine. See attached pdf file pages 4-6 for the results.

Moreover, if I copy these boxes to the search bar, the actual content can be shown. The tab view can also show the title of the webpage correctly in Bramic scripts. And if I change the locale in the settings, the UI of Fennec app can also properly display the scripts.

To further debug, I went to the about:config and search for fonts. Unlike western or CJK scripts, the value of font.default of Bramic scripts is serif rather than san-serif. The default value of font.name-list of these Bramic scripts are just Noto Sans or Noto Serif. I tried to switch between serif and san-serif but got no luck.

The issue happens both at Google Play version of Firefox Android and also FDroid version of Fennec.

Expected results:

The Brahmic scripts are properly rendered, like the web browser or the older version of Firefox Android.

Comment 1

[[:owlish] 🦉 PST]

It seems like the problem is related to fonts.

Comment 2

[Daniel Holbert [:dholbert]]

Hmm, so far I'm not able to reproduce the issue (on a Pixel 9 Pro XL and a Pixel 6a, each running Android 15). In my case, the Tamil and Telugu glyphs are successfully rendered using Noto Sans Tamil and Noto Sans Telugu according to Firefox devtools.

I wonder if jfkthame has any ideas?

Comment 3

[Jonathan Kew [:jfkthame]]

I'm guessing that the reporter's device does not have the standard Noto fonts for these scripts/languages, but some other set of fonts instead; and they're being blocked by the font visibility restrictions that apply to content (for anti-fingerprinting). That would account for the scripts displaying OK in the UI, but not in web content.

@chentz, what kind of Android device are you using? It's possible that the manufacturer/distributor has modified the font setup from the defaults that Google sets (e.g. on Pixel devices).

Comment 4

[chentz]

Thanks Jonathan and Daniel and others for help.

I am using a pretty old device Samsung S10 with Android 12.

Re Fingerprinting: I toggled back and forth privacy.fingerprintingProtection and privacy.resistFingerprinting but still the Brahmic letters cannot be shown.

I also discovered that the scripts can properly be rendered under reader mode as well. I am happy to provide more information to debug. How can I know which fonts I installed?

Comment 5

[chentz]

To add, probably many other fonts are messed up. With privacy.fingerprintingProtection and privacy.resistFingerprinting on, I am no more able to use fonts in the CSS. And in that condition, CJK fonts fall back to serif despite the font-family in the CSS sans-serif, which could mean that the san-serif version of CJK fonts is also off....

Comment 6

[chentz]

I also did a binary search using the Firefox from APKPure. The last good version for me is 133.0.3(2016060959) and the first culprit version is 134.0(2016064983). Wondering what has been changed.

Comment 7

[Daniel Holbert [:dholbert]]

Unfortunately 133 vs 134 is a pretty massive window (4 weeks' worth of changes).

If you're willing, you can narrow that to a one-day window (or maybe better) using the command-line mozregression tool, which helps you bisect our Nightly builds:
https://mozilla.github.io/mozregression/

You can install it via python's pip or pipx package manager, and then use it to install and launch a Nightly build (on your USB-connected phone, assuming you've got adb configured) like so:

mozregression -n fenix  --launch 2024-10-01

It might nudge you to specify --arch -- if so, you might want e.g. --arch armeabi-v7a or --arch arm64-v8a (or you can try the other arch options from mozregression --help and fine one that runs on your phone).

Once you've got known good / bad builds, you can bisect like so (I'm using dates for the start of the 133.0 nightly cycle and end of the 134.0 nightly cycle as examples - those might be accurate good/bad builds or might not be):

mozregression -n fenix  --good 2024-10-01 --bad 2024-11-26 --arch [whatever arch option works for your phone]

Comment 8

[chentz]

Thanks for the tips to debug. Here are the results:

4:33.18 INFO: Newest known good nightly: 2024-11-13
4:33.18 INFO: Oldest known bad nightly: 2024-11-14

I checked the diff list in mozilla central but I did not have any clues. https://hg.mozilla.org/mozilla-central/pushloghtml?startdate=2024-11-13&enddate=2024-11-14

BTW, in nightly, the error only happens in incognito mode, not in regular mode. Probably that was why Daniel was not able to reproduce the bug.

Comment 9

[Daniel Holbert [:dholbert]]

BTW, in nightly, the error only happens in incognito mode, not in regular mode. Probably that was why Daniel was not able to reproduce the bug.

I tested in private browsing mode as well, and I'm still not seeing the bug, i.e. the fonts that the page needs are all available in my "stock" set of system fonts. But see below...

I checked the diff list in mozilla central but I did not have any clues. https://hg.mozilla.org/mozilla-central/pushloghtml?startdate=2024-11-13&enddate=2024-11-14

Thanks for doing that bisection! The mozilla-central datestamps don't exactly match up to the Nightly datestamps, for various timezone and arbitrary-build-time reasons (but they're within ~24 hours either direction of matching up). So I broadened the mozilla-central pushlog slightly and I found this which I think is the relevant change: "Bug 1928705: Enable Font Visibility Restrictions in FPP"

That looks like it's the commit that enabled the feature that Jonathan was referencing in comment 3.

It's correct/expected that that commit will cause some fonts to stop working in private browsing mode, but they should still work in regular browsing mode. And based on this bug report, I think you're saying your fonts are broken in regular mode, right? (in your regular Firefox installation) Bug 1928705 mentioned that it intended to affect "PBMode and users who have configured ETP Custom" -- perhaps your regular Firefox installation is in that second category. Have you opted in to higher ETP (Enhanced Tracking Protection) in the Firefox settings? (Check the "Privacy and Security" section in Firefox settings -- is that set to Strict or Custom? If so, does the bug go away if you set it to Standard?)

Comment 10

[Daniel Holbert [:dholbert]]

(In reply to Daniel Holbert [:dholbert] from comment #9)

It's correct/expected that that commit will cause some fonts to stop working in private browsing mode

Elaborating on this^ a bit -- the this is a tradeoff between usability and privacy. Essentially, if you've installed a handful of custom fonts, then that makes your device uniquely or near-uniquely-identifiable to websites, which defeats some of the purpose of Private Browsing Mode and allows you to be tracked. This is because there are ways for websites to enumerate your installed fonts. So, we mitigate this sort of tracking specifically in modes where the user is indicating a heightened privacy/anti-tracking interest by limiting the available local fonts, and only allowing sites to use those that are installed by default on ~all devices. For Android, this is the list that we use:
https://searchfox.org/mozilla-central/rev/f256e50e068136275c1a2aff827aeddc92a75e07/gfx/thebes/StandardFonts-android.inc

I don't know offhand why some of your Noto fonts the fonts that your device uses to render this content (which are presumably "stock" fonts and not ones that you manually installed?) aren't getting recognized as "stock" fonts, but I guess they're not (maybe jfkthame has ideas about that).
[EDITED to remove mention of Noto since I don't know for sure whether Noto fonts are what your device is using for these scripts or not.]

If you want to disable these font restrictions entirely, it's possible via about:config but I don't recall the details exactly, but it might be just the settings listed in https://bugzilla.mozilla.org/show_bug.cgi?id=1881993#c49 (though I think those might also activate additional protections while turning off the font limitations).

Comment 11

[chentz]

Thanks Daniel for the explanation. That makes sense to me.

It's correct/expected that that commit will cause some fonts to stop working in private browsing mode, but they should still work in regular browsing mode. And based on this bug report, I think you're saying your fonts are broken in regular mode, right? (in your regular Firefox installation) Bug 1928705 mentioned that it intended to affect "PBMode and users who have configured ETP Custom" -- perhaps your regular Firefox installation is in that second category. Have you opted in to higher ETP (Enhanced Tracking Protection) in the Firefox settings? (Check the "Privacy and Security" section in Firefox settings -- is that set to Strict or Custom? If so, does the bug go away if you set it to Standard?)

I had ETP on. If I turned that off, I will not see the issue in regular mode.

If you want to disable these font restrictions entirely, it's possible via about:config but I don't recall the details exactly, but it might be just the settings listed in https://bugzilla.mozilla.org/show_bug.cgi?id=1881993#c49 (though I think those might also activate additional protections while turning off the font limitations).

Seems the following setting can bring the font back in regular mode with ETP on:

privacy.resistFingerpritning = false
privacy.fingerprintingProtection = true
privacy.fingerprintingProtection.overrides = +AllTargets,-FontVisibilityLangPack

Can you help explain or point to the docs what these 2 flags privacy.resistFingerpritning and privacy.fingerprintingProtection do? Thanks.

I don't know offhand why some of your Noto fonts (which are presumably "stock" fonts and not ones that you manually installed?) aren't getting recognized as "stock" fonts, but I guess they're not (maybe jfkthame has ideas about that).

I don't have any new fonts installed on my device and I am using the default one. No idea why that is broken. Hope jfkthame has some ideas. Thanks.

Comment 12

[BugBot [:suhaib / :marco/ :calixte]]

:tjr, since you are the author of the regressor, bug 1928705, could you take a look? Also, could you set the severity field?

For more information, please visit BugBot documentation.

Comment 13

[Daniel Holbert [:dholbert]]

(In reply to chentz from comment #11)

Thanks Daniel for the explanation. That makes sense to me.

It's correct/expected that that commit will cause some fonts to stop working in private browsing mode, but they should still work in regular browsing mode. And based on this bug report, I think you're saying your fonts are broken in regular mode, right? (in your regular Firefox installation) Bug 1928705 mentioned that it intended to affect "PBMode and users who have configured ETP Custom" -- perhaps your regular Firefox installation is in that second category. Have you opted in to higher ETP (Enhanced Tracking Protection) in the Firefox settings? (Check the "Privacy and Security" section in Firefox settings -- is that set to Strict or Custom? If so, does the bug go away if you set it to Standard?)

I had ETP on. If I turned that off, I will not see the issue in regular mode.

OK, cool. (I assume by "on" you meant "set to a non-default value". There isn't exactly an "off"; the default is "Standard", and the expectation is that things should work there.)

If I'm understanding you correctly there, then I think things are mostly working properly in terms of when-font-restrictions-are-applied.

The remaining mystery is why these Noto fonts are being misinterpreted as being outside of the standard set. I'll adjust the bug title to make that clearer.

Seems the following setting can bring the font back in regular mode with ETP on:

Great!

Can you help explain or point to the docs what these 2 flags privacy.resistFingerpritning and privacy.fingerprintingProtection do? Thanks.

I get them confused, but I think roughly:

• privacy.resistFingerprinting is a bucket of fairly-severe restrictions intended for users (and downstream browsers like TorBrowser) that are willing to sacrifice usability and break some sites in order to maximize privacy. It defaults to false in Firefox because it causes a fair amount of breakage.
• privacy.fingerprintingProtection is a bucket of more moderate interventions that take steps to reduce fingerprinting while preserving the usability of the vast majority of sites. These are still off-by-default since they're intentionally making web APIs lie in various ways, but they automatically turn on in various configs where the users have indicated that privacy/preventing-tracking is a priority (e.g. private browsing, non-default ETP levels).

I don't have any new fonts installed on my device and I am using the default one. No idea why that is broken. Hope jfkthame has some ideas.

I would guess maybe the fonts identify themselves by a different name on your Android version (something not on the list at https://searchfox.org/mozilla-central/rev/f256e50e068136275c1a2aff827aeddc92a75e07/gfx/thebes/StandardFonts-android.inc ), or something along those lines, but I'm just speculating.

Tom might have ideas too.

Comment 14

[chentz]

Thanks Daniel.

privacy.resistFingerpritning = false
privacy.fingerprintingProtection = true
privacy.fingerprintingProtection.overrides = +AllTargets,-FontVisibilityLangPack

Update: The right one that I am using is +AllTargets,-FontVisibilityLangPack,-FontVisibilityBaseSystem

OK, cool. (I assume by "on" you meant "set to a non-default value". There isn't exactly an "off"; the default is "Standard", and the expectation is that things should work there.)

There is a switch now. After turning it on we can choose standard, strict etc. https://support.mozilla.org/en-US/kb/enhanced-tracking-protection-firefox-android

If I'm understanding you correctly there, then I think things are mostly working properly in terms of when-font-restrictions-are-applied.

The remaining mystery is why these Noto fonts are being misinterpreted as being outside of the standard set. I'll adjust the bug title to make that clearer.

Thanks for updating the title. It is much more clear now.

I get them confused, but I think roughly:

privacy.resistFingerprinting is a bucket of fairly-severe restrictions intended for users (and downstream browsers like TorBrowser) that are willing to sacrifice usability and break some sites in order to maximize privacy. It defaults to false in Firefox because it causes a fair amount of breakage.
privacy.fingerprintingProtection is a bucket of more moderate interventions that take steps to reduce fingerprinting while preserving the usability of the vast majority of sites.

Thanks for the explanation!

I would guess maybe the fonts identify themselves by a different name on your Android version (something not on the list at https://searchfox.org/mozilla-central/rev/f256e50e068136275c1a2aff827aeddc92a75e07/gfx/thebes/StandardFonts-android.inc ), or something along those lines, but I'm just speculating.

This sounds plausible. Wondering how I can know what font firefox gets on my phone. Even for CJK, the san-serif font seems also missing but the serif is there.

Comment 15

[Jonathan Kew [:jfkthame]]

What would be really helpful is to have a list of the fonts that are actually present on your device; my guess is that Samsung has extensively customized the collection of fonts provided, and therefore we're not recognizing them as being "standard".

I see there's a "Font List" app available on the Play Store (https://play.google.com/store/apps/details?id=dev.cockscomb.FontList); are you able to use that, or some similar utility, to find out the names of the installed fonts?

Alternatively, if you can connect the device to a desktop system with Android Studio installed (or just the adb tool, which may be available by itself without a full Android Studio installation), it should be possible to use a command like adb shell ls /system/fonts to get a list of the installed font files.

If you're able to determine the installed fonts in some such way, please let us know what they are; not having an old Samsung S10 (or similar) on hand here, it's hard for me to directly confirm exactly what's going on. I think I recall some mention of fonts like "Samsung Devanagari", "Samsung Bengali", etc.; if the device has something like that, rather than Noto fonts, I think that could explain the issue here.

Comment 16

[Tom Ritter [:tjr]]

Let me clean up my font listing app today, I think it will be a good debugging tool for us with more detailed information than the above approaches.

Comment 17

[Tom Ritter [:tjr]]

You've been incredibly helpful and willing to help us debug this and I wanted to mention that we really appreciate the effort you're putting in. If you're willing, this app will dump all your current font information to a zip file that you can upload to Bugzilla (or if that's annoying to do from your phone, you can email it to me at tom at mozilla dot com). That would definitely tell us for certain and in detail what is on your phone.

At one point we had a large dataset of the common fonts installed on all Android phones, but that dataset reached its privacy expiration date and has been wiped, so we unfortunately can't compare you to a global baseline.

Comment 18

[chentz]

Attachment: fontlist

Thanks Jonathan and Tom for help.

Somehow I was not able to install the app that Tom provided. The error message is "App not installed as package appears to be invalid". Probably because my phone is too old :(.

I attach the font list that I got from adb.

Comment 19

[chentz]

I examined the font list. Noto fonts of most Brahmic scripts are there, with a suffix VF. What does that mean?

There does not seem to be a pattern.

It does not depend on whether the font name has the VF suffix.
e.g. There are NotoSansDevanagari-VF.ttf, NotoSansSinhala-VF.ttf. Devanagari cannot be rendered, Sinhala can.
Gujarati and Oriya do not have VF suffix. Gujarati cannot be rendered, Oriya can.

It does not depend on whether there is SEC support for the script.
e.g. There are SECDevanagari-Regular.otf, SECGujarati-Regular.ttf, SECLao-Regular.ttf
Devanagari cannot be rendered, Lao can.

Comment 20

[chentz]

Attachment: font_details.txt

Managed to hack Tom's java file to analyse the fonts under /system/fonts. Output is uploaded. It is in plain text and should contain all the necessary info.

I don't think I installed other third party fonts. So probably android.graphics.fonts.SystemFonts.getAvailableFonts() will not give anything.

Hope that helps.

Comment 21

[Jonathan Kew [:jfkthame]]

Thank you for all your efforts to help investigate this. The font list looks.... well, pretty reasonable really. All the "expected" fonts seem to be present, plus some extras that Samsung has decided to add (but they shouldn't cause things to break!)

As of now, I'm quite puzzled, and not sure how to get to the bottom of this. I'll see if I can think of more lines of investigation.....