2037909 - Perma CRASH on enterprise build | gfx/tests/mochitest/test_font_whitelist.html | gfxFontGroup::GetDefaultFont() "unable to find a usable font (sans-serif)"

Status: RESOLVED FIXED

(Core :: Layout: Text and Fonts, defect)

Description

[Janika Neuberger [:janika]]

Attachment: Test logs | gfx/tests/mochitest/test_font_whitelist.html

On Firefox Enterprise builds we're currently experiencing a perma crash when running gfx/tests/mochitest/test_font_whitelist.html.

Crash viewer:
https://tests.firefox.dev/crash-viewer.html?url=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/XmLaQT2gRYKzEo23MoQsNQ/runs/0/artifacts/public%2Ftest_info%2FD3B889E6-B3AA-4006-BD98-306C7CED6654.json.

All three fallback stages in GetDefaultFont() seem to fail to produce a usable font leading to the crash.

Let me know if there is anything I should provide beside the test logs that I've attached that help to reproduce the failure.

Comment 1

[Ashley Hale [:ahale]]

NeedInfo - any thoughts on what is going on here?

Comment 2

[Jonathan Kew [:jfkthame]]

Can you run it with MOZ_LOG=fontlist:5 and see what that logs?

What's different about a "Firefox Enterprise" build compared to a standard build of mozilla-central?

Comment 4

[Janika Neuberger [:janika]]

Attachment: MOZ_LOG=fontlist:5 logs

These are the logs running with MOZ_LOG=fontlist:5

Comment 5

[Janika Neuberger [:janika]]

What's different about a "Firefox Enterprise" build compared to a standard build of mozilla-central?

A lot. But it's difficult for me to know and point out what could effect gfx/tests/mochitest/test_font_whitelist.html. One thing that came to my mind is that in comparison to m-c we currently need to spin the event loop two additional times during the startup. I thought this changes timing and might lead to the test failure, but the test still fails when I remove the additional two event spinnings.

It might be more useful if you build Firefox Enterprise and reproduce the test failure?

Firefox Enterprise source code: https://github.com/mozilla/enterprise-firefox

Build from the branch enterprise-main with ac_add_options --enable-enterprise and run the test with MOZ_BYPASS_FELT=1 ./mach test gfx/tests/mochitest/test_font_whitelist.html

Comment 6

[vlopezgarcia]

Can you check if this could be the issue?

https://searchfox.org/firefox-main/rev/10725540f638ba15d8138079dab01ecbac808be5/gfx/thebes/gfxPlatformFontList.cpp#2685

I added this

    if (!SharedFontList()->Families()->IsInitialized()) {
      MOZ_CRASH("SharedFontList fallback family is uninitialized");
    }

And got

USER/*/Firefox Enterprise.app/Contents/MacOS/firefox

Thread 0 Crashed:: MainThread Dispatch queue: com.apple.main-thread
0   XUL                           	       0x142f69a34 gfxPlatformFontList::GetDefaultFontLocked(FontVisibilityProvider*, gfxFontStyle const*) + 528
1   XUL                           	       0x142f65364 gfxPlatformFontList::InitFontList() + 1356
2   XUL                           	       0x142f65f14 gfxPlatformFontList::FontWhitelistPrefChanged(char const*, void*) + 28
3   XUL                           	       0x141e99f38 NotifyCallbacks(nsTString<char> const&, PrefWrapper const*) + 112
4   XUL                           	       0x141eeead0 pref_SetPref(nsTString<char> const&, mozilla::PrefType, mozilla::PrefValueKind, PrefValue, bool, bool, bool) + 1440
5   XUL                           	       0x141e96010 mozilla::Preferences::SetCString(char const*, nsTSubstring<char> const&, mozilla::PrefValueKind) + 212

So it looks then
https://searchfox.org/firefox-main/rev/10725540f638ba15d8138079dab01ecbac808be5/gfx/thebes/gfxPlatformFontList.cpp#838
https://searchfox.org/firefox-main/rev/10725540f638ba15d8138079dab01ecbac808be5/gfx/thebes/gfxPlatformFontList.cpp#842

You might get a nullptr? Not sure if that is the reason, or something else is hidden in here

Comment 7

[Jonathan Kew [:jfkthame]]

The crash is occurring because modifying the whitelist pref causes the (available) font list to be reconstructed, and for some reason in the enterprise build, the very first attempt to use a font after that update is coming from stylo rather than the main thread. That seems to be different timing than we see in a standard Firefox build, and is resulting in the issue here.

Unfortunately when this happens, none of the font family records have yet been fully initialized -- this is done on first access for each family -- and because we're not on the main thread, the InitializeFamily call can't happen synchronously but gets deferred to a main-thread runnable. But if no font whatsoever has been initialized since the font-list rebuild, then even our last-ditch attempt to find a default is doomed.

Probably the easiest way to work around this is to eagerly call InitializeFamily() on the first record in the font list, right when it is constructed (which happens on the main thread). That way there should always be at least a default fallback that can be used even if stylo (or potentially an offscreen-canvas worker thread) wants it before any other main-thread font setup has happened.

I suspect it's possible for a "normal" Firefox to hit this issue as well; it just isn't happening in exactly the same scenario. We've seen occasional crash reports with a similar error, but never had a reproducible case. So the fix here should be beneficial for all builds.

Comment 8

[Jonathan Kew [:jfkthame]]

Attachment: Bug 2037909 - Check the default font's Family is initialized before trying to look up the face, to ensure we have a usable default font. r=#layout

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/d9d04b3c6813
https://hg.mozilla.org/integration/autoland/rev/6b4466630957
Eagerly initialize the first Family record in the font list, to ensure at least one family is usable as a default. r=layout-reviewers,emilio

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/f28f2403ab12
https://hg.mozilla.org/integration/autoland/rev/7661a385718d
Revert "Bug 2037909 - Eagerly initialize the first Family record in the font list, to ensure at least one family is usable as a default. r=layout-reviewers,emilio" for causing bc failures @ browser_taskbarTabs_chromeTest

Comment 11

[amarc]

Backed out for causing bc failures

• Backout link
• Push with failures
• Failure Log

Comment 12

[Intermittent Failures Robot]

1 failures were associated with this bug in the last 7 days.

Repository breakdown:

• enterprise-main: 1

Table

	no_variant
macosx1500-x86_64/opt	1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=2037909&startday=2026-05-04&endday=2026-05-10&tree=all

Comment 13

[BugBot [:suhaib / :marco/ :calixte]]

There is an r+ patch which didn't land and no activity in this bug for 1 week.
:jfkthame, could you have a look please?
If you still have some work to do, you can add an action "Plan Changes" in Phabricator.
For more information, please visit BugBot documentation.

Comment 14

[Intermittent Failures Robot]

1 failures were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Table

	no_variant
android-x86_64/opt	1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=2037909&startday=2026-05-25&endday=2026-05-31&tree=all

Comment 15

[Jonathan Kew [:jfkthame]]

It's not really clear to me why the patch here lead to bc failures (looks like those tests may be quite timing-sensitive), but I think a better version of the patch may be to ensure the default font's family (rather than the first in the list) is initialized by InitFontList. In my local testing, at least, this seems to avoid triggering the bc failures we saw on the previous landing attempt.

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/a76e3c8c520c
https://hg.mozilla.org/integration/autoland/rev/d170a31aac29
Check the default font's Family is initialized before trying to look up the face, to ensure we have a usable default font. r=layout-reviewers,emilio

Comment 17

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/d170a31aac29

Comment 18

[Jonathan Kew [:jfkthame]]

Janika, could you confirm whether the issue with Enterprise builds is resolved with the patch that landed here? Thanks!

We have some other longstanding reports (e.g. see bug 1322437, bug 2031855) that may turn out to be this same issue; if the associated crashes seem to stop happening, we may be able to dupe them here too.

Comment 19

[Janika Neuberger [:janika]]

On the positive side, the test is no longer failing in our enterprise CI. However, the failure has not occurred in our CI since we merged firefox-main at commit 15541e0 into the enterprise codebase on May 12. I can't identify which merged change affected the test result.

To verify whether the patch would have fixed the failure, I took a build that still reproduced the issue and cherry-picked commit a76e3c8c520c onto it. In that setup, I can confirm that the patch resolves the test failure.

Comment 20

[Jonathan Kew [:jfkthame]]

Thanks for the confirmation, that's good to know. Hopefully with this patch also in place, things are more robust and this issue is less likely to find a way to resurface.

Comment 21

[Ryan VanderMeulen (PTO, back July 27)]

Please nominate this for ESR140 and ESR115 uplift when you get a chance. We're seeing non-trivial crash volume with this signature there as well.

Comment 22

[Phabricator Automation]

firefox-esr115 Uplift Approval Request

• User impact if declined/Reason for urgency: Possible crash ("unable to find a usable font") on startup or when font list modified during the session
• Code covered by automated testing?: no
• Fix verified in Nightly?: no
• Needs manual QE testing?: no
• Steps to reproduce for manual QE testing:
• Risk associated with taking this patch: low
• Explanation of risk level: Simple addition of explicit early initialization of the default font
• String changes made/needed?: none
• Is Android affected?: yes

Comment 23

[Jonathan Kew [:jfkthame]]

Attachment: Bug 2037909 - Check the default font's Family is initialized before trying to look up the face, to ensure we have a usable default font.

Original Revision: https://phabricator.services.mozilla.com/D299492

Comment 24

[Phabricator Automation]

firefox-esr140 Uplift Approval Request

• User impact if declined/Reason for urgency: Possible crash ("unable to find a usable font") on startup or when font list modified during the session
• Code covered by automated testing?: no
• Fix verified in Nightly?: no
• Needs manual QE testing?: no
• Steps to reproduce for manual QE testing:
• Risk associated with taking this patch: low
• Explanation of risk level: Simple addition of explicit early initialization of the default font
• String changes made/needed?: none
• Is Android affected?: yes

Comment 25

[Jonathan Kew [:jfkthame]]

Attachment: Bug 2037909 - Check the default font's Family is initialized before trying to look up the face, to ensure we have a usable default font.

Original Revision: https://phabricator.services.mozilla.com/D299492

Comment 26

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-esr115/rev/608470f1bbb7

Comment 27

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-esr140/rev/6eb3ae276fe2