1787551 - Crash in [@ gfxFT2FontList::FindFonts]

Status: RESOLVED FIXED

(Core :: Graphics: Text, defect, P1)

Description

[Chris Peterson [:cpeterson]]

Crash report: https://crash-stats.mozilla.org/report/index/e769cfb1-341f-421d-bb63-349e90220826

Reason: SIGSEGV / SEGV_MAPERR

Top 10 frames of crashing thread:

0 libandroid.so libandroid.so@0x000000000001eeac 
1 libandroid.so libandroid.so@0x000000000001eea4 
2 libandroid.so libandroid.so@0x000000000001d5b0 
3 libandroid.so libandroid.so@0x000000000001d444 
4 libxul.so gfxFT2FontList::FindFonts gfx/thebes/gfxFT2FontList.cpp:1460
5 libxul.so gfxPlatformFontList::InitFontList gfx/thebes/gfxPlatformFontList.cpp:576
6 libxul.so gfxPlatformFontList::Initialize gfx/thebes/gfxPlatformFontList.cpp:259
7 libxul.so gfxPlatform::Init gfx/thebes/gfxPlatform.cpp:979
8 libxul.so gfxPlatform::GetPlatform gfx/thebes/gfxPlatform.cpp:460
9 libxul.so gfxPlatform::InitializeCMS gfx/thebes/gfxPlatform.cpp:2111

Comment 1

[Chris Peterson [:cpeterson]]

Jonathan, do you know why this Android font crash might have spiked starting around September 1?

We're crashing inside libandroid.so's ASystemFontIterator_open here:

https://searchfox.org/mozilla-central/rev/3c194fa1d6f339036d2ec9516bd310c6ad612859/gfx/thebes/gfxFT2FontList.cpp#1459

I looked at the mozilla-central push log between August 80 - September 1, but I didn't see anything that obviously looks related to Android fonts.

https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=4c76664026b55d57999e109b5bc5429d986df9ab&tochange=136f13b75057ef4c2419dba0d63e913d502c7458

Over 91% of the crash reports are from Android 12 (API 31), even though only Android 12 has only a 19% share of among all Fenix users. Android 12 was released way back in October 2021, but maybe this is a regression in a September 2022 patch release to Android 12? But none of the fixes in Android's September security bulletin look related to fonts, from what I can tell:

https://source.android.com/docs/security/bulletin/2022-09-01

Comment 2

[Jonathan Kew [:jfkthame]]

Sorry, I don't have any ideas here.... nothing in the history stands out to me as particularly suspicious. I do notice that Ryan landed some Android-related updates (JDK version, Kotlin version) on Aug 31st, but don't know any reason that would make ASystemFontIterator_open start crashing under us.

cc'ing Makoto for any thoughts he may have -- though to me this looks more like some kind of Android bug.

Comment 3

[Pascal Chevrel:pascalc]

We are in 107 RC week, wontfix 106.

Comment 4

[Makoto Kato [:m_kato] (fever, slow response)]

I think that this depends on device vendor. Most crashes are OPPO (ColorOS 12), realme and OnePlus. This Font API will require to enumerate custom fallback fonts (https://source.android.com/docs/core/fonts/custom-font-fallback). But we might have to disable this on these vendors device.

Comment 5

[Chris Peterson [:cpeterson]]

I think that this depends on device vendor. Most crashes are OPPO (ColorOS 12), realme and OnePlus.

Good catch! I wonder why these devices all broke at the same time.

Comment 6

[Jonathan Kew [:jfkthame]]

My guess is that there was an update of ColorOS that decided to add some kind of extra fonts for their "branding" or "look-and-feel", and did something a bit wrong (a bad font file, or a garbled configuration of some kind). If we could get a copy of the installed fonts from an affected device, we might be able to identify the culprit.

Comment 7

[Jonathan Kew [:jfkthame]]

Interestingly, https://www.oppo.com/en/coloros13/ talks about new "Eye-friendly fonts" as one of the features of the new version. Maybe eye-friendly, but Fenix-hostile?!

Comment 8

[Chris Peterson [:cpeterson]]

Makoto, do you need those devices to test the crash or fonts? Can we prevent Fenix from enumerating ColorOS's bad fonts?

Tracking this bug for Fenix 109

Comment 9

[Makoto Kato [:m_kato] (fever, slow response)]

(In reply to Chris Peterson [:cpeterson] from comment #8)

Makoto, do you need those devices to test the crash or fonts? Can we prevent Fenix from enumerating ColorOS's bad fonts?

Although I don't know what is trigger of this crash, if QA has OPPO device with ColorOS 12, could they test whether Fenix crash? I guess that this may occur if OPPO provides font update by their updater, not OS update. (Android 12 can update internal fonts without OS update, and font match API can support it)

Comment 10

[Makoto Kato [:m_kato] (fever, slow response)]

Attachment: Bug 1787551 - Disable Android's Font Match API on OPPO, realme and OnePlus. r=jfkthame

According to crash data, some devices by OPPO and realme and OnePlus seem to
crash when using Font Match API. I would like to disable it until root cause
finds.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/38c6f4ac8433
Disable Android's Font Match API on OPPO, realme and OnePlus. r=jfkthame

Comment 12

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/38c6f4ac8433

Comment 13

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:m_kato, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
• If no, please set status-firefox108 to wontfix.

For more information, please visit auto_nag documentation.

Comment 14

[Makoto Kato [:m_kato] (fever, slow response)]

Comment on attachment 9303642 [details]
Bug 1787551 - Disable Android's Font Match API on OPPO, realme and OnePlus. r=jfkthame

Beta/Release Uplift Approval Request

• User impact if declined: Startup crash in Fenix when using OPPO, realme and OnePlus device with Android 11+.
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Don't use newer Font Match API on OPPO, realme and OnePlus device even if Android 10+.
  Although newer Android 12+ can use system font update by OEM/Play store, not system update, this feature will be disabled on these devices.
• String changes made/needed:
• Is Android affected?: Yes

Comment 15

[Dianna Smith [:diannaS]]

Comment on attachment 9303642 [details]
Bug 1787551 - Disable Android's Font Match API on OPPO, realme and OnePlus. r=jfkthame

Approved for 108.0b5

Comment 16

[Dianna Smith [:diannaS]]

https://hg.mozilla.org/releases/mozilla-beta/rev/865857977bab

Comment 17

[Donal Meehan [:dmeehan]]

:m_kato could you consider adding a release uplift request on this?
We could consider it for the planned dot release

Comment 18

[Makoto Kato [:m_kato] (fever, slow response)]

Comment on attachment 9303642 [details]
Bug 1787551 - Disable Android's Font Match API on OPPO, realme and OnePlus. r=jfkthame

Beta/Release Uplift Approval Request

• User impact if declined: Startup crash when using OPPO, realme or OnePlus device with Android 11+.
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Don't use newer Font Match API on OPPO, realme and OnePlus device even if Android 10+.
  Although newer Android 12+ can use system font update by OEM/Play store, not system update, this feature will be disabled on these devices.
• String changes made/needed:
• Is Android affected?: Yes

Comment 19

[Donal Meehan [:dmeehan]]

Comment on attachment 9303642 [details]
Bug 1787551 - Disable Android's Font Match API on OPPO, realme and OnePlus. r=jfkthame

Approved for 107.0.1

Comment 20

[Donal Meehan [:dmeehan]]

https://hg.mozilla.org/releases/mozilla-release/rev/a81e7a38bde7