1862182 - CJK Ext I characters are rendered as boxes despite font available

Status: VERIFIED FIXED

(Core :: Layout: Text and Fonts, defect)

Description

[lilydjwg]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0

Steps to reproduce:

• install jigmo fonts
• visit https://en.wikipedia.org/wiki/CJK_Unified_Ideographs_Extension_I

Actual results:

Those CJK Ext I characters are rendered as boxes, unless I set font-family to Jigmo2 explicitly with devtools.

Expected results:

Firefox should fallback to any available font before rendering characters as boxes.

Google Chrome displays the webpage fine, as well as other GTK apps.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Layout: Text and Fonts' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Masatoshi Kimura [:emk]]

On Windows 10, both Firefox and Chrome fail to display without explicit font-family: jigmo2.

Comment 3

[Masatoshi Kimura [:emk]]

Also failed on Windows 11. Regression range:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=e6fa159738fa2f0b460c3867d0b4d439d64e59cd&tochange=508a0cc2f6d446e4b016ebb6d2c740c80f830dd9

Apparently this change is intentional, but could you please reconsider? We could not use new ideographs until the embedded UCD is updated to the latest Unicode version. Non-default pref is OK (like gfx.font_rendering.fallback.always_use_cmaps).

Comment 4

[Jonathan Kew [:jfkthame]]

Oh, nice find - that certainly explains the behavior.

Adding a pref like gfx.font_rendering.fallback.unassigned_chars seems reasonable.

Comment 5

[Jonathan Kew [:jfkthame]]

Attachment: Bug 1862182 - Create a pref to allow font fallback for unassigned Unicode codepoints. r=#gfx-reviewers

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1656156

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2f522971a074
Create a pref to allow font fallback for unassigned Unicode codepoints. r=gfx-reviewers,lsalzman

Comment 8

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/2f522971a074

Comment 9

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:jfkthame, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
• If no, please set status-firefox120 to wontfix.

For more information, please visit BugBot documentation.

Comment 10

[Masatoshi Kimura [:emk]]

Could you please consider uplifting this to ESR 115 (and optionally beta)? Bug 1859752 is unlikely to uplift. ESR users may not use Ext.I characters until the next summer.

Comment 11

[Jonathan Kew [:jfkthame]]

Comment on attachment 9361385 [details]
Bug 1862182 - Create a pref to allow font fallback for unassigned Unicode codepoints. r=#gfx-reviewers

ESR Uplift Approval Request

• If this is not a sec:{high,crit} bug, please state case for ESR consideration: As noted in comment 10, ESR users will have problems using newly-encoded Unicode characters (e.g. CJK Extension I, but others will also be affected) until a version with the full ICU update appears
• User impact if declined: (see above - failure to render new Unicode characters, despite having a font installed)
• Fix Landed on Version: 121
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Trivial change to add a (default-disabled) pref allowing font fallback to kick in here

Comment 12

[Jonathan Kew [:jfkthame]]

(In reply to Masatoshi Kimura [:emk] from comment #10)

Could you please consider uplifting this to ESR 115 (and optionally beta)? Bug 1859752 is unlikely to uplift. ESR users may not use Ext.I characters until the next summer.

That makes sense to me, and any risk seems extremely minimal; I've nominated it for consideration.

Comment 13

[Jonathan Kew [:jfkthame]]

Comment on attachment 9361385 [details]
Bug 1862182 - Create a pref to allow font fallback for unassigned Unicode codepoints. r=#gfx-reviewers

Beta/Release Uplift Approval Request

• User impact if declined: Recently-assigned Unicode characters (e.g. "CJK Extension I" block) may not be usable even if the user has installed a font that supports them
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: On Windows 10:

(1) install jigmo fonts (see comment 0)

(2) visit https://en.wikipedia.org/wiki/CJK_Unified_Ideographs_Extension_I

• Note that the table of characters shows missing-char blocks

(3) toggle pref gfx.font_rendering.fallback.unassigned_chars to true

• Confirm that the table of chars now shows the proper glyphs

(similar behavior is likely to be seen on other platforms as well)

• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Trivial change to add a (default-disabled) pref allowing font fallback to kick in here
• String changes made/needed:
• Is Android affected?: Yes

Comment 14

[Dianna Smith [:diannaS]]

Comment on attachment 9361385 [details]
Bug 1862182 - Create a pref to allow font fallback for unassigned Unicode codepoints. r=#gfx-reviewers

Approved for 120.0b9

Comment 15

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-beta/rev/8b41a52794e1

Comment 16

[Dianna Smith [:diannaS]]

Comment on attachment 9361385 [details]
Bug 1862182 - Create a pref to allow font fallback for unassigned Unicode codepoints. r=#gfx-reviewers

Approved for 115.5esr

Comment 17

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-esr115/rev/43df4a961cbd

Comment 18

[Hani Yacoub, Desktop QA]

Verified as fixed on Firefox 121.0a1 (2023-11-08) on Windows 10 x64, macOS 11.6 and on Ubuntu 20.04 x64.

Comment 19

[Hani Yacoub, Desktop QA]

Verified as fixed on Firefox 120.0b9 on Windows 10 x64, macOS 11.6 and on Ubuntu 20.04 x64.

Comment 20

[Hani Yacoub, Desktop QA]

Verified as fixed on Firefox 115.5.0esr (treeherder build) on Windows 10 x64, macOS 11.6 and on Ubuntu 20.04 x64.

Comment 21

[Masatoshi Kimura [:emk]]

For the record, gfx.font_rendering.fallback.unassigned_chars is no longer required due to bug 1927706.