582987 - CJK fallback strategy causes bad rendering on Mac with zh-TW locale

Status: RESOLVED WORKSFORME

(Core :: Internationalization, defect)

Description

[Hsiao-Ting Yu (Littlebtc)]

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; zh-TW; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8
Build Identifier: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; zh-TW; rv:1.9.2) Gecko/20100115 Firefox/3.6

Background:

* LiHei Pro is a 2003 font, and was the default font for Traditional Chinese locale (zh-TW) of MacOS X 10.5. Only Traditional Chinese characters were included.

* Since MacOS X 10.6, a brand new Unicode font, STHeiti, with Traditional Chinese (zh-TW) and Simplified Chinese (zh-CN) variations, was set to be the default font of Chinese system.
But a lot of Mac users dones't like the change of the typeface, and there is even a utility to fix the default font back (see http://zonble.github.com/tcfail/en.html ).
So in Firefox, the default sans-serif font for zh-TW was still set to LiHei Pro.

The Real Problem:
The text rendering on zh-TW Firefox is bad due to the CJK fallback stragegy.

For content which locale is not set to zh-TW, when the text rendering system encounters a character that is not in LiHei Pro, it will try to render it in Simplified Chinese default font (STHeiti for most cases) or Japanese default font. And then the font will be used for any below content. (Is this because other fonts are contained with Traditional Chinese Characters, and the system won't want to do fallback again?) It doesn't happen when locale was set to zh-TW.

The strange fallback stragegy also makes UI fonts on Firefox inconsistent -- Some menu item will be displayed in STHeiti randomly for unknown reason, which is annonying.

Will changing the fallback for lang not equal to "zh-TW" to be similar to lang being "zh-tw" fix the problem? Or is there any better way?

Reproducible: Always

Steps to Reproduce:
1. Watch a page with HTML lang is not equal to zh-TW and with language-mixed CJK sentence, e.g. the attached twitter URL.
Actual Results:  
The typeface was changed to STHeiti (which is the default font for Simplified Chinese) right after encountering Simplified Chinese Characters, and applied to all below content (LiHei Pro won't be used anymore).

Expected Results:  
When encountering Simplified Chinese Characters, it will displayed as STHeiti, but the below contents should be back to rendering by LiHei Pro.

"Change default zh-TW font to STHeiti" seems to be a good way to fix the problem, but it may have strong impact on user experience, and doesn't fix the real fallback issue.

This problem is not significant on other OSes, since Unicode fonts with rich CJK set are set to be default on Windows (2000 and higher) and popular distribution of Linux.

Comment 1

[Hsiao-Ting Yu (Littlebtc)]

Notes: Though the user agent was set to 3.6, it still happens on mozilla-central.

Comment 2

[Makoto Kato [:m_kato]]

I cannot reproduce this now. Feel free to reopen this with test case.