Last Comment Bug 678561 - Firefox fails to use fallback font assigned in css rule in Linux
: Firefox fails to use fallback font assigned in css rule in Linux
Status: RESOLVED FIXED
: regression
Product: Core
Classification: Components
Component: Layout: Text (show other bugs)
: 5 Branch
: x86 Linux
: -- normal (vote)
: mozilla9
Assigned To: Karl Tomlinson (:karlt)
:
Mentors:
Depends on: 712900
Blocks: 546315
  Show dependency treegraph
 
Reported: 2011-08-12 11:23 PDT by Hemiola SUN
Modified: 2011-12-22 15:52 PST (History)
7 users (show)
karlt: in‑testsuite+
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
testcase (1.01 KB, text/html)
2011-08-12 11:23 PDT, Hemiola SUN
no flags Details
Firefox 5, Ubuntu 10.04, incorrect case (66.16 KB, image/png)
2011-08-12 11:25 PDT, Hemiola SUN
no flags Details
Firefox 3.6.18, Ubuntu 10.04, display correctly (66.98 KB, image/png)
2011-08-12 11:26 PDT, Hemiola SUN
no flags Details
Firefox 5, Windows 7, works well (21.65 KB, image/png)
2011-08-12 11:27 PDT, Hemiola SUN
no flags Details
Chrome, Ubuntu (68.43 KB, image/png)
2011-08-12 11:28 PDT, Hemiola SUN
no flags Details
Testcase 2 (959 bytes, text/html)
2011-08-14 18:16 PDT, Hemiola SUN
no flags Details
screenshot on linux for testcase 2 (61.06 KB, image/png)
2011-08-14 18:17 PDT, Hemiola SUN
no flags Details
Firefox 5.0 screenshot on linux for testcase 2 (55.00 KB, image/png)
2011-08-14 18:19 PDT, Hemiola SUN
no flags Details
Firefox 5.0 screenshot on Windows 7 for testcase 2, correct (19.15 KB, image/png)
2011-08-14 18:21 PDT, Hemiola SUN
no flags Details
fix strcmp logic error (1.29 KB, patch)
2011-08-24 17:32 PDT, Karl Tomlinson (:karlt)
jfkthame: review+
Details | Diff | Splinter Review

Description Hemiola SUN 2011-08-12 11:23:43 PDT
Created attachment 552704 [details]
testcase

Since Firefox 4.0, if the first font listed in font-family is a latin font, Firefox fails to use fallback font to display CJK characters and always use default font.

There are three tests in the testcase.html:

Test 1, only one CJK font (cwTeXHeiBold in this case) is assinged in css rule. Latin alphabets and CJK characters are all displayed by cwTeXHeiBold font.

Test 2, using rule: "font-family: Helvetica, cwTeXHeiBold, sans-serif;" CJK characters are expected to be displayed by cwTeXHeiBold, but actual result is by default sans-serif font.

Test 3, no CJK font assinged. Use default sans-serif font to display CJK chracters.

This bug only exists in Linux but no in Windows.
Comment 1 Hemiola SUN 2011-08-12 11:25:49 PDT
Created attachment 552706 [details]
Firefox 5, Ubuntu 10.04, incorrect case
Comment 2 Hemiola SUN 2011-08-12 11:26:56 PDT
Created attachment 552707 [details]
Firefox 3.6.18, Ubuntu 10.04, display correctly
Comment 3 Hemiola SUN 2011-08-12 11:27:48 PDT
Created attachment 552708 [details]
Firefox 5, Windows 7, works well
Comment 4 Hemiola SUN 2011-08-12 11:28:37 PDT
Created attachment 552709 [details]
Chrome, Ubuntu
Comment 5 Hemiola SUN 2011-08-14 18:16:22 PDT
Created attachment 553078 [details]
Testcase 2

testcase 2, uses system (ubuntu) default fonts (DejaVu Serif and FreeSerif), contains no more CJK characters. same results.
Comment 6 Hemiola SUN 2011-08-14 18:17:55 PDT
Created attachment 553079 [details]
screenshot on linux for testcase 2
Comment 7 Hemiola SUN 2011-08-14 18:19:19 PDT
Created attachment 553080 [details]
Firefox 5.0 screenshot on linux for testcase 2
Comment 8 Hemiola SUN 2011-08-14 18:21:32 PDT
Created attachment 553081 [details]
Firefox 5.0 screenshot on Windows 7 for testcase 2, correct
Comment 9 Karl Tomlinson (:karlt) 2011-08-14 22:44:43 PDT
Is cwTeXHeiBold freely available for download?

I'm having trouble reproducing with the second test case here.  (I seem to be getting the correct fallback to FreeSerif when I use the older DejaVu Serif from Ubuntu 10.04).

Do you get the same behavior with a new profile?

Can you paste the output of 'fc-match --all "DejaVu Serif,FreeSerif,serif:slant=roman" | head -n 30', please?
Comment 10 Hemiola SUN 2011-08-15 02:34:38 PDT
(In reply to Karl Tomlinson (:karlt) from comment #9)
> Is cwTeXHeiBold freely available for download?

Yes, you can download cwheib.ttf here:
http://cle.linux.org.tw/fonts/cwttf/center/

> 
> I'm having trouble reproducing with the second test case here.  (I seem to
> be getting the correct fallback to FreeSerif when I use the older DejaVu
> Serif from Ubuntu 10.04).
> 
> Do you get the same behavior with a new profile?

Yes, same result with a new profile in Ubuntu 10.04 and Linux Mint 11.

However, the problem can be reproduced ONLY in Traditional Chinese (zh-tw) Ubuntu/Linux Mint. When I change the system language to English, the problem no longer exists.

> 
> Can you paste the output of 'fc-match --all "DejaVu
> Serif,FreeSerif,serif:slant=roman" | head -n 30', please?

In Chinese system, of course:

FreeSerif.ttf: "FreeSerif" "Mittel"
FreeSerifBold.ttf: "FreeSerif" "Fett"
FreeSerifItalic.ttf: "FreeSerif" "Kursiv"
FreeSerifBoldItalic.ttf: "FreeSerif" "FettKursiv"
DejaVuSerif.ttf: "DejaVu Serif" "Book"
DejaVuSerifCondensed.ttf: "DejaVu Serif" "Condensed"
DejaVuSerif-Bold.ttf: "DejaVu Serif" "Bold"
DejaVuSerifCondensed-Bold.ttf: "DejaVu Serif" "Condensed Bold"
DejaVuSerif-Italic.ttf: "DejaVu Serif" "Italic"
DejaVuSerifCondensed-Italic.ttf: "DejaVu Serif" "Condensed"
DejaVuSerif-BoldItalic.ttf: "DejaVu Serif" "Bold Italic"
DejaVuSerifCondensed-BoldItalic.ttf: "DejaVu Serif" "Condensed Bold Italic"
uming.ttc: "AR PL UMing TW" "Light"
uming.ttc: "AR PL UMing HK" "Light"
odosung.ttc: "AR PL New Sung" "Regular"
ukai.ttc: "AR PL UKai TW" "Book"
ukai.ttc: "AR PL UKai HK" "Book"
DejaVuSans.ttf: "DejaVu Sans" "Book"
DejaVuSansCondensed.ttf: "DejaVu Sans" "Condensed"
DejaVuSans-ExtraLight.ttf: "DejaVu Sans" "ExtraLight"
DejaVuSans-Bold.ttf: "DejaVu Sans" "Bold"
DejaVuSansCondensed-Bold.ttf: "DejaVu Sans" "Condensed Bold"
DejaVuSans-Oblique.ttf: "DejaVu Sans" "Oblique"
DejaVuSansCondensed-Oblique.ttf: "DejaVu Sans" "Condensed Oblique"
DejaVuSans-BoldOblique.ttf: "DejaVu Sans" "Bold Oblique"
DejaVuSansCondensed-BoldOblique.ttf: "DejaVu Sans" "Condensed Bold Oblique"
wqy-microhei.ttc: "WenQuanYi Micro Hei" "Regular"
TW-Sung-98_1.ttf: "TW-Sung" "Regular"
Sun-ExtB.ttf: "Sun-ExtB" "Regular"
wqy-zenhei.ttc: "WenQuanYi Zen Hei" "中等"
Comment 11 Hemiola SUN 2011-08-15 08:09:35 PDT
Output of 'fc-match --sort "DejaVu Serif"':

DejaVuSerif.ttf: "DejaVu Serif" "Book"
DejaVuSerif-Bold.ttf: "DejaVu Serif" "Bold"
DejaVuSerif-Italic.ttf: "DejaVu Serif" "Italic"
DejaVuSerif-BoldItalic.ttf: "DejaVu Serif" "Bold Italic"
uming.ttc: "AR PL UMing TW" "Light"
uming.ttc: "AR PL UMing HK" "Light"
odosung.ttc: "AR PL New Sung" "Regular"
Times_New_Roman.ttf: "Times New Roman" "Normal"
n021003l.pfb: "Nimbus Roman No9 L" "Regular"
uming.ttc: "AR PL UMing CN" "Light"
Kinnari.ttf: "Kinnari" "Medium"
KhmerOS.ttf: "Khmer OS" "Regular"
MuktiNarrow.ttf: "Mukti Narrow" "Regular"
wqy-zenhei.ttc: "WenQuanYi Zen Hei" "中等"
wenquanyi_10pt.pcf: "WenQuanYi Bitmap Song" "Regular"
wenquanyi_9pt.pcf: "WenQuanYi Bitmap Song" "Regular"
wenquanyi_11pt.pcf: "WenQuanYi Bitmap Song" "Regular"
wenquanyi_12pt.pcf: "WenQuanYi Bitmap Song" "Regular"
UnBatang.ttf: "UnBatang" "Regular"
lohit_bn.ttf: "Lohit Bengali" "Regular"
lohit_gu.ttf: "Lohit Gujarati" "Regular"
lohit_hi.ttf: "Lohit Hindi" "Regular"
lohit_pa.ttf: "Lohit Punjabi" "Regular"
lohit_ta.ttf: "Lohit Tamil" "Regular"
Meera_04.ttf: "Meera" "Regular"
FreeSerif.ttf: "FreeSerif" "Mittel"
(...)

For Test 3 of Testcase 2 (font-family: DejaVu Serif, FreeSerif, serif;), ths missing Vietnamese alphabets are displayed by "AR PL UMing TW" font.
Comment 12 Karl Tomlinson (:karlt) 2011-08-15 19:52:20 PDT
(In reply to Hemiola SUN from comment #10)
> (In reply to Karl Tomlinson (:karlt) from comment #9)
> > Can you paste the output of 'fc-match --all "DejaVu
> > Serif,FreeSerif,serif:slant=roman" | head -n 30', please?
> 
> In Chinese system, of course:
> 
> FreeSerif.ttf: "FreeSerif" "Mittel"
> FreeSerifBold.ttf: "FreeSerif" "Fett"
> FreeSerifItalic.ttf: "FreeSerif" "Kursiv"
> FreeSerifBoldItalic.ttf: "FreeSerif" "FettKursiv"
> DejaVuSerif.ttf: "DejaVu Serif" "Book"

I expected DejaVuSerif.ttf to appear before FreeSerif.ttf, but then bugzilla removed a space from my command.  Did you run this?:

> fc-match --all "DejaVu Serif,FreeSerif,serif:slant=roman" | head -n 30
Comment 13 Hemiola SUN 2011-08-15 20:04:01 PDT
(In reply to Karl Tomlinson (:karlt) from comment #12)
> I expected DejaVuSerif.ttf to appear before FreeSerif.ttf, but then bugzilla
> removed a space from my command.  Did you run this?:
> 
> > fc-match --all "DejaVu Serif,FreeSerif,serif:slant=roman" | head -n 30

This is the output:

DejaVuSerif.ttf: "DejaVu Serif" "Book"
DejaVuSerifCondensed.ttf: "DejaVu Serif" "Condensed"
DejaVuSerif-Bold.ttf: "DejaVu Serif" "Bold"
DejaVuSerifCondensed-Bold.ttf: "DejaVu Serif" "Condensed Bold"
DejaVuSerif-Italic.ttf: "DejaVu Serif" "Italic"
DejaVuSerifCondensed-Italic.ttf: "DejaVu Serif" "Condensed"
DejaVuSerif-BoldItalic.ttf: "DejaVu Serif" "Bold Italic"
DejaVuSerifCondensed-BoldItalic.ttf: "DejaVu Serif" "Condensed Bold Italic"
FreeSerif.ttf: "FreeSerif" "Mittel"
FreeSerifBold.ttf: "FreeSerif" "Fett"
FreeSerifItalic.ttf: "FreeSerif" "Kursiv"
FreeSerifBoldItalic.ttf: "FreeSerif" "FettKursiv"
uming.ttc: "AR PL UMing TW" "Light"
uming.ttc: "AR PL UMing HK" "Light"
odosung.ttc: "AR PL New Sung" "Regular"
ukai.ttc: "AR PL UKai TW" "Book"
ukai.ttc: "AR PL UKai HK" "Book"
DejaVuSans.ttf: "DejaVu Sans" "Book"
DejaVuSansCondensed.ttf: "DejaVu Sans" "Condensed"
DejaVuSans-ExtraLight.ttf: "DejaVu Sans" "ExtraLight"
DejaVuSans-Bold.ttf: "DejaVu Sans" "Bold"
DejaVuSansCondensed-Bold.ttf: "DejaVu Sans" "Condensed Bold"
DejaVuSans-Oblique.ttf: "DejaVu Sans" "Oblique"
DejaVuSansCondensed-Oblique.ttf: "DejaVu Sans" "Condensed Oblique"
DejaVuSans-BoldOblique.ttf: "DejaVu Sans" "Bold Oblique"
DejaVuSansCondensed-BoldOblique.ttf: "DejaVu Sans" "Condensed Bold Oblique"
wqy-microhei.ttc: "WenQuanYi Micro Hei" "Regular"
TW-Sung-98_1.ttf: "TW-Sung" "Regular"
Sun-ExtB.ttf: "Sun-ExtB" "Regular"
wqy-zenhei.ttc: "WenQuanYi Zen Hei" "中等"
Comment 14 Karl Tomlinson (:karlt) 2011-08-15 21:04:55 PDT
I configured LC_ALL=zh_TW.UTF8, but am still failing to reproduce with either testcase on my Gentoo system.

Might be worth moving away /etc/fonts/conf.d/ and ~/.fonts.conf temporarily (and restarting the browser) to check they are not having an effect.
Comment 15 Hemiola SUN 2011-08-18 00:39:52 PDT
(In reply to Karl Tomlinson (:karlt) from comment #14)
> I configured LC_ALL=zh_TW.UTF8, but am still failing to reproduce with
> either testcase on my Gentoo system.
> 
> Might be worth moving away /etc/fonts/conf.d/ and ~/.fonts.conf temporarily
> (and restarting the browser) to check they are not having an effect.

Thank you, Karl.

I think the problem exist only in Ubuntu, which has its own way to handle font in different localization. For example, the font setting in Chinese Taiwan system is defined in /etc/fonts/conf.d/69-language-selector-zh-tw.conf.

Could you please try again in Ubuntu?
Comment 16 Hemiola SUN 2011-08-24 01:09:01 PDT
This bug is similar to bug 546315. Some friends and I have done some works for testcase 1 of this bug and the testcase of bug 546315, and the results are listed below:

http://hemiolapei.free.fr/temp/fallback/fallback-test.html

Current conclusions:

1. Firefox 3.5, 3.6 and 4 work well. The two bugs occur only in Firefox 5 and later.

2. For bug 678561:

a. In Ubuntu based system, /etc/fonts/conf.d/69-language-selector-*.conf files (used for zh, jp, ko) cause bug 678561. 

b. In systems which do not have these files, it works well, no matter the LANG and LC_ALL settings are. That's why Karl could not reproduce the bug.

c. Try adding '<edit name="family" mode="prepend" binding="strong">' (this is just what 69-language-selector-*.conf do) in local.conf or .fonts.conf, and you would get the wrong result.

3. For bug 546315:

a. It seems that bug 546315 is not really fixed. In many cases, the last line of the testcase is not correctly displayed. It is displayed with default font instead of Georgia font. I am not sure how to reproduce this bug. It occurs in Chinese and Japanese systems. In English system without any change of the language or font setting, it seems to work well.
Comment 17 Karl Tomlinson (:karlt) 2011-08-24 02:10:50 PDT
Thanks for the analysis.

Can you tell me which package provides /etc/fonts/conf.d/69-language-selector-zh-tw.conf please?

I don't see it at http://packages.ubuntu.com/lucid/all/fontconfig-config/filelist
Comment 18 Hemiola SUN 2011-08-24 03:29:26 PDT
It's here:
http://packages.ubuntu.com/lucid/language-selector-common
Comment 19 Karl Tomlinson (:karlt) 2011-08-24 17:30:52 PDT
Thanks for all your help on this.

It's unusual to use binding="strong" for default aliases like this.  I guess binding="strong" is used to select DejaVu for Latin and common characters amongst zh characters (even though DejaVu does not support zh).  binding="same" may have been enough -- I'm not sure.  The zh fonts used as defaults should probably have binding="weak", and the lang-contains-zh test added is sensible, at least for the Latin fonts.

However, this all picks up a logic error introduced in bug 546315.
Comment 20 Karl Tomlinson (:karlt) 2011-08-24 17:32:41 PDT
Created attachment 555602 [details] [diff] [review]
fix strcmp logic error
Comment 21 Jonathan Kew (:jfkthame) 2011-08-25 01:54:49 PDT
Comment on attachment 555602 [details] [diff] [review]
fix strcmp logic error

Wow - great job analysing and hunting this down!

Note You need to log in before you can comment on or make changes to this bug.