223653 - [Xft] support localized font family name

Status: RESOLVED FIXED

(Core Graveyard :: GFX: Gtk, defect)

Description

[Jungshik Shin]

At the moment, this is just a place holder because this bug  depends on 
fontconfig which doesn't yet offer a way to retrieve localized font names for UI 
and font matching.  Numerous web sites in CJK countries      use localized font 
family names in CSS [1] so  that Mozila have to match them against  fonts 
available on the system.  In addition,  the font selection  menu  needs to give 
localized names corresponding to the current locale (i.e.  Korean names under 
Korean locale if available) 
 
 
[1] There's an evangelisim issue here.  Because not  all web browsers can 
understand localized font names,  web page authoers in CJK countries should 
list  a single font  twice ( Non-ASCII name and ASCII-only name) in CSS.

Comment 1

[Jungshik Shin]

fontconfig  bug I've just filed :
http://freedesktop.org/cgi-bin/bugzilla/show_bug.cgi?id=171

Comment 2

[Jungshik Shin]

http://firefly.idv.tw/setfont-xft/patches/mozilla/mozilla-1.6-xft_cjkfamilyname-20040117.patch

This patch, by itself, doesn't work. fontconfig should be fixed as well. See 

http://firefly.idv.tw/setfont-xft/patches/fontconfig/fontconfig-2.2.92-multifamily-20031217.patch

Comment 3

[keithp]

Fontconfig has been (partially) fixed.  It now presents all available names to
the application.

But, mozilla is now confused by the UTF-8 names it sees, displaying garbage
instead of the correct name.  I assume it doesn't know what encoding the font
names are presented in; fontconfig takes some care to present them in UTF-8 only.

Comment 4

[Jungshik Shin]

Keith, it's fixed on fontconfig CVS, right? I'll grab it sometime soon and work
on Mozilla's side to take advantage of. 

Comment 5

[keithp]

Fontconfig 2.2.97 has support for multi-lingual names.  You can find that
release at http://fontconfig.org.  This was released yesterday, so CVS is
identical at the present hour.  Use either; you'll be among the first on your block.

Because Mozilla boldly uses the first name in each font, most names work just
fine; TrueType fonts almost always place an English name first.  I have one font
(ＭＳ ゴシック,MS Gothic) which places a Japanese name first.  For this font,
Mozilla presents me with (I think) "￯ﾼﾭ￯ﾼﾳ ￣ﾂﾴ￣ﾂﾷ￣ﾃﾃ￣ﾂﾯ" or perhaps "￯ﾼﾭ￯ﾼﾳ
￦ﾘﾎ￦ﾜﾝ".  Presumably, this is just a misunderstanding of the encoding used for
font names, although Fontconfig has long specified that all strings are in UTF-8
encoding.

Comment 6

[Jungshik Shin]

Thanks.

reposting of comment #5 (with the character encoding set to UTF-8 in Mozilla)

Because Mozilla boldly uses the first name in each font, most names work just
fine; TrueType fonts almost always place an English name first. I have one font
(£Í£Ó «´«·«Ã«¯,MS Gothic) which places a Japanese name first. For this font,
Mozilla presents me with (I think) "￯ﾼﾭ￯ﾼﾳ £þﾂﾴ£þﾂﾷ£þﾃﾃ£þﾂﾯ" or perhaps "￯ﾼﾭ￯
ﾼﾳ£Üﾘﾎ£Üﾜﾝ". Presumably, this is just a misunderstanding of the encoding used
for font names, although Fontconfig has long specified that all strings are in
UTF-8 encoding.

Comment 7

[Jungshik Shin]

sorry for spamming. The encoding was not set to UTF-8 when posting comment #6.

Because Mozilla boldly uses the first name in each font, most names work just
fine; TrueType fonts almost always place an English name first. I have one font
(ＭＳ ゴシック,MS Gothic) which places a Japanese name first. For this font,
Mozilla presents me with (I think) "￯ﾼﾭ￯ﾼﾳ ￣ﾂﾴ￣ﾂﾷ￣ﾃﾃ￣ﾂﾯ" or perhaps "￯ﾼﾭ￯
ﾼﾳ￦ﾘﾎ￦ﾜﾝ". Presumably, this is just a misunderstanding of the encoding used
for font names, although Fontconfig has long specified that all strings are in
UTF-8 encoding.

Comment 8

[Jungshik Shin]

I've got a patch (but I couldn't make a diff because I can't access
cvs.mozilla.org due to some firewall set-up mistake on my network. Tomorrow,
I'll ask them to fix it) and it works well. 

I found one problem on fontconfig's part. fontconfig's family name matching is
case-insensitive for ASCII range, but it seems like it's case-sensitive for
characters like 'ＭＳ' in 'ＭＳ ゴシック' (note that they're drawn from
'Fullwidth ASCII variants' at U+FF00 block). We lowercases family names to avoid
problems with 'Sans vs sans vs SANS'. Keith, what do you use for family name
comparison in fontconfig?

Comment 9

[Jungshik Shin]

Attachment: test case

Comment 10

[keithp]

I'm using a custom string comparison function which ignores case only in the
ASCII range.  I know this is wrong.  I can't use the libc string functions as I
can't set the locale.  Should I import a Unicode case table so I can do a full
Unicode case insensitive match?  Is there an easy place to get such a table?

Comment 11

[Jungshik Shin]

http://www.unicode.org/Public/UNIDATA/CaseFolding.txt

is what you want. However, given that the vast majority of fonts only have
English names and only Japanese fonts have full-width ASCII variants in their
names (Korean and Chinese fonts just use Korean script and Chinese characters),
wouldn't it be an overkill for fontconfig? Well, I guess you can come up with a
very compact data structure for that :-) ... Although it's not perfect, you may
(at least for now) case-folds just ASCII range AND full-width ASCII-variant range. 

On Mozilla's side, I can just lowercase 'ASCII' range for now to work around it.

Comment 12

[Jungshik Shin]

Attachment: patch

Comment 13

[Jungshik Shin]

Attachment: patch updated

NS_IsASCIIFontName is not necessary any more so that I got rid of it.

Comment 14

[rbs]

Comment on attachment 170444 [details] [diff] [review]
patch updated

sr=rbs

Comment 15

[keithp]

I didn't see an explicit  comment, so I wanted to make sure this patch takes
into account that fontconfig now has full Unicode case folding support.

Comment 16

[Jungshik Shin]

Either way, this patch should work as it is now (in most of cases, if not all)

I may move up 'ToLowercase' so that it's applied to the whole range of Unicode
characters (instead of just [A-Z]) and get rid of the comment about fontconfig.
On the other hand, if fontconfig does the full case-folding, we may as well be
just lazy here (in which case I have to reword the comment to that effect)
assuming that Japanese web authors always use the same case when specifying font
names in 'wide' ASCII variant.  That will save us a string copy and a
full-fledged case folding....

 

Comment 17

[keithp]

Yes, fontconfig has complete Unicode case folding.  I didn't apply the Turkic
rules for İ/I/ı/i, but I'm thinking I should just fold all of those to i and be
done with it.  It seems rather unlikely to me that a font name would differ only
in the dottedness of an i in a non-Turkic language.

I managed to compress the case folding data into a 2213 bytes, which should be
shared across all fontconfig applications as I was careful to not have any
relocations in the data.

Comment 18

[Jungshik Shin]

Comment on attachment 170444 [details] [diff] [review]
patch updated

asking for a1.8

Thanks for the comment and reviews.

When checking this in, 
I'll rephrase the comment about fontconfig keeping the code as it is now.

Comment 19

[Asa Dotzler [:asa]]

jshin, we're down to the very end of alpha6 and can't afford any regressions
that would delay it longer. what kind of risk does this patch pose and what kind
of testing has been done on it. 

Comment 20

[Jungshik Shin]

I tested the patch [1]  with an old fontconfig and with a new fontconfig and
found that it works as expected. In the first case, native font names are
effectively skipped because the old fontconfig doesn't support non-ASCII font names.

[1] I used http://jshin.net/moztest/cjkfont.html for testing and
https://bugzilla.mozilla.org/attachment.cgi?id=139409

Only CJK Linux users visiting sites with CJK native fonts specified in CSS would
be affected. 

Risk: low (with an old fontconfig,  it tries next font if the first one has a
non-ASCII name)
 

Comment 21

[Asa Dotzler [:asa]]

Comment on attachment 170444 [details] [diff] [review]
patch updated

a=asa for checkin to 1.8a6. thanks for the quick reply, jshin.

Comment 22

[Jungshik Shin]

landed before 1.8a6 shipped. marking as fixed. thanks all.

Comment 23

[Christian :Biesinger (don't email me, ping me on IRC)]

*** Bug 299382 has been marked as a duplicate of this bug. ***

Comment 24

[Christian :Biesinger (don't email me, ping me on IRC)]

*** Bug 312812 has been marked as a duplicate of this bug. ***