196312 - Gfx-Xft needs a work-around for Xft/Freetype problem with CJK 'monospace' (bi-width) fonts

Status: RESOLVED INCOMPLETE

(Core Graveyard :: GFX: Gtk, defect)

Description

[Jungshik Shin]

There are several CJK truetype fonts with fixed-width 'bit' set,
that are actually 'bi-width' fonts with glyphs for CJK
characters (East Asian width property 'full') being
twice as wide as glyphs for (most) Latin alphabetic
letters and numerals (East Asian width property 'half'
See Unicode Technical Report 11 :http://www.unicode.org/reports/tr11/
).

Currently, Freetype2 and Xft don't have a special provision
to handle those fonts and regard all the glyphs
in them as having the same width as CJK characters(max_advance).  
The result is that alphabetic letters and numbers
are rendered in a rectangular cell twice wider than necessary 
to 'enclose' glyphs for them. 


There are a couple of patches around
(see http://bugzilla.gnome.org/show_bug.cgi?id=106618
and links refered to there), but it may take a while
until this issue is addressed in those libraries.
In the meantime, their 'clients' may need to haev
a work-around. For gnome 'vte' and gtk2 port
of Vim, I provided a patch (which was committed
about a week ago) that worked quite well. 

Mozilla may need to implement a similar work-around
that would kick in only when upperstream libraries
(Xft, FT2 and possibly fontconfig) don't take care of 
the issue.

Comment 1

[Jungshik Shin]

Attachment: a screenshot (Mozilla-Xft) showing the problem with a CJK bi-width/fixed font

Note that in the screenshot, all alphabets and numbers
have the same width, exactly _half_ the width of 
Korean Hangul syllables. However, they're rendered
in cells of the same width as  cells for Korean
characters leading to 's p a c e d o u t' rendering.

Comment 2

[Jungshik Shin]

Attachment: a screenshot of Mozilla-win with the same font

Mozilla-Win does not have this problem apparently because
Windows font library takes care of the issue unlike
FT2 and Xft. 

Both screenshots were taken with 'Gulimche' (available
in any language version of Windows 2k and Windows XP
or Global IME for Korean for Windows 9x/ME). 'Gulimche'
is one of fonts in 'gulim.ttc'.

Comment 3

[keithp]

fontconfig believed fonts that said they were 'monospaced' and set the
FC_SPACING value to FC_MONO.  FC_SPACING is used by Xft to force fonts to be
monospaced by setting their glyph widths to the maximum over the whole font. 
Hence, any font advertised as monospaced is forced to be monospaced by Xft.

The work around is pretty simple for older verisons of fontconfig.  After the
font is matched and before it is opened, set the FC_SPACING value to
FC_PROPORTIONAL, that will make Xft obey the individual glyph widths.

Comment 4

[Jungshik Shin]

Were you implying that a newer version of fontconfig behaves differently? When
did that change get effective? I pulled
fonconfig and Xft out of the xf86 cvs repository in late February
and it appeared that it worked the old way. 

BTW, wouldn't it be nice to keep setting FC_SPACING to FC_MONO for CJK bi-width
fonts in fontconfig and to deal with them in Xft? 
 

Comment 5

[keithp]

Yes, current fontconfig bits don't ever set FC_SPACING; most real monospaced
fonts (i.e. those suitable for use in a simplistic terminal emulator) don't
bother to indicate this, and (as we've just seen), many non-monospaced fonts set
the attribute.  I'd originally assumed that fonts indicating monospacing were
likely to be correct.

XFree86 CVS doesn't hold the canonical fontconfig repository, that's at
cvs.fontconfig.org.  I'm working towards a 2.2 release a this point; all that's
left is to reformat some of the documentation.

Comment 6

[Jungshik Shin]

Thank you for the info. With libfontconfig built from the cvs
snapshot, Mozilla works fine as it is. 

As for old fontconfig, IMHO,
we'd rather not do anything because I found that putting
the following lines into font config. file also works:

<match target="font">
		<test name="family"><string>GulimChe</string></test>
		<edit name="globaladvance"><bool>false</bool></edit>
</match>

BTW, I did try changing FC_SPACING from FC_MONOSPACE to FC_PROPORTIONAL 
in mozilla, but somehow I got segfaulted in xftfreetype.c (of Xft)


> (as we've just seen), many non-monospaced fonts set
> the attribute.

  This is not so simple, I guess. Literally, they're not
monospace, but 'semantically', CJK bi-width fonts are 'monospace' and
are used where 'monospace' fonts are used (terminal
emulators, rendering of <pre> in html). That's why foundries set the attribute.
Because it's kinda off-topic here, I'll write more
about it in other forums.
 

Comment 7

[Jungshik Shin]

Attachment: a patch to work around the problem for fc version < 2.2

> we'd rather not do anything because I found that putting
> the following lines into font config. file also works:
>  <match target="font">
>		<test name="family"><string>GulimChe</string></test>
>		<edit name="globaladvance"><bool>false</bool></edit>
>  </match>

It was premature to say that the above worked for me. (I heard that it worked
for someone, but it never worked for me). Anyway, presumably it'll take 
fontconfig 2.2 to get widely deployed. For those who still use earlier
versions,
I think it's a good idea to offer a work around (similar to that for Xrender
crash 
problem with Xft.). I would have done it back in March, but at that time
I was modifying the code at a wrong place and couldn't make it work. 

Because monospace font is also used for textarea, textarea always comes 
up twice as wide as the specified width (when a CJK bi-width font is used 
for monospace).  

blizzard, can we get this simple patch in? This will remove one item
from I18N release notes of 1.4 final. Of course, if 'globaladvance'
setting can be made to work, I'd withdraw the patch and release-note
that, instead.

Comment 8

[Jungshik Shin]

Attachment: a patch with typo fixed

I'm sorry that attachment 122833 [details] [diff] [review] had a typoe. It's beyond me why the prev.
patch had a typo because I think I compiled it and confirmed that it worked as
intended.

Comment 9

[Christopher Blizzard (:blizzard)]

Comment on attachment 122853 [details] [diff] [review]
a patch with typo fixed


>+
>+        // bug 196312 : work around problem with CJK bi-width fonts
>+        if (gFcWorkaroundBiWidthFont ) 
>+            FcPatternDel(pat, FC_SPACING);

This isn't called all that often.  Just use FcGetVersion() and compare it
against the value directly.  And make the comment a little bigger, explaining
what's going on.

Comment 10

[Jungshik Shin]

Attachment: two liner + comment

blizzard, can you review?

Comment 11

[Christopher Blizzard (:blizzard)]

Comment on attachment 122916 [details] [diff] [review]
two liner + comment

r=blizzard

Thanks!  Go ahead and check this in.

Comment 12

[Jungshik Shin]

Purely by chance (I was looking for another bug), I 
got to know that fontconfig 2.2 still has the problem
(2.1.90 - 2.1.92 don't have the problem). I confirmed
the problem by installing fc 2.2. I took keith's word
in comment #5 :-). Apparently, he added some optimization
after writting comment #5 (in 2.1.93.) 
See http://fontconfig.org/cgi-bin/bugzilla/show_bug.cgi?id=82
This time it appears that it's FT2's fault. Perhaps,
we just have to call |FcPatternDel| unconditionally in
the meantime.
 

Comment 13

[Mats Palmgren (inactive)]

Does this bug still occur in a recent trunk build?
(If so, please change the Component to GFX:Thebes)

Comment 14

[Zack Weinberg (:zwol)]

The code changed by the patch no longer exists, and we're long past fontconfig 2.2 on current Linux distros, so I suspect this is no longer a problem.  If it is still a problem, please file a new bug; it would be very helpful if that bug had a test case using @font-face so everyone was looking at the same fonts.