108412 - nsFontMetricsGTK/nsFontMetricsXlib cannot find correct jisx0201 fonts

Status: RESOLVED WONTFIX

(Core :: Internationalization, defect)

Description

[Shoji Matsumoto]

bugzilla-jp : 1538

on Unix, nsFontMetricsGTK finds illegal jisx0201 fonts.

Test Page :
http://bugzilla.mozilla.gr.jp/showattachment.cgi?attach_id=361Illegal shot :
http://bugzilla.mozilla.gr.jp/showattachment.cgi?attach_id=362

---------------
My setting: 

  System has XLFDs below (all of them are scalable font : ttf)

    -microsoft-mspmincho-...-p-{iso8859-1,jisx0201.1976-0,jisx0208.1983-0}
    -microsoft-mspgothic-...-p-{iso8859-1,jisx0201.1976-0,jisx0208.1983-0}
    -microsoft-msmincho-....-c-{iso8895-1,jisx0201.1976-0,jisx0208.1983-0}
    -microsoft-msgothic-....-c-{iso8859-1,jisx0201.1976-0,jisx0208.1983-0}

  Mozilla prefs for Japanese fonts (0.9.5) :

    serif : mspmincho
    sans-serif : mspgothic
    proportional : sans-serif

----------------
Problem:
see http://bugzilla.mozilla.gr.jp/showattachment.cgi?attach_id=362

1st section : font face name is nothing, then use prop=sans-serif=mspgothic. OK.

2nd section : font face name is "MS-Mincho" in Japanese. Because  mozilla
currently cannot treat non-ascii familynames, it's ignored. then use
prop=sans-serif=mspgothic. OK.

3rd section : font face name is "msmincho". System has
-microsoft-msmincho-...-c-{iso88591-1,jisx0201.1976-0,jisx0208.1983-0}, so
mozilla should use these fonts (fontset), but Katakana(jisx0201) seems
mspgothic. BAD.

4th section : font face name is "serif" (generic), then use serif=mspmincho. OK.

------------
Solution: 

http://bugzilla.mozilla.gr.jp/showattachment.cgi?attach_id=369

I tested on 0.9.5/Linux with this patch.

Result:

http://bugzilla.mozilla.gr.jp/showattachment.cgi?attach_id=370

All OK.

Comment 1

[Koike Kazuhiko]

Attachment: patch

Can somebody review this patch?

Comment 2

[Roy Yokoyama]

shanjian: can you review his patch?  It's in linux GTK.

Comment 3

[Shanjian Li]

Koike, refers to bug 81528, you fix will broke that one, and it is much more 
serious. In bug 81528, bstell does realize that in certain situation, his method 
of checking for empty fonts will disable some of the fonts. We have to find a 
new way to detect empty font. 

Comment 4

[kill this account]

I am very aware the the test for "empty" fonts can disable valid fonts.

A better solution would be nice.

Comment 5

[kill this account]

But I do not know a better solution. 

So the choices available right now are:

  do not check and display blank (read: unreadable) chars

  do check and disable some valid fonts

Comment 6

[Shanjian Li]

accepting

Comment 7

[Roland Mainz]

Comment on attachment 58317 [details] [diff] [review]
patch

The patch does not patch the nsFontMetricsXlib code... ;-(

Comment 8

[Roland Mainz]

Attachment: Patch for 2002-03-23-08-trunk (both GTK+ and Xlib fontmetrics)

Comment 9

[kill this account]

Let me state explicitly that I do not intend to make comment or review the
attachment. It is my time to step aside and it is time for Shanjian to be the 
GTK font master.

Comment 10

[kill this account]

has this patch been tested to see if it regresses bug 81528 ?

Comment 11

[Roland Mainz]

Brian Stell wrote:
> has this patch been tested to see if it regresses bug 81528 ?

I do not have Redhat Japanese 6.2 ... I can't test it... ;-(

Comment 12

[Teruko Kobayashi]

Changed QA contact to [e-mail] since she is QA contact in the related
bug 81528 .

Comment 13

[kill this account]

can you arrange a test build for IQA to test against?

Comment 14

[Roland Mainz]

Brian Stell wrote:
> can you arrange a test build for IQA to test against?

I can provide a test build for Solaris/SPARC - does thaht help ?

Comment 15

[kill this account]

humm... the problem was Redhat 6.2

Comment 16

[Shoji Matsumoto]

attachment 35436 [details] [diff] [review] in bug 81528 is invalid.

It invalidates ALL valid jisx0201 font entries generated by X-TT with -c-
spacing. X-TT does not add xFont->per_char with -c- spacing.

X-TT is defacto standard X TrueType renderer in Japan.
I think all Linux ditributions for Japanese use X-TT.

Mutsumi Ishikawa, generator of watanabe.ttf & wadalab.ttf, says
they have error. Fixed ttf is here.

http://packages.debian.org/unstable/x11/ttf-xtt-wadalab-gothic.html
http://packages.debian.org/unstable/x11/ttf-xtt-watanabe-mincho.html

update package for RH6.2J is not found.

# I don't know about cns11643...

Comment 17

[kill this account]

> attachment 35436 [details] [diff] [review] in bug 81528 is invalid.

Attachment 35436 [details] [diff] detects JISX201 fonts that are lacking ascii glyphs. If these 
fonts are used then Japanese pages show blank areas where the ascii should be.

Has anyone tested that the new attachment retains this required safeguard?

Comment 18

[Shoji Matsumoto]

Really?

I tested the routine on my machine, it invalidates many valid fonts.

Comment 19

[Shoji Matsumoto]

Attachment: fonttest.cpp : test program

Comment 20

[Shoji Matsumoto]

Attachment: screenshot

kochi-gothic is valid, but NOPCHAR.
wadalab-gothic is invalid, but not EMPTY.

Comment 21

[Shoji Matsumoto]

Ishikawa-san said watanabe.ttf / wadalab.ttf has no glyph for jisx0201.

So, jisx0201 entries of /usr/lib/X11/TrueType/fonts.dir in RHL6.2J is INVALID.
It's nonsense. MISTAKE OF RHL6.2J.

# I confirmed that these ttfs has no glyph for jisx0201 with ftview

Comment 22

[kill this account]

> it invalidates many valid fonts.

Yes, but how would we detect the invalid ones?

Comment 23

[Shoji Matsumoto]

> Yes, but how would we detect the invalid ones?

I believe it is impossible to detect them without scanning all glyph patterns :(

Isn't it possible to use pref("font.x11.rejectfontpattern",
"fnames=-(wadalab|watanabe)-.*-jisx0201.*; ...") ?

The fndry "watanabe" and "wadalab" is very famous for Japanese (watanabe.ttf,
wadalab.ttf). And now we know they don't have glyphs for jisx0201.

# I don't know about sun-ming-cns11643-[12] (related bug 67732, bug 81528).

Comment 24

[kill this account]

Font banning would be a more intelligent solution but when font banning enabled 
the startup performance suffered so it had to be turned off. Until that is 
fixed we cannot use it. :(

> I believe it is impossible to detect them without scanning all glyph patterns 

Actually, that's not a totally unreasonable thing to do. Moz already has font
catalog code for the direct FreeType2 code. I remember seeing a posting in the
last few months that some other project scans all the fonts for glyphs.

Another option to accessing Truetype is to use the Truetype fonts via a FreeType 
interface such as the direct FreeType code or Xft. In the near future the
direct FreeType path will also support printing (Truetype -> Postscript).

Comment 25

[Shoji Matsumoto]

fummm.... I feel so > starting performance issue

# It should be added ONLY for RH6.2J :b

Yes, FT2/Xft is very good solution, I think too. But many Japanese (and Chinese,
Korean) cannot be satisfied because lack of Oblique/Italic/Bold valiants for any
TTFs.
I know no TTF set for screen display with Oblique/Italic/Bold valiants.
TTF renderer of Windows/Mac generates them with calcuration.

If such set would be, they need many many memory.. (about 8M * 4 for each set)
 
XTT in X is not maintained any more (and has some bugs) but we (and all CJK
users, I think) use it because it has "glyph morphing" feature. It can generate
Oblique/Bold glyphs from normal glyphs.

attachment 35436 [details] [diff] [review] rejects all jisx0201 pseudo fonts  XTT generated with -c-
spacing. We cannot use scalable(TTF) ASCII glyphs on all Japanese pages.
XTT doesn't generate per_char FontStruct for performance.

# If mozilla (or freetype, Xft) could generate them, we can say good-bye to suck
XLFDs...

Comment 26

[Koike Kazuhiko]

Attachment: Screenshot(Vine Linux 2.6)

"Bugzilla Bug 108412" is displayed in ugly font.

Should we still care about RedHat 6.2?

Comment 27

[Frank Tang]

shanjian is no longer working on mozilla for 2 years and these bugs are still
here. Mark them won't fix. If you want to reopen it, find a good owner first. 

Comment 28

[Travis Chase]

Mass Re-assigning bugs that Frank Tang Closed on March 1st Spam is his fault

Mass Re-Open to follow

Comment 29

[Travis Chase]

Mass Bug Re-Open of bugs Frank Tang Closed with no good reason. Spam is his
fault not my own

Comment 30

[Travis Chase]

Reassigning Franks old bugs to Jungshik Shin for triage - Sorry for spam

Comment 31

[KENZ (Kenji Irie)]

Should this bug be WONTFIX?
Since GTK+1/Xlib code was removed by bug 326152.

Comment 32

[Hideo Oshima]

(In reply to comment #31)
> Should this bug be WONTFIX?
> Since GTK+1/Xlib code was removed by bug 326152.
 
I seconded.
GTK2 uses fontconfig and fontconfig uses font set.
So we don't need to fix this.

Comment 33

[Peter Weilbacher]

resolution as suggested.