81528 - Invalid JIS 201 fonts on Redhat Japanese 6.2

Status: VERIFIED FIXED

(Core :: Internationalization, defect)

Description

[Yuying Long]

Build: 05-17 linux trunk build.

Go to some Japanese or Chinese pages:
   http://home.netscape.com/ja, http://home.netscape.com/zh/cn/
   http://www.sina.com.cn/, http://www.snweb.com/, http://www.china.com
you will find the Alphabet and numbers in those pages text string(e.g. inside 
those news header together with double-byte characters) are display as blank.
However, on western pages, all those alphabet and numbers are display fine 
though.

This is a regression, I didn't see it till 05-15-08 trunk build, and started on 
05-16-05 build.

Comment 1

[Yuying Long]

Add keywords and change QA contact to myself.

Comment 2

[Yuying Long]

aActually, in Composer, if you creat a document that contains alphabet and 
numbers with double0byte charset(e.g. s-jis), then browser it or re-open from 
Composer, you won't able to see it also.

Comment 3

[nhottanscp]

Frank landed converter change on 5/15 and 5/16, might be related.
Reassign to ftang.

Comment 4

[nhottanscp]

*** Bug 81513 has been marked as a duplicate of this bug. ***

Comment 5

[nhottanscp]

Checkins between 05/15/2001 00:00 and 05/16/2001 00:05

http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=all&branch=HEAD&branchtype=match&dir=&file=&filetype=match&who=&whotype=match&sortby=Date&hours=2&date=explicit&mindate=05%2F15%2F2001+00%3A00%3A00&maxdate=05%2F16%2F2001+00%3A05%3A00&cvsroot=%2Fcvsroot

Comment 6

[ji]

Naoki, it started to happen from 2001-05-15-21-mtrunk build.
The last good build is 2001-05-15-08-mtrunk.

Comment 7

[nhottanscp]

05/15/2001 18:34, Brian changed mozilla/gfx/src/gtk/nsFontMetricsGTK.cpp.

Brian, please take a look at this.

Comment 8

[Yuying Long]

Also, you can not see those ascii and even a lots of double-byte character in 
Page Source too.

Comment 9

[nhottanscp]

ylong, could you attach a screen shot?

Comment 10

[Yuying Long]

Attachment: missing ascii characters shows blank

Comment 11

[nhottanscp]

I cannot reproduce this on my local debug build.
Can anybody else reproduce?

Comment 12

[Yuying Long]

I noticed the last comment for bug 72525 by Ervin Yan:
> nightly 2001051622 on Solaris 9
> All the GB18030 characters can display OK, but all the English characters are
disappeared. 

Comment 13

[Jaime Rodriguez, Jr.]

Is this a beta stopper? Looks pretty bad to me, if we can't display alpha and
numerics on CJK pages. 

Bstell - If this is a beta stopper, pls assign a M0.9.1 milestone.

Comment 14

[ji]

This problem also happens when opening an ASCII text file on the browser and
changing the charset to CJK.
Although the ASCII string is not visible, you still can copy/paste the string
from browser window to a terminal window.

With environment varible NS_FONT_DEBUG set to 4, netscape outputs some debug
info to the terminal.
I'll attach the output when browser uses Japanese Shift-JIS charset to view a
plain ASCII text file.

Comment 15

[ji]

Attachment: The output from application with the debug environment variable turned on.

Comment 16

[kill this account]

Changed title from:

  Can not display alphabet and numbers on CJK page

to

  Invalid JIS 201 fonts on Redhat Japanese 6.2

Comment 17

[kill this account]

Redhat Japanese 6.2 has some fonts that have no glyphs. 

eg: -wadalab-mincho-medium-r-normal--16-*-*-*-c-*-jisx0201.1976-0

The font system tries to use these and ends up drawing blanks.
Because these fonts pretend to have ascii all ascii characters are blank
making the browser unusable for Japanese.

Comment 18

[kill this account]

Attachment: patch that tests for fonts without any glyphs

Comment 19

[kill this account]

Attachment: previous patch was incorrect

Comment 20

[Masaki Katakai]

Thanks Brian for taking this,

yes, I have mentioned this problem in the comments of bug 67732.
But it's great you can fix this.Thank you very much.

I haven't tried this patch yet but do you think this patch
can solve cns4-7 problem also? I'll try cns4-7 on our Monday.

Comment 21

[kill this account]

Attachment: patch with "risk management" addition

Comment 22

[Masaki Katakai]

Good news. I checked this patch for bug 71199 and
I found it works. The current patch is checking
only for jisx201 but I believe it will be generic.

I have one concern about performance. Is there
any way (by having a flag in nsGtkFont or by
having global hash table) to cache the result
of IsEmptyFont()? It seems that it checkes always
even when the font has been already checked.

Comment 23

[Masaki Katakai]

*** Bug 81734 has been marked as a duplicate of this bug. ***

Comment 24

[kill this account]

We need to be careful here.

What the patch does is try to detect what broken fonts 
look like. Clearly, there is no spec on how to correctly 
build a broken font. Thus, there is not a well defined 
way to detect them. This test works by checking if the 
per-char array has non-zero sized glyphs. These fonts
do have a valid min_bounds/max_bounds value even though
they don't have glyphs.

It is *quite* possible that a font (read: CJK 
font) does not have a per-char array as it was considered
not to have helpful information and malloc'ing it would 
be expensive at run time. This test would disable a font
like this.

We need to be careful of disabling valid fonts like this
(false positives).

Comment 25

[kill this account]

Attachment: patch: include invalid (Sun) cns11643; remember the test result

Comment 26

[Frank Tang]

r=ftang I think it is right to do risk managent and only do this checking for 
the fonts which we know have problem (JIS0x0210 and CNS4-7 ) for now.

Comment 27

[Christopher Blizzard (:blizzard)]

+PRBool
+nsFontGTK::IsInvalidFont(GdkFont* aGdkFont)
+{
+  // use these to make the result easier to understand
+  PRBool isInvalid = PR_TRUE;
+  PRBool isValid = PR_FALSE;

Why not make the function called "IsValidFont" so you can use PR_TRUE and
PR_FALSE and avoid the double negatives?

+        mDontUseThisFont = PR_TRUE;

Same here.  Using negatives as variables makes things confusing.  mUseThisFont
would be less confusing.

Fix those and you can have an sr=blizzard

Comment 28

[kill this account]

I cannot call it IsValidFont: the function does not determine that
the font is valid; only that the font is not valid. I'll rename it 
IsEmptyFont.

I'll also change mDontUseThisFont to mAlreadyCalledLoadFont.

thanks

Comment 29

[kill this account]

checked in

Comment 30

[Yuying Long]

Fixed verified on 05-22 linux trunk build.

Comment 31

[Roland Mainz]

OT: Can anyone test the patch in bug 108412 if it causes problems on Redhat
Japanese 6.2, please ? Thanks!