45543 - [regression]Korean characters are displayed as solid upside down triangles

Status: VERIFIED FIXED

(Core :: Internationalization, defect, P3)

Description

[Alek Jeziorek]

-start browser with win32 commercial build 2000-07-14-08-M17 
-go to either of these sites or any other Korean site:
http://unicode.org/iuc/iuc10/x-utf8.html
http://home.netscape.com/ko
http://www.joins.com

-notice that the Korean characters are displayed as upside down triangles and 
not even ?????'s 
-on NT?
-even when going View|Character Coding|MultiByte|EUC-KR its still displayed the 
same

Comment 1

[leger]

Putting on [dogfood-] radar. 

Comment 2

[Teruko Kobayashi]

I can reproduce this on Winnt 4.0J.  I nominate this for beta2. 

Comment 3

[leger]

Putting on [nsbeta2+] radar.

Comment 4

[Katsuhiko Momoi]

This problem also affects mail messages. 
For some reason, the fonts assigned to Korean on a *new*
profile are: Times New Roman, Arial and Courier New. 

We have the default list for Korean and none of these
are on that list. 
There are other language categories for which the defaults are
wrong on a new profile, e.g. Thai, Unicode, and Traditional Chinese.
They all show the initial assignments like the above:

Times New Roman, Arial and Courier New. 

Under this condition, the font fallback mechanism (to find a glyph
in case the assigned font does not contain) seems to be failing.

Comment 5

[Katsuhiko Momoi]

As examples, under this failing condition, in addition to Korean,
you don't see Thai glyphs and Georgian glpyhs displayed. I could not
tell if Trad. Chinese display is missing is something but some glyphs
looked like they were taken from non-Chinese fonts.

The fallback mechanism continues to fail until the user explicitly sets
a font for at least one of the lang categories with wrong defaults showing. 
For exmaple, if you re-set the fonts for Thai, the display for Korean 
and Georgian also come back.

Comment 6

[Frank Tang]

Is this related to the 22031 erik check in this week ?
Erik check in the change 07/13/2000  14:08
and rods and rbs review the code. 

We should back out mozilla/gfx/src/windows/nsFontMetricsWin.cpp  to r=3.83 and 
see is the problem go away ?

erik is now on sabbatical so we need to cover for him. 
Shanjian, can you help ? We may need to back out 
mozilla/gfx/src/windows/nsFontMetricsWin.cpp  r=3.83 to fix this and reopen 
22031 till erik back. But please try the back out first. Thanks.

Comment 7

[rbs]

This looks like an upshot from Erik's check-in. I faced the similar problem with
MathML-enabled builds, and had to put back this hack in nsFontMetricsWin.cpp:

static int CALLBACK enumProc(const LOGFONT* logFont, const TEXTMETRIC* metrics,
  DWORD fontType, LPARAM closure)
{
#ifdef MOZ_MATHML
  // XXX need a better way to deal with non-TrueType fonts?
  if (!(fontType & TRUETYPE_FONTTYPE)) {
    //printf("rejecting %s\n", logFont->lfFaceName);
    return 1;
  }
#endif
...
}

You can try removing the #ifdef to make this part of the default build.
The advantage of this (as opposed to completely backing-out Erik's changes) is
that you still get the bitmap fonts necessary to display the Classic skin.

More specifically with this change, an author's font-family is honored, but
if no glyph is found, the fallback is restricted to only TrueType fonts.
However, the font pref will also only list these TT fonts, and will not
show other bitmap fonts that are on the user's system.

This appeared like a compromise (for MathML anyway) until a better general
solution is found when Erik returns. Erik and I were discussing about these
problems. So I recommend trying to remove the #ifdef, before backing-out
Erik's changes if it still doesn't work.

[Setting appropriate default font-family lists for new profiles is another
work-around, but they still won't work if none of the default fonts is on a
particular user's system. Sounds like a separate bug should be filed if the
default lists are empty/wrong.]

Comment 8

[Shanjian Li]

Backing out erik's fix does not seems like a good solution.
I think I have found out the real problem, I will try to find a 
fix.

Comment 9

[rbs]

What do you think is causing the problem?

Comment 10

[Alek Jeziorek]

still on win32 commercial build 2000-07-17-08-M17

Comment 11

[rbs]

Rebuilding the gfx DLL without the hack I indicated above, and visiting
http://home.netscape.com/ko, and http://www.joins.com, I see missing glyphs
displayed as upside down triangles.

Rebuilding gfx with the hack turned on, I see missing glyphs displayed as '?'.

Comment 12

[Shanjian Li]

What is the problem:
For all font charset values that are not easily (or possible) to 
get their cmap, default cmap is used. Default cmap marked all 
characters available, that is incorrect. 

My fix:
I did not let system generate a default cmap, instead, I 
let it return empty cmap, that will fix the problem. 
In future, if we want to handle some font charset value,
we can just add new functions to do this.

Comment 13

[Shanjian Li]

Attachment: proposed fix

Comment 14

[rbs]

Looks ok. r=rbs

Don't know what Erik intended here. The GenerateDefault() function was/is saying
that the font under consideration can represent all chars. Hmm.

Comment 15

[rbs]

Not quite accurate what I said above. The function is doing something else:

static void
GenerateDefault(nsCharSetInfo* aSelf)
{ 
printf("%s defaulted\n", aSelf->mName);
  PRUint32* map = aSelf->mMap;
  for (int i = 0; i < 2048; i++) {
    map[i] = 0xFFFFFFFF;
  }
}

Comment 16

[Shanjian Li]

fix checked in.

Comment 17

[Blake Ross]

fixing resolution

Comment 18

[Blake Ross]

marking fixed

Comment 19

[Alek Jeziorek]

no longer seen on win32 commercial build 2000-07-19-08-M17 verifying fix