148361 - Unnecessary font glyph substitution taking place

Status: RESOLVED WORKSFORME

(Core :: Internationalization, defect)

Description

[Michal 'hramrach' Suchanek]

On the page http://www.ms.mff.cuni.cz/index.html.cz.iso-8859-2 ecaron is not
rendered in the same font as the rest of the page.
I think this is a Mozilla bug because wrong fonts would likely produce a
different  character, not correct characer of different look.

Comment 1

[Michal 'hramrach' Suchanek]

Using 2002-05-30-05 Mac OS X 10.1.4 (Czech bundle for Mac OS 8.5)
IE 5.1.4 on the same machine renders the page correctly.
I suspect this is Mac-specific because 1.0rc2 on Linux renders the age correctly
as well.

Comment 2

[Greg K.]

Glyph substitution usually happens when a glyph is unavailable in the current font.

However, Times CE, which my Mozilla is set to use for Central European (ISO-8859
2) contains an ecaron character. Mozilla simply isn't using it for some reason.
It's pulling an ecaron from some other font. (And what's up with that extra
horizontal spacing?)

Thus confirming using FizzillaCFM/2002052305 (RC3). Editing Summary.

-->Internationalization (because anything involving other than US-ASCII
characters ends up there).

Comment 3

[Greg K.]

Attachment: Screenshot of affected page showing the problem.

Comment 4

[Greg K.]

Hm. Strictly speaking, those characters must be encoded as entities, anyway, and
HTML doesn't implement character names for ISO-8859-2 characters (see
[http://www.w3.org/TR/html401/sgml/entities.html#h-24.1]).

There is an evangelism angle to this bug, then. Mozilla isn't technically
expected to handle this since it's not valid. However, it's handling all the
other ISO-8859-2 characters, so why not the ecaron?

Comment 5

[Michal 'hramrach' Suchanek]

IMHO this is not an evangelism: If you specify iso-2 encoding of the document,
it is meant to tell the browser that the characters with 8th bit set should be
decoded as iso-2 (rendering the page correctly). Anyway, utf-8 recode of the
page renders the same way. I think it is  at
http://www.ms.mff.cuni.cz/index.html.cz.utf-8
The entity name problem is circumvented by using unicode numbers of the
characters in page with unspecified (or different) encoding such as ž
(zcaron) Ž (Zcaron) ě (ecaron) Ě (Ecaron) which renders the same
way (when the entities are used in an iso-1 page).

Comment 6

[Roy Yokoyama]

font substitution -> shanjian, cc ftang/nhotta

shanjian: let me know if you need my help with MacOS.

Comment 7

[Shanjian Li]

reassign to ftang, no idea about macos.

Comment 8

[Frank Tang]

do you have central european script installed in your macos x?

Comment 9

[Frank Tang]

we won't use QuickDraw script if the script bundle is not installed. 

Comment 10

[Greg K.]

Attachment: Testcase containing lowercase l and l+slash

Comment 11

[Greg K.]

Added a simplified testcase that fails on FizzillaCFM/2002071508 with Central
European script installed.

Experimental FizzillaMach+ATSUI build renders it great. I'll mark this dependent
on bug 121540 for now.

Comment 12

[Greg K.]

See also bug 150485.

Comment 13

[Greg K.]

*** Bug 158560 has been marked as a duplicate of this bug. ***

Comment 14

[Greg K.]

This isn't only ISO 8859-2. I've also seen it happen in Unicode pages and page
with Cyrillic text.

Comment 15

[Ignacy Niemczycki]

Attachment: A test page that shows what happens to Polish fonts in Mozilla for OSX

I have written a page with several ways of manipulating the way Mozilla should
display iso-8859-2 characters(in this case Polish). It's not an advenced
technique, but just using font face tags. The very first step is to change your
Mozilla preferences. For Western Fonts choose serif: Times, sans: Helvetica,
fixed:Courier; for central european choose serif:Capitals CE, sans: Sand CE,
fixed: Monaco CE;
You may also choose some diffrent fonts, but the point is to choose fonts from
diffrent families for western and for CE. If you don't have CE fonts, please
download them from Apple Poland:
http://mail.apple.com.pl/download/PL/Polskie_dodatki_do_systemu/Keyboard_Pl_OSX10.1.dmg.bin

Now just open an enclosed page: testpage.html What you will notice is that
Mozilla does some very strange things to display Polish characters. Let's go
step by step:
-In the first part of testpage.html Mozilla should use fonts specified in
preferences for Central European, but Mozilla does something else. Words that
don't contain Polish characters at all are displayed correctly using the font
specified in pref for CE(in this example it's Capitals CE). Words that contain
polish characters but no "ñ"("n" with dash) or "³"(looks like "t", but it's
slightly diffrent) or "£"(capital "³") or "ó" or "Ó" are displayed using a font
specified for Western in Preferences(in this example it's Times), but the
specific Polish character in this word are diplayed using the font specified
for CE(Capitals CE), please look at words like "¿aba" or "¯ABA" or "maæ" or
"MAÆ". Words that contain "³" or "£" or "ñ" are displayed using font for
western, but these 3 characters mentioned above are displayed using some
unknown font(they are too big, they use too big gap), but not any of the fonts
specified in Mozilla preferences. Words that contain only Polish characters are
displayed using font specified for CE, but "ñ", "³","£" are displayed using
some unknown font. Moreover, "ó" and "Ó" are treated like iso-8859-1 characters
and are displayed using a font specified for Western(in this example it's
Times).
I'm not gonna cover all of the possible examples, just look yourself, change
the preferences over and over and you will see the problem. To sum up Mozilla
for OS X does not handle displaying iso-8859-2 characters nearlly at all. It' s
just a big mess. I'm using Mozilla 1.1b 2002072203

Comment 16

[Ignacy Niemczycki]

Attachment: A screen shot of the testpage.html

Just a screen shot, if somebody has no idea what I am writing about.

Comment 17

[Ignacy Niemczycki]

I forgot to write that I have just reinstalled the system, so no font or program conflict is possible (maybe 
Macosx vs. Mozilla :) ).

Comment 18

[Henri Sivonen (:hsivonen)]

This is a duplicate.

*** This bug has been marked as a duplicate of 111728 ***

Comment 19

[Yuying Long]

Mark as verified.  Please re-open if disagree.

Comment 20

[Ignacy Niemczycki]

I would re-open this bug, it was marked as a duplicate of bug 111728, which is
now fixed. Yet still the unnecessary font substitution takes place. So this bug
is neither fixed nor a duplicate and I beg someone to solve it as it seems to be
a cause of Mozilla/Firefox not displaying polish characters well and this
problem has been going on for more than 2 years now...To put it in other words:
I love Mozilla, but I cannot use it because it makes pages written in my
language unreadable :-(
Moreover, I guess this bug is linked with bug 201817

Comment 21

[Henri Sivonen (:hsivonen)]

Right, there is still a problem, although the original screenshot suggested that
TEC was the culprit (and probably was at the time). Reopening. See also bug 165878.

Mozilla's text rendering on OS X is going to remain embarassingly bad until bug
121540 is fixed. I don't expect anyone to bother to fix the current gfx impl.
Everyone is just waiting for someone else to overhaul the Mac gfx. And that in
turn is pending changes in the layout/gfx interface for all platforms...

Comment 22

[Ignacy Niemczycki]

Just to comment on Henri Sivonen's last comment. I recently found out that
Safari's Debug menu (which can be activated using for example TinkerTool)
contains an item "Use Atsu", this made me think that actually Safari does not
use ATSUI by default (I'm not a font/ rendering specialist so if ATSU does not
equal ATSUI, then this whole comment is BS). However, Safari still is able to
display eastern european text correctly (and funny enough if the Use ATSU option
is enabled the text is not displayed correctly). So maybe Mozilla does not
really need ATSUI to display at least some "simple-languages" text correctly
(not Hind or Hebrew, but Polish or Czech). Are we really sure that Safari uses
ATSUI? Do we really need to use ATSUI in the short/mid-term? Moreover, please
see bug 158560, there is something really strange going over there, that may be
linked to this bug.

Comment 23

[Ignacy Niemczycki]

Guys, I requested this bug to be a blocker. This is why:
1. This problem makes Mozilla basically useless for the whole Central Europe (Mac only)
2. Any user will notice this bug immediately
4. It seems that it will be a long time until ATSUI is implemented, which is supposed to fix this bug
3. This bug affects all Mozilla Products: Mozilla/Firefox/Thunderbird/Sunbird....and is especially 
annoying on Thunderbird
4. It has been around for more than 2 years
5. My guess is, that the fix does not require that much work and certainly full ATSUI implementation is 
not the easiest option
What do you think?

Comment 24

[Frank Tang]

what a hack. I have not touch mozilla code for 2 years. I didn't read these bugs
for 2 years. And they are still there. Just close them as won't fix to clean up.

Comment 25

[Travis Chase]

Mass Re-open of Frank Tangs Won't fix debacle. Spam is his responsibility not my own

Comment 26

[Travis Chase]

Mass Re-assinging Frank Tangs old bugs that he closed won't fix and had to be
re-open. Spam is his fault not my own

Comment 27

[Daniel Fairchild]

Not sure if this is the same problem or not: many characters in the "Miscellaneous Symbols" Unicode block do not display correctly, but are instead displayed as other glyphs.  For example, the 'heart' character (♥), Unicode codepoint 2665 (UTF-8 E299A5) is rendered as a thick vertical bar.  This happens whether it is originally encoded as a Unicode character or as an HTML entity, so it seems likely that it is a font choice problem; however, changing the font settings to a font that includes the characters has no effect.  In addition, the characters are displayed correctly in the View Source window.  Confirmed as always reproducible on Mozilla 1.7.13 and Firefox 1.5.0.3 on Mac OS X 10.4.6.

Comment 28

[Chris Freeman]

I'm also getting peculiar characters in unexpected places. Firefox 2 versions display the characters normally, as does IE 6 and 7 and Safari. Viewing the source shows the normal, expected characters, but in the browser display some headings especially are quite odd and unreadable. It is quite consistent and always shows up on the cnn.com homepage, but just for the primary headline. Many other sites have the same issue, usually on the headings. Skiracing.com has its headings and most links and navigation show up as the same unreadable "font" that look like wingdings, math symbols, Yen sign etc.

Comment 29

[Simon Montagu :smontagu]

I can no longer reproduce this in trunk builds.