131401 - The detection result of Universial detector got result of latin2 for latin 1 page

Status: VERIFIED DUPLICATE of bug 115114

(Core :: Internationalization, defect)

Description

[bruno lapeyre]

The font display problem already reported by myself inside #123997, is remaining
under the version 0.9.9 of Mozilla but without console log auto-opening occuring
when I quit the app.
while entering some text in a versiontracker form : the text was written far
from where the mouse cursor was and the font had changed to a serif sans in the
box when a serif was my default setting in the Preferences.
This occured in the following circumstances : 
- Mozilla was open
- then I launch a software called 3DOSX;
(http://www.acm.uiuc.edu/macwarriors/eoh2k2/3dosx/index.html) and play with it
for a while;
- I quit 3DOSX and try to post my comment at versiontracker.com
Thezn the problem occurs (it never did before, for 2 days while using the 0.9.9
version and posting comments at versiontracker.com)
Then I play with the "Appearance"->"Fonts" preferences.
Western was selecting at the top. I changed this setting to "User Defined" use
the "OK" button and then revert to "Western" and use the "OK" button again.
This was the solution !

Comment 1

[Boris Zbarsky [:bzbarsky]]

Font stuff -- to intl.

Comment 2

[Yuying Long]

Is there any possible that you can attach a screen shot for the problem? thanks! 

Comment 3

[bruno lapeyre]

Attachment: Screenshots

Hi,
The issue just came back a few minutes ago on my iMac DV 400 (256 Mb of RAM).
So I made some screenshots.
Very amazingly the bug was appearing inside only one field this time. Inside
the versiontracker "Search" field from the Mac OS X home page only and not
inside the same field found on some other pages...
Yes, only one field from only one page was affected...
Very very strange.
I took one screenshot using upper cases and one other using lower cases to help
you understanding what font was displayed (I can't recognize that font...) and
what was happening.
Using uppercase you will see (fontbugUpp.png) that the cursor is far on the
right from where it should be (normally the cursor should touch the right
border of the "M" : I didn't type any space after the "M"). The space between
the last letter and the cursor will change depending on what letter is typed.
I discovered that some letters weren't producing the same issue (it could help
you, I hope to identify the problem and the font...)  : typing only some "C" in
the the field and the cursor will always be touching the last "C". The same
thing will be true with the letter D, H, I, K, M, N, O, Q, T (see tletter.png
screenshot), U, V, X, Y. And I can mix these letters without encountering any
gap in the cursor location.

This won't work with A (A is the only capital letter making the cursor go to
the left - see aletter.png screenshot), B, E, F, G, J (see jletter.png
screenshot), L, P, R, S, W, Z.

You will have probably noticed one strange phenomenon inside these screenshots
: evrything is acting as if no space between letters were available. The
xletter.png screenshot will illustrate this point again.

Using lower case (see fontbugLow.png) the problem is the same but with
different letters :
a (see alowletter.png), c, e, s, v and y are the only letters that I can use
without encountering a cursor gap.

The problem disappeared very easily this time : opening
Preferences->Appearance->Fonts	(then I had to wait for 6 or 7 seconds before
to see the fonts settings - see MozPreferences.png) and using the "OK" button
(without changing any setting before) solved the problem.

You will also find the complete list of my fonts : fontlist.txt

I hope this can help...

Two days ago, I found what seems to be another bug but, before posting anything
to Bugzilla, I prefer to send you a screenshot of the issue, thinking that this
bug could maybe help you to understand the previous one. The screenshot
"newbug.png" will show you what happened while downloading a very large file
(more than 100mb - Giants game demo) when I tried to create a pdf file (using
the pdf output of the print command) from a web page.  I had to force-quit
Mozilla then.

Thanks.

Comment 4

[bruno lapeyre]

The attachment is a .sit file. Please use StuffIt Expander to open the
downloaded file.

Comment 5

[Yuying Long]

Thanks very much for the attachment, I'm confirming this bug per the attached
screen shots.

I have similar problem with when I set defult charset to Hebrew.
What's your default setting character coding (encoding) in Preferences?
Also I think it might related to bug 128394.

Comment 6

[bruno lapeyre]

Western 8859-1 has always been the default setting inside the Preferences before
encountering this bug and I have never used the View menu to change any coding
setting but just to display Source code.
Yes bug 128394 and this one could probably be related. 
First step : I recognize what font was appearing on my screenshoots : Lucinda
Grande. Isn't it the system font ;) Then it could help.
Second step : trying to look at versiontracker source code using Mozilla, I had
a great surprise... The whole source code was displayed using Lincida Grande ! I
will post a screenshot right now).
Then, looking at "View"->"Character coding" I discovered that everything was now
set to "Central European (ISO-8859-2)" when it shouldn't (another screenshot
will be sent).

So :
1) displaying everything using the system font when ISO-8859-2 encoding is
selected isn't normal I guess. For what I saw this problem can be reproduced
with many web page, including Mozilla home page for example (you will find
another screenshot...)
2) for some reasons, encoding can change by itself.

Comment 7

[bruno lapeyre]

Attachment: coding have changed by itself to 8859-2 - Lucinda Grande is used

Comment 8

[bruno lapeyre]

Attachment: Coding changed to 8859-2 affects Mozilla home page and source code display

Comment 9

[Yuying Long]

> For what I saw this problem can be reproduced with many web page, 
> including Mozilla home page for example

mozilla.org is a page without charset meta-tag page, so encoding for those kind
of page can be affected by many previous pages or some other charset settings.

Seems like when charset is iso-8859-2, we have problem with display:
1. Cursor position is different with the actual input postion in text field.
2. Overlapped in page source.

-> nsbeta1, may potentially affect a lot on central European or bidi. 

bug 119861 might related to problem here too.

Comment 10

[bruno lapeyre]

> 1. Cursor position is different with the actual input postion in text field.
> 2. Overlapped in page source.

Yes but I insist too on the fact that system font (Lucinda Grande) is used then.
Why system font would be called and used to display everything when 8859-2 is
selected ?
Is Mozilla having some peculiar connections with Mac OS X system font somewhere?
Answering to that question could probably lead to a beginning of solution I
guess... 
I am not a specialist and my idea could be stupid, but don't you think that the
cursor gap could be explained by an ATSUI rendering story? The cursor gap, for
what I saw, is only happening when the system font is in use -for some actually
unknown reasons. Then couldn't the system font be expecting an ATSUI rendering
when Mozilla would continue to use a different rendering ?

Comment 11

[nhottanscp]

Reassign to ftang. Also need his input for nsbeta1 triage.

Comment 12

[bruno lapeyre]

For what I saw bug will always be occuring when 
1) View->Character Coding->Auto-Detect is set to "Universal"
2) a page without charset meta tag is opened
So 8859-1 won't be used anymore but 8859-2, even if then :
View->Character Coding->Auto-Detect->Customize is just displaying "Western
(ISO-8859-1)" in the "Active Character Codings:" area.
Something seems to be broken in the "Universal" settings from Auto-Detect.
Setting Auto-Detect to (Off) will solved the problem.

Comment 13

[Frank Tang]

could you put down step-by-step reproduce procedure here and make screen shot
with it? I cannot reproduce it with the current description.

Comment 14

[bruno lapeyre]

Attachment: 11 step by step screenshots reproducing the bug (StuffIt Expander needed to open the attachment)

These screenshots are showing Mozilla switching the encoding without good
reasons, from 8859-1 to 8859-2 even if preferences are set to use 8859-1 by
default. At this time, the bug didn't appear inside the search box from
versiontracker web site but I am using a new build since yesterday : 2002040108
and this could explain. But the bug will be found inside some other
versiontracker pages, selecting a software name-link from the Mac OS X home
page. The bug is also appearing when reaching some MacCentral page (following a
link from the left margin of VersionTracker Mac OS X home page).
The system font, Lucinda Grande, is used to display the text of the pages
affected by a bad encoding (pictures 5 and 10) but not the default fonts
selected in the Preferences (picture 2). The presentation of the source code of
these pages is affected too: Lucinda Grande is used again and the gaps founds
while entering some text in forms, using previous versions of Mozilla, can be
seen in the bad spaces found between characters inside the displayed source
code (pictures 11 and 12). Even the Page Info window seems to be affected by
what could be a similar problem (picture 13).
Amazingly, selecting by myself 8859-1 encoding for an affected page using the
View menu and then switching back to 8859-2 by myself, always within the View
menu, won't give a so bad result inside the Page Info window and produce
another effect: then closing the Page Info window and opening the source code
page, the correct font (the one chosen in the preferences) will be used to
display the source code. Amazing because this won't happen if I don't use the
Page Info window before: if I don't open and close that window before asking
for the source code to be displayed, the source code will be displayed using
the system font (as shown in pictures 11 and 12...).

Comment 15

[Frank Tang]

first of all, please do not attach a stuffed thing to the attachement, please
attach your attachment one by one
second , please put down step-by-step reproduce procedure, something line
1. do x
2. do y
3. do z
expect result:
show "a"
actual result:
show "b"
screenshot of actual result
screenshot of expect result in other brwser

Comment 16

[bruno lapeyre]

Attachment: 1. My Preferences->fonts settings

Comment 17

[bruno lapeyre]

Attachment: 2 - My Preferences->Language settings

Comment 18

[bruno lapeyre]

Attachment: 3 - My encoding settings

3 - The encoding setting is correct for http://www.mozilla.org/start/ (I have
just launch Mozilla and the startup page is that one)

Comment 19

[bruno lapeyre]

Attachment: 4 - connecting to versiontracker.com

4 - I use a bookmark to reach versiontracker's Mac OS X home page :
http://www.versiontracker.com/macosx/
Everything is normal then. 8859-1 is used as set in Preferences->Language

Comment 20

[bruno lapeyre]

Attachment: 5 - following VT link, MacCentral page displayed with bad encoding (8859-2)

in the picture 4, the mouse cursor was over the link "'Executive Look at Mac OS
X' to be held today". Selecting this link we are now discovering a MacCentral
page using a bad encoding : the encoding changed by itself to 8859-2 without
known reasons. This link isn't displayed anymore over VersionTracker website.
But you can have the same result selecting, for example, today's link : " Apple
 , Adobe to host 'driven by Design' seminar" to reach
http://maccentral.macworld.com/news/0204/05.seminar.php were the same encoding
problems will appear.

Comment 21

[nhottanscp]

The third screen shot shows universal detection is selected. There is a bug in
the unversal detector.
bug 115114 - autodetect universal detects french as Central European (ISO-8859-2)

Comment 22

[bruno lapeyre]

Attachment: 5 bis - back to versiontracker.com and then selecting another link switch encoding to japanese

5 bis : I came back to VersionTracker home page using the "Back" button. Then I
checked its encoding: nothing was wrong. 8859-1 was used to display the page.
Then selecting today's VersionTracker page links, I found a new encoding
problem: encoding can switch by itself to japanese, and not only 8859-2.
just select the link "Apple DVD Studio Pro 1.5" from VersionTracker Mac OS X
home page (I already posted the address) and you should have the same result
than shown on my screenshot (be carefull, the encoding changes are not
automatically displayed in the menu. Sometimes I need to wait for a few seconds
or to close and then reselect the menu to see the new encoding checked).
Other amazing point: the previous screenshots were made yesterday and no
japanese encoding item was then appearing in the menu... It is today...

Comment 23

[bruno lapeyre]

Attachment: 5ter - selecting another VT link is displaying a page using the 8859-2 encoding

5ter - this screenshot was taken today again. Right after 5bis, I used the
"back" button again, found back the VersionTracker Mac OS X home page
(displayed with 8859-1 encoding again) and selected the link "Dock Extender -
1.3". The page displayed, as shown in the screenshot, is now using the 8859-2
encoding...

Comment 24

[bruno lapeyre]

Attachment: 6 - 8859-2 encoding again while following another link from VersionTracker

6 - this screenshot was taken yesterday but the result are the same today
(except that japanese encoding item wasn't appearing in the menu yesterday...).
Just select the link "Deneba Canvas - 8.0.3" from VT Mac OS X home page. Then
encoding is switching to 8859-2.

Comment 25

[bruno lapeyre]

Attachment: 7 - then selecting View->Page Source is diplaying the code source using the system font

7 - (screenshot taken yesterday but the result is the same today) right after
the screenshot number 6, I selected "Page Source" from the "View" menu and
discovered that the source code of page 6 was displayed using the system font,
Lucinda Grande, even if that font isn't selected in my font settings (that was
used to display the page because of the bad encoding, I forget to mention this
in my comments concerning number 6). You will discover abnormal spaces between
characters. Similar abnormal gaps and use of Lucinda Grande were already met in
the versiontracker Mac OS X home page "Search" box when I was trying to enter
some text using previous versions of Mozilla (as I already explained, this bug
seems to have disappear since I moved to 2002040108 build).

Comment 26

[bruno lapeyre]

>bug 115114 - autodetect universal detects french as Central European (ISO-8859-2)

The two bugs could have links. But I am reporting some other things :
1- the problem doesn't only appear with pages using French language.
VersionTracker and MacCentral are not using French at all;
2- japanese encoding can be used too and not only 8859-2;
3- system font, Lucinda Grande, is used to display the whole source code when,
for what I saw inside #115114 attachments
(http://bugzilla.mozilla.org/showattachment.cgi?attach_id=61723 for example)
system font is only partially used to display source code of pages using French
language (interesting, I guess again : why the system font, Lucinda Grande,
would be involved inside page and source code display? I can't find a reason to
this because Lucinda Grande is not set in the Preferences->Fonts. To understand
why this is happening could be a key to the solution... Under old versions of
Mac OS, I would think that a such thing could be related to a memory problem,
the system font being used because not enough memory was available to load and
use other fonts to display something... But under Mac OS X, I really have no
idea...)

Comment 27

[nhottanscp]

If there are any font problems then we want to isolate them from the universal
detector problem.
Could you try setting auto-detection to "Off" and see you still can reproduce
your problems?

Comment 28

[bruno lapeyre]

Setting auto-detection to "Off" is giving excellent results only if the disk
cache has been cleared. Without clearing the disk cache this setting wouldn't be
efficient for the already visited pages: each page using a bad encoding would
continue to use this bad encoding even if you use the "Reload" button. 

If forget to mention that the same bug can be found on my two machines.

Comment 29

[nhottanscp]

Could you put the result after turned off the auto-detector and cleared the cache?
Which problem still exist after turned off the auto-detector and cleared the cache?

Comment 30

[bruno lapeyre]

Attachment: Iso-8859-1 encoding just disappeared from the preferences->Languages setting

> Which problem still exist after turned off the auto-detector and cleared the
cache ?

No problem for what I saw. 
But later I tried to apply another solution that had previously been working:
changing fonts settings and language settings and then applying the previous
settings again. But trying to do that, I discovered that the 8859-1 setting had
disappeared from the Preferences->Languages list of encoding. Western is set
inside the Preferences->Font window. I only can find the 8859-1 encoding inside
the View->Character Coding menu...

Comment 31

[bruno lapeyre]

I found back the 8859-1 encoding! The scroller didn't wan't to appear in the menu. 
I had to quit and to launch Mozilla again to find it back. The problem is
happening again, after 3 opening-closing of the Preferences->Language window. 
Anyway, what seem to be a solution in previous build isn't working anymore with
the new buid. So setting auto-detect to "Off" seems to be the only solution.

Comment 32

[Frank Tang]

Change the subject from "Font display bug in forms : corrupted preferences ?" to "
The detection result of Universial detector got result of latin2 for latin 1 page"

reassign to shanjian
nsbeta1- because universial detector is turn off by default 

Comment 33

[Shanjian Li]

*** This bug has been marked as a duplicate of 115114 ***

Comment 34

[Yuying Long]

Mark as verified base on only happened when auto-detect All.

Please open a seperate bug if still see some other problems.