104302 - mozilla.org homepage appears in UTF-8 with multiple profiles configured

Status: VERIFIED FIXED

(Core :: Internationalization, defect, P2)

Description

[Peter Janes]

With more than one profile configured, the mozilla.org homepage is detected as
having a UTF-8 character coding instead of the default.

To duplicate:
- visit mozilla.org and confirm that View/Character Coding is set to Western
(ISO-8859-1)
- create a new profile
- exit the Profile Manager immediately after creation, before starting the
browser with an existing one or the new one
- restart Mozilla, choosing the original profile
- visit mozilla.org and confirm that View/Character Coding is set to Unicode
(UTF-8).  (This will probably be obvious because the fonts will be different; on
my Linux box I get a large misc-fixed, even though I have different fonts
selected for UTF-8).
- exit Mozilla
- remove the new profile using Profile Manager; exit immediately
- restart Mozilla (Profile Manager will be skipped)
- visit mozilla.org and confirm that Western (ISO-8859-1) is used as the
character coding.Linux nightly trunk build 2001100908, -sea distribution, on
fully-RPM-patched Red Hat 7.1 (kernel 2.4.2-2, XFree86 4.0.3).  This behaviour
occurs independently of proxy settings, but if it helps I'm running against
squid version 2.2.STABLE4.

Comment 1

[Frank Tang]

it seems related to 102026

Comment 2

[Roy Yokoyama]

Attachment: Use window._content.document instead of focusedWindow

Comment 3

[Roy Yokoyama]

Brian: can you duplicate this bug on Linux?  I have a similar bug on Windows and 
the patch should fix this bug.  Can you verify?  Thanks

Comment 4

[Andreas Becker]

qa -> ylong.

Comment 5

[Yuying Long]

Sounds similar with bug 102026.

Comment 6

[Peter Janes]

I don't think it's the same as bug 102026--that one deals with the menu item not
matching the charset being used for the page.  Here the menu item and the
charset being used match, but they're both incorrect.  That is, the behaviour is
the same as if I used |View/Character Coding| to set Unicode encoding on that page.

Comment 7

[Frank Tang]

shanjian- can you help to do the verification?

Comment 8

[Roy Yokoyama]

This may be assigned to jbetak.
-> jbetak

Comment 9

[Katsuhiko Momoi]

Question:

What happens if you take the steps to reproduce the problem on Mozilla.org,
and then you try to change Viewe | Character Coding to something like
ISO-8859-1?
After making the change, go back to see if the menu still says UTF-8 or 
truly changed to ISO-8859-1. 

In my case, when this problem occurs, I am not able to change the menu 
to something other than UTF-8. Also changing the menu does not reload
the window.

You can also set your start up page to be "blank" instead of real URL.
And when I do, I consistently get UTF-8 as the startup page encoding
even though I set the default encoding to Shift_JIS Japanese.

From the facts I have seen, we might be defaulting to UTF-8 when 
there is no window content loaded, i.e. blank.

And the problem being reported here may be because the code does not 
think that there is any content loaded into the window and it defaults
to UTF-8. A typical symptom of this is that you cannot change the menu 
encoding. 

Comment 10

[Peter Janes]

I'm able to change the encoding to ISO-8859-1 from UTF-8, and the page is
redisplayed properly.  Also, this is just from navigating to mozilla.org--my
home page is on a local host.  (The behaviour doesn't change if the homepage
encoding is set to UTF-8 or ISO-8859-1, BTW.)

Also, other pages at mozilla.org are fine, it's just the homepage that's
displayed incorrectly.

Comment 11

[[e-mail] (away - not reading bugmail)]

OK I think I found the culprit - since Peter is using a Mozilla build, a link 
to mozilla.org is built-in. Even when he creates a new profile, mozilla.org is 
going to be included in his default bookmarks.html. When the page finishes 
loading the first time, the checkmark might be misplaced due to bug 102026 and 
this resulting charset will be automatically cached in the bookmarks, since 
mozilla.org is contained there. 

Next time he starts the app we will used said (and incorrect) charset from his 
bookmarks as a hint and since Mozilla's homepage doesn't supply any charset 
information (is this correct?) we will actually finish loading the page using 
that misleading hint. The page will henceforth always load using the wrong 
charset, unless he overrides it manually and thus resets the cached bookmark 
charset or deletes the page from both bookmarks and browser cache.

I'm not 100% sure, but it seems reasonable to me and I have been seeing this 
problem myself. I have a patch rolled up for bug 102026 and Peter is welcome to 
try it in his build. When testing, it is essential to remove mozilla.org from 
bookmarks.html and to clear (or turn off) browser cache.

Comment 12

[[e-mail] (away - not reading bugmail)]

ohh - now this is getting really interesting. I'm attaching a default 
bookmarks.html file generated by a recent Mozilla trunk build. It's from a 
profile that has never been used for loading any page.

Please not the liberal use of "UTF-8" in the "LAST_CHARSET" attribute. I'll try 
to identify who and why is generating these in the default bookmarks file and 
attach a patch.

Comment 13

[[e-mail] (away - not reading bugmail)]

Attachment: default bookmarks.html file generated by Mozilla

Comment 14

[[e-mail] (away - not reading bugmail)]

Attachment: got patch - nhotta, yokoyama - could one of you guys review this?

Comment 15

[Peter Janes]

Unfortunately I don't do my own builds, so I can't use this patch myself. 
However, I have noticed that in the 2001102308 build, I don't get the incorrect
fonts any more.

I will try to remove the mozilla.org bookmarks (I have two, for some reason). 
However, I have overridden the charset previously and it hasn't been re-cached,
either later in the same session or after restarting--both bookmarks still show
LAST_CHARSET="UTF-8".  (Unless you're saying I need to re-bookmark mozilla.org
once I've reset the charset?)

Comment 16

[[e-mail] (away - not reading bugmail)]

>I have overridden the charset previously 

manual override should fix these issues and "correct" the cached charset hint 
in both bookmarks and the browser cache. This was one of the design goals and I 
believe it worked at one time. If this is not the case it probably has 
regressed and we should file a new bug.

Peter, please try this: pick one of your profiles (or create a new one) and 
then use the override approach first and see what the results are. Then 
physically remove the browser cache from the profile directory and change 
the "UTF-8" charset to "ISO-8859-1" in bookmarks.html with a text editor.

Comment 17

[[e-mail] (away - not reading bugmail)]

Peter - seems like Mozilla trunk builds from 10/23 and more recent have some 
other and unrelated issues with the charset menu due to a landing of some 
significant code changes. 

We are addressing them right now, but they might interfere with the efforts to 
isolate the causes and a possible remedy for this particular bug. I'd recommend 
using builds before 10/22 for testing and verification.

Comment 18

[Peter Janes]

OK, here are the results I've gotten:

Manual override of bookmark: no change.  I started Mozilla, navigated to
mozilla.org via the bookmark, changed to ISO-8859-1, and exited Mozilla. 
Grepped bookmarks.html for mozilla.org and saw it was still UTF-8.  Started a
second time and saw the same behaviour.

Manually changing bookmarks.html to ISO-8859-1 (without removing cache): correct
display.

Removing cache (after changing bookmarks.html back): original (broken) behaviour.

So I'm guessing the "new bug" option is the way to go, because it looks like
manual override is broken.

Comment 19

[Peter Janes]

Also, for what it's worth, using the "UTF-8" bookmark to get to mozilla.org,
then re-bookmarking with "ISO-8859-1", correctly saves the new charset in the
bookmark.

Comment 20

[[e-mail] (away - not reading bugmail)]

>So I'm guessing the "new bug" option is the way to go, because it looks like
>manual override is broken.

Peter - thanks for testing this - I really appreciate your feedback! Please 
feel free to file a bug against the "International" component for the charset 
override issue.  I believe it's a result of the changes that landed on 10/22, 
but we'll have to see about that. 

From your review of the problem I'm quite certain now that we need to change 
the default bookmarks file and address bug 102026.

Roy:

could you please review the proposed change to bookmarks.html? I'd really 
appreciate it.

Comment 21

[Roy Yokoyama]

Comment on attachment 55199 [details] [diff] [review]
got patch - nhotta, yokoyama - could one of you guys review this?

/r=yokoyama

Comment 22

[[e-mail] (away - not reading bugmail)]

thanks Roy! Now if I could only receive sr in time for M095.

Blizzard - could you help? This is a patch for bookmarks.hmtl, strictly i18n-
related and w/o much impact on anything else :-)

Comment 23

[Christopher Blizzard (:blizzard)]

Comment on attachment 55199 [details] [diff] [review]
got patch - nhotta, yokoyama - could one of you guys review this?

sr=blizzard

Comment 24

[[e-mail] (away - not reading bugmail)]

fixed - thanks everyone!

Comment 25

[Yuying Long]

I can not reproduce it on 11-14 trunk build, mark it as verified.  Please
re-open if still see it.