93274 - Cached charset is ignored when auto-detector is on

Status: RESOLVED FIXED

(Core :: Internationalization, defect)

Description

[ji]

**********Observed with 07/26 linux branch build****

When auto-detector is on, the cached charset for previous page is ignored.

Steps to reproduce:
1. Launch browser with a new profile or clear cach with a old profile.
2. Turn on auto-detector on Simplified Chinese.
3. Go to a page w/o charset meta tag  in different charset, like
http://www.kinokuniya.co.jp, which is a Japanese shift-jis page w/o charset meta
tag. This page is garbled as expected.
4. Correct the display by selecting charset menu to Japanese Shift-JIS.
5. Click on any link on this page
6. Click on Back icon to go back to the previous page. You will see the page is
garbled, the cached charset for this page is ignored.
7. Turn off auto-detector and clear cache, repeat step 3-6, you won't see this
problem.

Comment 1

[Yuying Long]

Shanjian, is this a same problem as 92632?

Comment 2

[Roy Yokoyama]

assigning to shanjian.
This may also related to 93269.

Comment 3

[ji]

Bug 93269 is for view page source window, and this one is for navigator window.

Comment 4

[Shanjian Li]

This is a known issue. We are putting auto-detection on top of cached charset. 
Considering such a scenario. If user see a page incorrect, and he realize that 
it is caused by incorrect charset, he choose a charset auto-detector to find the 
right charset. If cached charset is on top of auto-detection, he will always get 
cached charset and his newly selected detector won't work. 

Momoi suggested to let newly-selected detector take priority of cached charset, 
and let cached charset take priority of auto-detector in other situations. That 
makes some sense, but still not perfect. With same charset detector, users might 
get different result with or w/o cached charset, that will confuse certain 
users. So for time being, nothing will be done. 

Comment 5

[ji]

I would agree with Kat's suggestion, since the priority handling in the scenario 
in the original report doesn't make sense.

Comment 6

[ji]

I'd like to keep this bug open as a reminder until we have a perfect solution 
for most of the scenarios.

Comment 7

[Yuying Long]

On my comment at 2001-08-02 10:45, I put the wrong bug number 92632 there, and I
meant is this the same problem as bug 93017?

Comment 8

[Shanjian Li]

No, 93017 is a different problem. 

Comment 9

[hirata masakazu]

*** Bug 94043 has been marked as a duplicate of this bug. ***

Comment 10

[Jun MUTO]

If bug 94043 (Win98 bug) is a duplicate of this bug as hirata say,
OS should be changed to ALL.

Comment 11

[Jun MUTO]

As written in bug 94043, on Windows98SE, this bug has been reproduced 
on 2001-08-04-11-trunk (win32) build or later.
2001-08-03-10-trunk (win32) or earlier builds didn't have this bug.

Comment 12

[hirata masakazu]

 Changing Platform and OS to All/All, since this is happening in Linux and Mac
builds.ts

Comment 13

[hirata masakazu]

Actually, the current status is that even with shift-JIS meta tag, charset for
the cached page gets ignored.
If this behavior is not covered by this bug, I think I should reopen bug   94043.

Comment 14

[Katsuhiko Momoi]

We are currently not behaving as was defined in the 
Browser Charset document here:

http://www.mozilla.org/projects/intl/uidocs/browsercharmenu.html

The section which concerns this issue is:

Semantics | Current Encoding/Charset

In particular, I wrote:

"* The above algorithm will not be used when the users surf back to pages
already visited. We reply on cached encoding info in such cases."

In other words for any "new" pages that have not been visited 
or for any occasion when the Character Coding menu or 
Auto-detect menu has been engaged, the algorithm is 
applied, whatever has been changed will rank above
the cached charset. In other cases, e.g. when displaying a page that
has been already visited during the current session, the 
cached charset is used. 

As Hirata-san mentions, this is not being followed now.
Shanjian, if you have cases that would cause problems, please
mention them here. We should see if not fixing this problem 
would be worse than te problem you think will occur.

> Actually, the current status is that even with shift-JIS meta tag, 
> charset for the cached page gets ignored.

Hirata-san, this could be a different bug. Can we discuss this issue
in another bug? I know that Mozilla currently has a problem with
pages that have a meta-tag when auto-detection is ON.

Comment 15

[hirata masakazu]

OK. Following Momoi-san's suggestion, I should take the current problem
separated from the original discussion.

As I can not yet find a likely bug entry that describes the problem in cached
pages with meta tag, I am going to reopen the bug 94043 with modified summary.

I can confirm that the problem with cached S-JIS page regardless of meta tag
started between 0803 and 0806 builds, which are the available builds for Mac. As
jmuto-san describes above, the problem seems to have started in 0804 build.

Comment 16

[Shanjian Li]

As we once talked over the phone, the biggest problem is how to implement this. 
We have to let auto-detect override cached charset when it is newly selected. I 
need to do some investigation to find out a way to do this. 
One the other hand, I still believe there is certain side-effect associate with 
this solution, though it is the best we have proposed at this time. In UI 
design, status (or history) will alway confuse users. Considering such a 
scenario, a user select no auto-detection and visit a webpage, and the webpage 
is not displayed correctly because there is not meta charset specified, and 
user's default charset is not right either. Instead of getting it right, (s)he 
just walk away. While visiting other pages, an auto-detection is enabled. When 
the user visit the previous page again, the page will still be displayed wrongly 
because autodetection is not  newly selected. So to me, improve auto-charset 
detection's accuracy is more important that handling this issue. 

Comment 17

[Shanjian Li]

Mark this one for future now, I will try to look into this (and several other 
related problem) sooner. 

Comment 18

[Shanjian Li]

*** Bug 156367 has been marked as a duplicate of this bug. ***

Comment 19

[Koike Kazuhiko]

WFM now with 2002100108-trunk/Linux.

Comment 20

[Max Alekseyev]

Linux build 2004020607
Still see the following instructions in dupped bug 156367
reopenning...

Comment 21

[Mike Cowperthwaite]

*** Bug 191247 has been marked as a duplicate of this bug. ***

Comment 22

[Frank Tang]

shanjian is no longer working on mozilla for 2 years and these bugs are still
here. Mark them won't fix. If you want to reopen it, find a good owner first. 

Comment 23

[Max Alekseyev]

I don't see correllation between existence of the bugreport and missing owner.
Many bugreports in the database have no owner at all. So this bug as many others
can wait for the "good owner" in open state.
Reopenning...

Comment 24

[Travis Chase]

Mass Re-assinging Frank Tangs old bugs that he closed won't fix and had to be
re-open. Spam is his fault not my own