94397 - Folder charset is ignored when auto-detector is on.

Status: VERIFIED INVALID

(MailNews Core :: Internationalization, defect)

Description

[ji]

*****Observed with Ja rtm build*****

When auto-detector is on, folder charset is not used to display mail body.

Steps to reproduce:
1. Turn on the auto-detector on Japanese.
2. Go to a Chinese newsgroup, like alt.chinese.text
3. Select View | Folder charset to set the folder charset to GB2312 for this 
newsgroup.
4. Select a Chinese new article which doesn't have charset info in the body.
   The mail body is supposed to be displayed in GB2312 which is set as the 
folder charset for this newgroup, but it's not. It's displayed incorrectly.
It's displayed in a charset sniffed by Japanese auto-detector.
5. Turn off the auto-detector and reselect the same article, it will be 
displayed alright.

Comment 1

[ji]

Nominating for the next release.

Comment 2

[Roy Yokoyama]

== assigning to jbetak ===

Comment 3

[Katsuhiko Momoi]

This behavior is correct according to the spec
we have published.
 
Auto-detection applies to all bodies which lack
charset specification before the default charset is
used. 

If you use Chinese or Simplified Chinese auto-detector,
you get the desired behavior. Is this not the default
setting for people reading Chinese mail/news?

If you use "force the folder charset" option, you get the 
desired behavior.

The current behavior makes it possible to apply auto-detection
to attachments without charset info. If we follow your suggestion,
we probably will lose the ability to display Shift_JIS or EUC-JP
attachments in Japanese. Is that what we want?

Comment 4

[ji]

Yes, it's a problem for attachment display if we use folder charset first.
We can use "force the folder charset" option, I thought it's only used for the 
mails with wrong MIME charset info...
But assuming that I'm a Chinese reader living in Japan, I always turn on 
auto-detector on Japanese, but read Chinese mails/news sometimes. It could be 
confusing, although I can reselect autodetector or force the folder charset.
By the way, where is the spec published?

Comment 5

[Katsuhiko Momoi]

> By the way, where is the spec published?

This is not the most noticeable place but here it is:

http://home.netscape.com/eng/intl/docs/iuc17/mail/iuc17mail.html

Look at section 6.2.

The very thing that you want, to have the default folder charset
to come before auto-detection, also hurts cases in which
messages are sent from a form without charset conversion.

In languages like Japanese and Russian, the form charset 
tends to be different from the mail charset. In such cases,
having auto-detection is really convenient. In fact that is 
how I read feedback msgs to Netscape 6.1 JA version. Now if we
follow the folder charset, this will necessitate the user to
manually adjust the charset menu because the encoding used
is Shift_JIS.

Another important thought is that the default encoding is
always the last resort to fall back on. If someone has turned on
an auto-detector, it is a normal expectation to use that first.
This also is consistent with the way auto-detectors and default
encoding work on the browser side.

So, while there are cases which can be resolved, normal
cicumstances point to honoring auto-detection above the
default encoding. Reversing the hierarchy also hurts users
on more typical cases. 

For these reasons, I suggest not making this change. 

Comment 6

[ji]

Marked it as invalid based on the spec.
Can we have more simple explanation for "force the folder charset option" 
checkbox on the folder charset UI?
It's also used for the mails w/o MIME charset info in this case.

Comment 7

[Katsuhiko Momoi]

> Can we have more simple explanation for "force the folder 
> charset option" checkbox on the folder charset UI?

Good point. Can you suggest a draft of change in another/new
bug? Then we can refine the wording. Thanks.

Comment 8

[ji]

Filed bug 94560 for checkbox wording change.