234958 - Reply ignores the default send charset and uses the charset of a message being replied to.

Status: RESOLVED FIXED

(MailNews Core :: Internationalization, defect)

Description

[Jungshik Shin]

A few days ago, I found that the default send charset is ignored and the charset
of a message being replied to is used for 'reply'. This might be regarded as a
feature, but not everyone likes this. 

It can even lead to an interoperability problem. For instance, a lot of Korean
emails come with 'ks_c_5601-1987' as the MIME charset (this is a non-standard
name used by 'evil' MS) for Windows 949 of their own invention (that has nothing
to do with the Korean standard with that name). The __internal__ canonical name
for Windows-949 is 'x-windows-949' (because MS never bothered to register it
with IANA). Since 'x-windows-949' is not registered with IANA, it's not listed
in the  charset list in the mail composition window so that users are not able
to select it. However, when replying to an email with 'ks_c_5601-1987', Mozilla
generates an outgoing message with 'x-windows-949' label. This should be avoided
. There are two ways to fix this. One is special-casing 'x-windows-949' (mapping
it to its 'standard cousin' EUC-KR) and the other is to honor the default send
charset (which can't be x-windows-949) or to add a pref. entry to use the
default send charset.  


Some Japanese people (mindful of the standard) may be surprised to find that
they're generating outgoing emails in Shift_JIS/EUC-JP while replying to an
email in Shift_JIS/EUC-JP even when their default send charset is set to
ISO-2022-JP.

Comment 1

[Katsuhiko Momoi]

We debated this issue a lot when we implemented charset reflection
feature in mail. Overall it's working well. For Japanese, it's very rare
to receive msgs marked with EUC-JP or Shift_JIS. More often than not,
mangled msgs are in iso-8859-1 (for Japanese) instead.

I would like to suggest 2 possible ways to improve on this behavior:

1. Special case the Korean problem and create an appropriate
   response to "ks_c_5601-1987" labeled msgs. (Do this unless
   the option 2 below is checked.)
2. Create a toggle box to apply the default charset to any msg
   in reply. (This should be next to the default encoding setting 
   dialog.) Keep the current behavior as the default.

For most people, the current behavior should be OK. However, to counter
MS problems, we should do something appropriate, e.g. turn 
"ks_c_5601-1987" mail into "euc-kr". And at the same time, offer an
option to users to force the default charset unless
the original msg contains characters outside the normal coverage of
that default charset.

BTW, we should tell users that when non-default charset is used,
that charset name is shown at the top of the Compose window. If
the default is used, this name is not shown. This should give users 
a good clue.

Comment 2

[Jungshik Shin]

Attachment: patch

Kat, thanks for the help. I implemented most of what you proposed.
'x-windows-949' is special-cased and replaced by EUC-KR. I added
'mailnews.reply_in_default_charset' preference entry. The default value is
false. With that set to true, 'default_send_charset' is used instead of the
charset of a message replied to. I haven't yet added a front-end / UI for the
pref. entry. (about:config can be used as is the case for other entries) We may
add it later.

David and Scott, can you take a look at the patch?

Comment 3

[Jungshik Shin]

Comment on attachment 141818 [details] [diff] [review]
patch

I'd better put 'special-casing' of 'x-windows-949' _after_ checking
'reply_in_default_charset'.

Comment 4

[Jungshik Shin]

Comment on attachment 141818 [details] [diff] [review]
patch

asking for r/sr.

The following if-clause (for special-casing x-windows-949 ) will be put after
(not before) 'reply_in_default_charset' checking.

+      if (charset.Equals("x-windows-949",
+	     nsCaseInsensitiveCStringComparator()))
+	 charset = "EUC-KR";

I'll remove two debug statements as well.

Comment 5

[Jungshik Shin]

Thanks for r/sr. It's checked in.

Comment 6

[Stefan Borggraefe]

*** Bug 224262 has been marked as a duplicate of this bug. ***

Comment 7

[Mike Cowperthwaite]

Bug 263797 seems to be about this same problem for the inline-forward case.