29592 - Reply uses charset from body for subject

Status: VERIFIED FIXED

(MailNews Core :: Backend, defect, P3)

Description

[Richard Zach]

Non-ASCII characters in the subject line will be converted to the charset of the
body on reply.  They should stay as they are.

To reproduce:
1. Send an empty message to yourself, with an 8859-1 character in the subject.
The body will get charset=us-ascii, e.g.,:

Subject: Test: =?ISO-8859-1?Q?=C4?=
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

2. Get Msg
3. Reply

Actual result: The charset of the non-ASCII characters in the subject line will
be encoded as us-ascii instead of ISO-8859-1:

Subject: Re: Test: =?us-ascii?Q?=C4?=
References: <[e-mail]>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

Epected result: Subject line taken verbatim from original message.

Linux build 2000.02.27.09

Comment 1

[Phil Peterson]

Reassign to ducarroz, cc rhp.

Comment 2

[Katsuhiko Momoi]

Both rhp and nhotta worked on this. So cpoying nhotta also.

Comment 3

[Jean-Francois Ducarroz]

reassign to rhp.

Comment 4

[rhp (gone)]

Hi Naoki,
Is ISO-8859-1 and us-ascii the same? 

Also, I'm not sure this is a problem. If you want to preserve the original 
source of the message, you should use "Forward as Attachment", not reply. When 
you reply, you are quoting into a new editor and all bets are off on the 
actually contents. As long as the display is correct, that is all we can 
guarantee with a reply operation.

Naoki, if you agree, please mark invalid.

- rhp

Comment 5

[nhottanscp]

us-ascii covers 0-127, ISO-8859-1 is 0-255. So us-ascii is a subset of 
ISO-8859-1.

For reply, we are getting MIME header charset based on mail body's charset. 
In many cases, mails are labeled as us-ascii even characters in ISO-8859-1 
are used because we use entities in case of text/html. Also, body may contain 
only us-ascii range of characters (e.g., emoticons, etc..) or even empty as 
reported in this bug. Our sending default charset is ISO-8859-1 and we use 
us-ascii depends on the characters used in the body (so we have no charset item 
us-ascii in the charset menu). Actually, if the charset is explicitly set by the 
menu before sending the reply, ISO-8859-1 is used in MIME header.

So, I need to change MIME encoder. At the point of the encoding code, it is 
known that we need encoding the string which means it is known it is not 
us-ascii. I can change it to the default mail send charset.
This resolve most of the case. Not if the user has mails in multiple charset but 
those case it can be changed by the menu. Also, we are going to have a charset 
feedback somewhere in UI (20249).

Comment 6

[nhottanscp]

This is fixed in M16.
I didn't change to use the default charset but other change to fix a forward 
quoting problem fixed this. So it works for Latin1 only but I think that's the 
majority of the cases (like latin1 html mail with named entities labeled as 
us-ascii).

Comment 7

[nhottanscp]

mark as fixed

Comment 8

[Katsuhiko Momoi]

** Checked with 6/5/2000 Win32 build **

This has been fixed for some time even before 4/20. 
I believe the fix is to assume "ISO-8859-1" when we get
to the point of encoding the headers when the body charset says 
US-ASCII. If it contains no 8-bit, then nothing occurs but
if the header contains 8-bit, then we get to have ISO-8859-1
Q=encoded string generated.
For other Latin encodings, mailers tend to use QP rather than
entities and so the body charset will normally not be US-ASCII
but rather something like ISO-8859-2. And Mozilla will simply
inherit this in replies. 
All in all, I think this solution works pretty well.

Marking it verified as fixed.