Closed Bug 3979 Opened 25 years ago Closed 25 years ago

[BETA] Mail- Content-Type: charset of reply and forward is always us-ascii

Categories

(MailNews Core :: Internationalization, defect, P3)

x86
Windows NT
defect

Tracking

(Not tracked)

VERIFIED FIXED

People

(Reporter: nhottanscp, Assigned: rhp)

References

Details

Content-Type: charset of the original message should be used.
Assign to me (for now).
Status: NEW → ASSIGNED
Target Milestone: M5
Summary: Content-Type: charset of reply and forward is always us-ascii → Mail- Content-Type: charset of reply and forward is always us-ascii
Requested bienvenue@netscape.com to add charset name support to nsIMessage.
Requested bienvenu@netscape.com to add charset name support to nsIMessage.
I've added it, but there's no code that sets it to anything other than the empty
string. I'm drowning in IMAP and will be until after M6.
Changed to use the new code. Use the value when the string is not empty.
Target Milestone: M5 → M6
Moving to M6, this will not work until nsIMessage change.
Assignee: nhotta → bienvenu
Status: ASSIGNED → NEW
Reassinging to bienvenu@netscape.com.
Target Milestone: M6 → M7
What more do you want me to do? I've added the methods to nsIMessage. I'm
guessing you want me to parse every message for its content type and set the
field appropriately. This is of course impossible for IMAP and News...didn't we
used to rely on libmime having parsed the message in the past?
Assignee: bienvenu → phil
Reassinging to phil@netscape.com.
I don't know who is responsible for reply implementation.
Assignee: phil → rhp
Reassigning to rhp
Status: NEW → ASSIGNED
Whiteboard: DEPEND - Intl
*** Bug 6995 has been marked as a duplicate of this bug. ***
Blocks: 7228
Target Milestone: M7 → M8
Will look at for M8.
Target Milestone: M8 → M9
Naoki,
How should this work? Should I just set the charset of the reply to the same
charset of the replied to message? This seems like we could have problems doing
that, but just guessing here.

- rhp
For reply and forward, the charset of the original message main body should be
used.
Target Milestone: M9 → M10
Won't get to this before tonight...kind of a tough problem.

- rhp
Naoki,  As a temporary improvement, can we just omit the charset parameter
in the headers for replies and forwards until we get this fixed?
It is not usual to send messages without Content-type charset.
I don't think we get workaround by sending without charset, do we?
Rich, are you proposing to omit the charset parameter from all parts including the 1st of the multi-part bodies?
If so, that wouldn't be a good idea,
I'm not suggesting we change anything...I just latered the bug because this is
not a trivial fix and then bobj and naoki had the other comments.

- rhp
Sorry, Rich. My question should have been aimed at bobj.
I talked to bobj and momoi, bob's suggestion depended the charset override
feature which is not working yet (5938). So we should not omit a charset label
from reply.
Does our 4.x product do what's being suggested here; i.e. use the charset from
the message being replied to, rather than the charset of the author of the
reply.

Effectively, I'm asking whether this is an RFE or a regression.
Whiteboard: DEPEND - Intl
I don't know about 'RFE' but this is not a regression of 4.x.
I think 4.x depended on the charset menu selection to decide the charset of
sending out messages.
But this is needed for 5.0.
In 4.x, reply/forward quoted message cound not be displayed unless the charset
menu is set properly.
5.0 uses Content-type charset and usually the user doesn't have to set charset
menu for viewing. Reply/forward also does not require to set charset menu for
viewing/editing but sending out message needs to specify a charset. For initial
charset for reply/forward, it is appropriate to use the charset of the original
message than the current charset menu selection.
I assume confusions if we rely on the menu selection for reply/forward charset
(e.g. sending out with a wrong charset by accident and the receiver cannot see
the message at all).
BTW, there is also a bug of no menu check mark.
Target Milestone: M10 → M13
Well, two thoughts:

> For initial charset for reply/forward, it is appropriate to use the charset of
> the original message than the current charset menu selection.

I don't understand this. If the reply isn't in the same character set as the
original message, it seems like the charset on the reply is wrong -- i.e.
doesn't reflect the data which is actually in the message.

Second, Rich tells me that changing the MIME parser so that we have the original
character set hanging around in memory is a major change. It's not likely we'll
attempt that before he returns, if at all. TFV -> M13.
>I don't understand this. If the reply isn't in the same character set as the
>original message, it seems like the charset on the reply is wrong -- i.e.
>doesn't reflect the data which is actually in the message.
That case (e.g. replying to French message by Japanese), it is users
responsibility to reset the charset. There is also a risk that the original
message may not be mapped to the sending charset if sending out as different
charset than original unless send as unicode (e.g. UTF-8). For that, we're
planning for replacing those unmapped characters as entities (but that's for
HTML only).
I'll investigate further when time allows.

- rhp
Target Milestone: M13 → M14
Summary: Mail- Content-Type: charset of reply and forward is always us-ascii → [BETA] Mail- Content-Type: charset of reply and forward is always us-ascii
If there's an i18n interop issue here, we should look at it for B1.
For reply we do, but not for forward.

- rhp
So, how much extra work needed or performance increase for getting Content-Type
of the main body in case of we're not quoting?
this is a test
Hello,
I have an idea how to fix this problem, but I need to understand one thing. Do 
we need to use the charset of the mail message, or the part that is displayed 
as the primary body part. Please tell me the first one :-) Also, I could use a 
message that has a non us-ascii charset for the top level mail message itself. 
Can you post an example I can work from.

Thanks!

- rhp
Finally, after putting my brain into this one, I think I have it fixed. Now, 
when you reply to a non US-ASCII email, you will reply in that charset. 
Forwarding INLINE also has this same behavior, but if you forward as an 
attachment, I do not do this. The reason is that the attachment is an 
encapsulated message/rfc822 message that should render properly since it has 
all of the appropriate charset information inside of the message structure.

I will check this in when the tree opens for M14.

- rhp
Summary: [BETA] Mail- Content-Type: charset of reply and forward is always us-ascii → [FIXED] [BETA] Mail- Content-Type: charset of reply and forward is always us-ascii
Does this do something like:

1. For single part msg --> relfect the content-type charset info.
2. For multi-part msg --> reflect first body par charset

for reply/forward inline?
Not currently. This increases the complexity to have to determine the type of 
message and decend to the main body part to pull out the charset. Is is this 
the right thing to do???

- rhp
For multi-part, if you are not descending into the body to get the reply/forward-inline charset,
where do you get the charset info? 
I get it from the RFC822 message header. If one isn't specified it will default 
to the users default settings.

- rhp
Rich, I remember discussing some of this with nhotta. The spec has been 
published to this page:

http://www.mozilla.org/projects/intl/mail-news-i18n-spec.html

"Charset of the original message (main body) is used." for charset for reply/forward mail.

Naoki argued above also that we should not use the default charset set by the user if
there is a charset info already in the mail. We would not go crazy like descending into 2nd, 3rd parts
for multi-part msgs. The 1st (main) part should be sufficient. 
Thus, falling back to user's default fro multi-art is not what the spec would call for.

Could we wait until Monday when Naoki is back to settle this? I'm working on Messenger
charset menu related specs and some of them may have bearing on this issue.
This makes sense. I will see if I can handle this situation...I have some ideas 
how to make this happen.

- rhp
Good news. I think after some more hacking that I am able to decend into the 
body part of a multipart/ message and get the charset to be used for replies 
and forwards.

We'll need more testing, but so far, I think this is working.

- rhp
Just checked in the fix for this problem.

- rhp
Summary: [FIXED] [BETA] Mail- Content-Type: charset of reply and forward is always us-ascii → [BETA] Mail- Content-Type: charset of reply and forward is always us-ascii
Marking fixed.
Status: ASSIGNED → RESOLVED
Closed: 25 years ago
Resolution: --- → FIXED
I'm in the process of looking at this fix. It's going to take a bit of time for
we need to cover a variety of cases.
There seems to be one problem now. On our smoketest mailbox, the 1st msg is 
Latin 1 but using HTML entities and so marked as US-ASCII. The last msg is in UTF-8
and marked as so. These two will not go out reflecting the original charset label
if you have the following setting in prefs.js.

user_pref("intl.character_set_name", "iso-2022-jp");

I tried, Trad. Chinese (Big5), Simplified Chinese (gb2312), Korean (EUC-KR),
Czech (ISO-8859-2), Russian (KOI8-R), and they are OK even with the above setting.

Naoki, the above is not the mail default setting, is it? Could you remind me what the mail default
setting formual is? I'll also try that later.
Momoi san, your setting for the mail send default charset is correct.
And that setting should not affect the behavior of reply/forward charset 
setting.
I should add that when you remove: user_pref("intl.character_set_name", "iso-2022-jp");
reply msgs work as intended by this fix. 
By the way, forward/inline seems to have this problem with the above special setting.
Correction: "By the way, forward/inline seems to have this problem with the above special setting."

to: By the way, forward/inline DOES NOT seem to have this problem with the above special setting.
Re-opening it since there is a limited but nonetheless real problem with some
msg charset headers when the mail default charset setting exists in prefs.js.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Ok, here is the problem. We make a call to nsMsgI18NGetDefaultMailCharset() in 
the compose back end when we create a message. This is the charset we will use 
unless there is a non us-ascii specified charset on the mail message. If we 
have a non-usascii, it shows up on the content type of the quoting operation, 
but other than that, we don't see any and we don't change it.

I'm looking to see where exactly that is stripped off the line.

- rhp

Status: REOPENED → ASSIGNED
I have a fix that Naoki is reviewing.

- rhp
Ok, I think I have a fix for this one now. Will try to get checked in tonight.

- rhp
Due to Beta indication in Summary, putting beta1 into keyword field.
Keywords: beta1
Checked in this fix today.

- rhp
Status: ASSIGNED → RESOLVED
Closed: 25 years ago25 years ago
Resolution: --- → FIXED
Half-way through. So far tried replying to some 16 different
language msgs with some charset buried in the main body part.
Also tried the problem ones with default charset set to ISO-2022-JP.
So far none failed. I will move on to forard/inline to finish
the verification.
Depends on: 28055
At this point, I cannot verifyu this until Bug 28055 is fixed.
** Checked with 2/21/2000 Win32 build **

This feature is working very well with the above build.
I checked with some 17 different charsets -- some in
single part msgs and others in the first part of
multipart msgs. We are now able to reflect the original
charset regardless of the mail-send default charset setting
with all these test msgs.

This is in accord with our current spec.

Marking it verified/fixed.
Status: RESOLVED → VERIFIED
Product: MailNews → Core
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.