Inconsistent charset when copying headers in Reply/Reply-all

NEW
Unassigned

Status

Thunderbird
Message Compose Window
13 years ago
7 years ago

People

(Reporter: Pierre Sarrazin, Unassigned)

Tracking

Trunk
x86
Windows XP

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(3 attachments)

(Reporter)

Description

13 years ago
User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; fr-FR; rv:1.7.12) Gecko/20050919 Firefox/1.0.7
Build Identifier: Thunderbird 1.6a1 (Windows/20051201)

Bug appears when the "é" character of a mail contact name is coded by "?" in the source of the message. This is the case when there is no specific iso form. For instance, "d&#65533;emo test" <demotest@test.fr>). 

Then:
1)if ?'s contact is the sender.
reply operation provide:
"d&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;" wrote: at the beginning of the message.

2) if ?'s contact is included in Bcc or Cc..
Reply-to-all loose all the contacts which follows the ?'s contact in the cc: field of the answer!

Reproducible: Always

Steps to Reproduce:
Case 2: Receive a mail with a ?'s contact in cc
I have replaced the real email adress by xxx@yyy.fr

...
Message-ID: <8422910.1132070173817.JavaMail.www@wwinf1536>
From: "ln.gottlob" <xxx@wanadoo.fr>
Reply-To: xxx@wanadoo.fr
To: sophie hallauer <xxx@yahoo.fr>,
	Pierre-Joseph Bascourt <xxx@gmail.com>,
	CZERWONKA carine <xxx@agf.fr>
Subject: Re: hello tt le monde!
Cc: "b&#65533;atrice GIBAULT" <xxx@yahoo.fr>,
	HENNECHART B DsicRis <xxx@socgen.com>,
	sophie hallauer <xxx@yahoo.fr>,
	Corinne Minc <xxx@yahoo.fr>, xxx@free.fr
Mime-Version: 1.0

Then Reply to all....
Actual Results:  
case 2:

...
From: Pierre Sarrazin <xxx@free.fr>
X-Mozilla-Draft-Info: internal/draft; vcard=0; receipt=0; uuencode=0
User-Agent: Thunderbird 1.6a1 (Windows/20051201)
MIME-Version: 1.0
To:  xxx@wanadoo.fr
CC: sophie hallauer <xxx@yahoo.fr>, 
 Pierre-Joseph Bascourt <xxx@gmail.com>,
 CZERWONKA carine <xxx@agf.fr>, 
 b&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;
Subject: Re: hello tt le monde!
References: <8422910.1132070173817.JavaMail.www@wwinf1536>
In-Reply-To: <8422910.1132070173817.JavaMail.www@wwinf1536>
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit

you see?


I can send to you the messages which cause the errors (case 1 and 2)
Send me a mail to pierre.sarrazin@mail.com 

Problem of reply-to-all (case 2) can be avoid by setting "use the default character encoding in reply" in tools/options/Display menu.

Regards, Pierre

Comment 1

13 years ago
Please save the problem message as a .EML file, and attach the file to this bug using the Create New Attachment link above.  As you can see by looking at the bug, pasting in non-ASCII data doesn't always work as expected.
Summary: 1) Bad diplay of sender name when reply 2)Loose of "Cc: contacts" when reply-all → 1) Bad display of sender name when reply 2)Loss of "Cc: contacts" when reply-all
Version: unspecified → Trunk
(Reporter)

Comment 2

13 years ago
Created attachment 204789 [details]
Case 2: Loss of "Cc: contacts" when reply-all
(Reporter)

Comment 3

13 years ago
Created attachment 204790 [details]
Case 1: Bad display of sender when reply

Can not exactly reproduce the case 1 with the eml file.
You will meet another problem. (perhaps endly the same)

1) Open the eml file in text editor
The sender name is "Stéphane Rivat"
2)Open the eml file in thunderbird
The sender name becomes "st"
(Reporter)

Comment 4

13 years ago
Created attachment 204792 [details]
Inital problem description

To avoid the pasting in non-ASCII data and improve the comprehension !

Comment 5

13 years ago
The basic problem is, whatever email program is being used to generate those messages is breaking the rules: it's not legal to put non-ASCII characters 
(such as "é") in a header without MIME-encoding it.  

(Note that your "case 1" message is a bad example because the From: address contains a problematic name but the message has a Reply-To: header with only 
his email address -- so when replied to, only his email address is copied to 
the To: header of the new message.  This sidestpes part of the problem you're reporting.  In my testing, I removed the Reply-To header.)

As you may have noticed, there can be problems with the display of such messages; the charset used to display an unencoded header is taken from the folder's default.  If you're not seeing a problem here, then you have 
ISO-8859-1 (or a related charset) as the default for the folder.

Testing this, I observe the following:

When you Reply to a message, the original From: is transcribed to the new To: header assuming the character set *of the original message body* .  (Even if 
you change the encoding used to view the message, via the View|Encoding menu item, the header is transcribed according the charset in the body's 
Content-Type header.)

When you Reply All to such a message, an address from the To: or CC: headers will be transcribed to the new CC: header assuming the character set 
*of the new message* 

These headers are ISO-8859-1, but the messages themselves are UTF-8.
A reply to Stéphane therefore always ends up with a decoding error in the new To: field, even if it was displayed correctly.

A reply-all with Béatrice in the CC: list will mis-decode her name (and all subsequent names in the CC: list, which is handled as a single string) if you have specifed ISO-8859-1 as the preferred outgoing charset *and* have checked the option to use the default for replies (if that option is unchecked, the reply will use the original message's UTF-8 charset).

I think this is inconsistent.  It seems more sensible to me that the headers should be transcribed using the charset with which they're displayed in the Envelope panel, regardless of how that charset was selected.  Jungshik, what 
do you think?
Status: UNCONFIRMED → NEW
Ever confirmed: true
Summary: 1) Bad display of sender name when reply 2)Loss of "Cc: contacts" when reply-all → Inconsistent charset when copying headers in Reply/Reply-all
QA Contact: message-compose

Updated

10 years ago
Assignee: mscott → nobody
Attachment #204789 - Attachment mime type: message/rfc822 → text/plain; charset="windows-1252"
Attachment #204790 - Attachment mime type: message/rfc822 → text/plain; charset="windows-1252"
Attachment #204789 - Attachment mime type: text/plain; charset="windows-1252" → text/plain; charset="iso-8859-1"
Attachment #204790 - Attachment mime type: text/plain; charset="windows-1252" → text/plain; charset="iso-8859-1"

Comment 6

7 years ago
I also experience such issues and I found out that if a message arrives that has as a first CC-Entry a recipient with special character(s), then everything works fine. If the second recipient has special character(s) in it, then cc entries are lost.

So this works (excerpt from header):
To: Wildam Martin <martin.wildam@something.at>, "mwildam@something.com"
	<mwildam@something.com>
CC: =?iso-8859-1?Q?Wei=DFthanner_Alois?= <Alois.Weissthanner@something.de>,
	"mwildam@something.at" <mwildam@something.at>

But this does not:
To: Wildam Martin <martin.wildam@something.at>
CC: <mwildam@something.at>, =?UTF-8?Q?Wei=C3=9Fthanner_Alois?=
	<Alois.Weissthanner@something.de>

In my opinion it does not have anything to do with the encoding itself, but with the position. I am pretty sure, that in CC field only start of the line is checked for encoding prefix ("=?...?Q?) and not each recipient string start.

It tested sending from MS Outlook as well as from GMail or Exchange over Thunderbird - behaviour is the same for all cases: Encoding prefix is put for each recipient on its own and not once at the beginning of CC-line.

Could this be the real problem, that encoding is not checked for each recipient seperately?

Comment 7

7 years ago
I do consider this as a critical bug because I am pretty sure that - not checking - I already missed to put some important persons on CC and hence they were not informed although they should! Putting a longer list of CC back manually is a work that costs several minutes. I definitely can't push a company to use Thunderbird instead of Outlook if something like reply to all does not work reliable.

Thunderbird 6.0.2 on Ubuntu 10.04 - same issue, so this problem is _not_ platform specific.
You need to log in before you can comment on or make changes to this bug.