Closed Bug 494912 Opened 11 years ago Closed 10 years ago

Wrong Character Set used when forwarding

Categories

(MailNews Core :: MIME, defect, major)

1.9.1 Branch
x86
Windows XP
defect
Not set
major

Tracking

(thunderbird3.1 beta2-fixed)

RESOLVED FIXED
Tracking Status
thunderbird3.1 --- beta2-fixed

People

(Reporter: kristo, Assigned: bugzilla)

Details

Attachments

(2 files)

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1
Build Identifier: Until 26.Mai 2009

when receiving mails with the following encoding
Content-Type: text/plain; charset=utf-8
... with german umlauts....
and forwarding then the wring charset is being used and all special characters
are displayed the wrong way. (äüö...)
When replying everything's ok.


Reproducible: Always

Steps to Reproduce:
1. Receive a UTF-8 encoded mail with german umlauts
2. press forward and you'll see the errors
3. press reply and everything's fine


Expected Results:  
Correct codeset when forwarding....
Flags: blocking-thunderbird3?
(In reply to comment #0)
> User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.1)
> Gecko/2008070208 Firefox/3.0.1
> Build Identifier: Until 26.Mai 2009

Kristo, what is the correct build identifier (obtained from help -> about)?

Is this a regression from 2.x functionality?
Keywords: qawanted
The problem remains for nightly builds up to june 3rd 2009.
TB3 of course.
So this WFM , with Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1pre) Gecko/20090603 Shredder/3.0b3pre. I've received an forwared text in UTF8 containing ü.

A slight difference is that :

Content-Type: text/plain; charset=UTF-8; format=flowed

Which is I think the default as I don't remember changing any of these settings.

Kristo how come your headers don't contain format=flowed ?
No idea. 

But then, the problem doesn't seem to be in the header!

Because if I press reply then everything's fine.
Only pressing Forward creates the corrupt characters.

Here's the source of one of those mails before forwarding:
---------------------------------------------------------
From - Mon Jun 01 11:43:51 2009
X-Account-Key: account3
X-UIDL: 1243621456.H811217P7432.satyr.ispgateway.de,S=567726
X-Mozilla-Status: 1003
X-Mozilla-Status2: 10000000
X-Mozilla-Keys:                                                                                 
Received: from [80.67.18.6] (helo=mx06.ispgateway.de)
	by satyr.ispgateway.de with esmtp (Exim 4.68)
	(envelope-from <#############################>)
	id 1MA6ka-0001vq-Lo; Fri, 29 May 2009 20:24:16 +0200
Return-path: <###############################>
X-Envelope-To: ##########################
Received: from [212.227.17.10] (helo=moutng.kundenserver.de)
	by mx06.ispgateway.de with esmtp (Exim 4.68)
	(envelope-from <###########################>)
	id 1MA6ka-0006R0-EG
	for #########################; Fri, 29 May 2009 20:24:16 +0200
Received: from mailnix.buxnet.allg (p4FCF6450.dip.t-dialin.net [79.207.100.80])
	by mrelayeu.kundenserver.de (node=mrbap0) with ESMTP (Nemesis)
	id 0MKsym-1MA6jq18jy-000dSy; Fri, 29 May 2009 20:24:16 +0200
Received: from [192.168.100.123] (golem.buxnet.allg [192.168.100.123])
	by mailnix.buxnet.allg (8.14.3/8.14.3/Debian-5) with ESMTP id n4TINSY3020780
	for <######################>; Fri, 29 May 2009 20:23:28 +0200
Subject: Frage
From: ######################################
To: ####################################
Content-Type: multipart/mixed; boundary="=-yiyXQj116VM5W1FNfI1K"
Date: Fri, 29 May 2009 20:21:06 +0200
Message-Id: <1243621266.3665.7.camel@golem.buxnet.allg>
Mime-Version: 1.0
X-Mailer: Evolution 2.22.3.1 
X-Provags-ID: V01U2FsdGVkX18WISFwJpmUbbEp10Au4rogtQ4f38Eg8FwnY8T
 Ut678rUHr9HnvjFR3mUyXBxO4EJ1GYOh9zzTNmnt7mdf4Px/iU
 nWMTlE4XDnW/EOfBsY9lg==
X-EsetId: 5A40E72B2E7C39694546E77C2C277D


--=-yiyXQj116VM5W1FNfI1K
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hi Kristo

Das mit dem Importieren sollte jetzt gehen. Das Homeverzeichnis des
FTP-Users war noch auf dem alten Pfad, das hatte ich übersehen...

Ich habe das neue Bild und das PDF hier nochmal an die Mail drangehängt, schau dir das nochmal an - vielleicht spinne ich ja auch einfach nur...

Grüße
Component: Message Compose Window → MIME
Product: Thunderbird → MailNews Core
QA Contact: message-compose → mime
Version: unspecified → 1.9.1 Branch
andrew anything phishy here ?
Denying blocking until better confirmation.
Flags: blocking-thunderbird3? → blocking-thunderbird3-
Kristo: can you attach a sample? Save as .eml
The bug is due because when a new window is open for forward, reply, etc to an email, the flag charsetoverride is not set. Then, the conversion is not build.
Comment on attachment 438064 [details] [diff] [review]
A patch to correct this bug.

Philip thank you very much for the patch. If you want your patch to ever be commited you need to ask for review and Super review as documented at https://developer.mozilla.org/en/comm-central#Requirements.

We do now require unit tests for patches could you have a look at https://developer.mozilla.org/en/Thunderbird/Thunderbird_Automated_Testing and see if you can come up with a patch, we can provide help if you need for the test writing.

Dmose, David, this is a one liner.
Attachment #438064 - Flags: superreview?(dmose)
Attachment #438064 - Flags: review?(bienvenu)
Assignee: nobody → bugzilla
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Keywords: qawanted
Who is the reviewer and superreview ?
I don't understand what i need to do now.
Attachment #438477 - Attachment mime type: application/x-mimearchive → text/plain
I'm the reviewer and dmose is the sr. I'll try to look at this today.
I will too, this evening, though this patch looks like a very good candidate for r+sr, if bienvenu is so inclined.
Comment on attachment 438064 [details] [diff] [review]
A patch to correct this bug.

thx, this fixes it. We'd like a mozmill (or xpcshell) test for this, but I'd also like the fix for 3.1 so I will land this fix.
Attachment #438064 - Flags: superreview?(dmose)
Attachment #438064 - Flags: superreview+
Attachment #438064 - Flags: review?(bienvenu)
Attachment #438064 - Flags: review+
fixed for b2. Thx, Philippe.
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Flags: in-testsuite?
Resolution: --- → FIXED
(In reply to comment #15)
> (From update of attachment 438064 [details] [diff] [review])
> thx, this fixes it. We'd like a mozmill (or xpcshell) test for this, but I'd
> also like the fix for 3.1 so I will land this fix.

Philippe would you be interested in writing the tests ?
The change landed for this bug
https://hg.mozilla.org/comm-central/rev/60e7a12dcc96

was backed out in bug 1239658:
https://hg.mozilla.org/comm-central/rev/8de961aed2b4

because it (frankly) was completely wrong. It caused a very bad regression documented in bug 597369 (and its five duplicates!) and bug 1239658.

Basically the override should only be set when the user manually overrides the text encoding. Setting the override programmatically has the fatal consequence that a charset used for a previously viewed message is - under certain circumstances - carried over to another message which is used for forwarding/"edit as now"/"edit draft" later.

Is is correct that the message in attachment 438477 [details] doesn't not get displayed correctly when forwarding it. This has to do with the fact that the message is "multipart" and has an incorrect structure.

Instead of (correct) (added indentation to show the structure) ...
Content-Type: multipart/alternative; boundary="ALT"
ALT
 Text part
ALT
  Content-Type: multipart/related; boundary="REL"
  REL
    HTML part
  REL
    image
  REL
ALT

... it has the structure the other way around:
Content-type: multipart/related; Boundary="REL"
REL
  Content-type: multipart/alternative; Boundary="ALT"
  ALT
    text part
  ALT
    HTML part
  ALT
REL
  image
REL
  image
REL

So basically this special message falls victim of a "multipart" bug of which there are many (see meta-bug 505172). The patch proposed here to enforce the UTF-8 display of this message was unacceptable.
Also see bug 1251783.
You need to log in before you can comment on or make changes to this bug.