User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:188.8.131.52) Gecko/20060421 Firefox/184.108.40.206 Build Identifier: Thunderbird 220.127.116.11 FR downloaded from http://www.mozilla.com/products/download.html?product=thunderbird-18.104.22.168&os=win&lang=fr When TB sends a return receipt in French, it sends non-ASCII characters in the subject header. When the recipient of the return receipt has configured amavisd-new to reject bad headers, I get error messages such as : INVALID HEADER (INVALID CHARACTERS OR SPACE GAP) Non-encoded 8-bit data (char E9 hex) in message header 'Subject': Subject: Accus\351 de r\351ception (... and the recipient doesn't get the return receipt. The problem comes from some localized messages are using non-ASCII characters . For example : MdnDisplayedReceipt=Accus\u00e9 de r\u00e9ception (affich\00e0) These messages should be encoded by TB (RFC 2047). Same problem for the following messages : MdnDispatchedReceipt,MdnProcessedReceipt,MdnDeletedReceipt,MdnDeniedReceipt,MdnFailedReceipt. This problem certainly occurs for several other languages. Reproducible: Always Steps to Reproduce: 1. Install a french version of Thunderbird 22.214.171.124 2. Send a message to Thunderbird from another place, asking for a return receipt 3. When the message arrives, accept to send the return receipt 4. At the other place, look at the headers of the return receipt message sent by Thunderbird Actual Results: The header of the return receipt contains : Subject: Accusé de réception (affiché) - Essai AR jcr Expected Results: The header od the return should contain someting like : Subject: =?iso8859-1?Q?Accus=E9_de_r=E9ception_(affich=E9)?= - Essai AR jcr
Same problem in version 126.96.36.199
Yes, this is a Thunderbird bug for the return receipt function. The bug is reported for the French version of TBird but I suspect it will be a problem for all languages that use 8bit characters. For example, 1. If someone sends a message header: le ministère des Finances TBird/French version returns the following subject header in French. Subject: Accusé de réception (affiché) - le =?ISO-8859-1?Q?minist=E8re_des_Finances?= The canned header part "Accusé de réception (affiché)" does not go through MIME encoding. 2. The body part that acknowledges the receipt a message always looks like this: --------------mdn080702080101020103080400 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Ceci est un accusé de réception pour le courrier éléctronique envoyé à email@example.com. Note : Cet accusé de réception indique seulement que le message a été affiché sur l'ordinateur du destinataire. Il n'y a aucune garantie que le destinataire ait lu ou appréhendé le contenu du message. ====================== What's wrong is that if the MIME encoding is OFF in the preference, Tbird always adds: Content-Transfer-Encoding: 7bit even if the body contains 8bit data. The return receipt part quotes the original headers -- which is always supposed to be in 7bit (ASCII) but what TBird is adding are 8bit character data. Is there a way make the return receipt go though the normal message formatting process so that the MIME requirements are met properly?
Status: UNCONFIRMED → NEW
Ever confirmed: true
(In reply to comment #2) > > 1. If someone sends a message header: le ministère des Finances > > TBird/French version returns the following subject header in French. > > Subject: Accusé de réception (affiché) - le > =?ISO-8859-1?Q?minist=E8re_des_Finances?= > > The canned header part "Accusé de réception (affiché)" does not go through > MIME encoding. Momoi-san, I'll ask an ignorant question. What if the subject is in a different charset encoding than the canned text? i.e. you are using a french build (Accusé de réception (affiché)) but the subject in the message we are generating a return receipt for us is encoded in say a japanese character set? Should we be encoding each of these separately in the subject? Or should the entire subject be encoded once? FYI, this should be trivial to fix, the code is right here: http://lxr.mozilla.org/mozilla/source/mailnews/extensions/mdn/src/nsMsgMdnGenerator.cpp#574 We just need to figure out if both strings should be encoded together for the subject or if the canned text should be encoded separately.
it seems prett obvious that my question was ignorant :). We'll want to encode each part of the subject separately, RFC 2047 talks about that. So this should be easy if we can get a string that maps to the charset of the current locale.
see also bug 139615 I'll attach a patch in a minute that addresses these issues, by using utf-8
Created attachment 257405 [details] [diff] [review] proposed fix for subject and body encoding This builds on Scott's initial patch to encode the return receipt part of the subject as utf8, and also generate 8 bit body, encoded as utf8. I changed the "structured" arg to nsMsgI18NEncodeMimePartIIStr to FALSE. I believe "structured" is used to indicate that it's an address list, and passing in TRUE was causing things before '(' to get trimmed off. I removed the localization comment about not translating the mdn body strings, since I think they can now be localized. I don't have to check that into 2.0 if you think that's a bad idea, but it's not going to break anything, most likely :-)
Assignee: mscott → bienvenu
Status: NEW → ASSIGNED
Attachment #257405 - Flags: superreview?(mscott)
A question & comment: I thought the spec was to return a localized return receipt only if the original sender's first accept language choice matched that of the user's localized UI language. That is, the French Thunderbird UI language user will send out the French version of the return receipt only if the message's X-Accept language header lists "fr" at the top. Otherwise, it falls back to the ASCII version of the return msg. Is this not true? If the above assumption is right, I don't understand the need to use UTF-8 in encoding body or the headers. In cases where the original contains non-Latin 1 headers such as Chinese characters, you can re-use the original MIME encoded words. Otherwise, simply use the encoding that matches the UI language. It is still true that in some parts of the world, e.g East Asia, UTF-8 causes garbling problem. If we use UTF-8, we could end up inconveniencing the requester of the receipt.
Kat, I think currently we're always returning the localized return receipt, except we tell the localizers not to localize the strings. How do we find out the current localization charset?
Comment on attachment 257405 [details] [diff] [review] proposed fix for subject and body encoding thanks David for doing this. I think it's fine to remove the comment on the branch.
Attachment #257405 - Flags: superreview?(mscott) → superreview+
fixed on trunk and branch We should spin Kat's issue into a new bug...
Status: ASSIGNED → RESOLVED
Last Resolved: 12 years ago
Resolution: --- → FIXED
> Kat, I think currently we're always returning the localized return receipt, > except we tell the localizers not to localize the strings. Thanks for the update. > How do we find out the current localization charset? The language of the TB build is in the X-user agent string. And there should be a file that maps from the language to default mail_charset. > We should spin Kat's issue into a new bug... I think this bug will be inevitable given the acceptance of UTF-8 in some locales.
Hah, I think this is one of the rare cases where a TB patch "fixed" a SeaMonkey file but not the TB one - even if it's just a L10n note ;-) You should probably remove the localization note from http://lxr.mozilla.org/mozilla/source/mail/locales/en-US/chrome/messenger/msgmdn.properties as well :)
Jean-Claude, can you help verify this bug with a branch nightly: http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/latest-mozilla1.8-l10n/ Thanks!
I have tested the windows version in french (thunderbird-2.0pre.fr.win32.installer.exe). I have sent myself a message with the following subject : Subject: Essai =?ISO-8859-1?Q?accus=E9_de_r=E9ception?= And I have received a mdn with that subject: Subject: =?UTF-8?B?QWNjdXPDqSBkZSByw6ljZXB0aW9uIChhZmZpY2jDqSkgLSA=?=Essai =?ISO-8859-1?Q?accus=E9_de_r=E9ception?= The subject is correctly displayed in Thunderbird : Accusé de réception (affiché) - Essai accusé de réception So, I think it is correct. Thanks to all the people that worked to fix this bug ! Jean-Claude
Dear all, I would like to bring to your attention the following bug: https://bugzilla.mozilla.org/show_bug.cgi?id=254149 I think it's related to this one. In this case, amavis complains that the subject header is not correctly encoded. My test bench are several mail servers (debian etch and sarge) with amavis as content filter, while on the client side I've got TB up to version 2.0pre (20070319) (french).
In Seamonkey/1.1.1 there is still a bug. The Subject is coded as follows (German locale): Subject: ?ISO-8859-15?Q?Empfangsbest=E4tigung_=28angezeigt=29?= But it should be Subject: =?ISO-8859-15?Q?Empfangsbest=E4tigung_=28angezeigt=29?= The fix for Thunderbird is probably not ported to Seamonkey. The file mozilla/mailnews/extensions/mdn/src/nsMsgMdnGenerator.cpp differs from Seamonkey to Thunderbird (see attachment). Do we need a separate bug report for Seamonkey?
Bjoern, I don't understand your diff. There is only one nsMsgMdnGenerator file in the tree. It's shared by seamonkey and thunderbird, as are most of the files in mailnews.
Created attachment 260858 [details] [diff] [review] Diff between Seamonkey and Thunderbird
Attachment #260854 - Attachment is obsolete: true
Yes, normally the file mozilla/mailnews/extensions/mdn/src/nsMsgMdnGenerator.cpp should be shared between Seamonkey and Thunderbird. But for the current versions (Seamonkey 1.1.1 and Thunderbird 2.0.0) this is not the case and therefore this bug is still present in Seamonkey. I wanted to show this with the attached patch. Currently I'm testing, if Seamonkey 1.1.1 works right with nsMsgMdnGenerator.cpp from Thunderbird 2.0.0.
Bjoern: I believe the bug you are seeing is actually in the language pack - and additionally, SeaMonkey 1.1.1 is much older source than Thunderbird 2.0, this will be fixed in SeaMonkey 1.1.2 which should get released about the same time as Thunderbird 2.0 final.
My tests with nsMsgMdnGenerator.cpp from Thunderbird 2.0.0 copied to Seamonkey 1.1.1 were successful: - the German language pack seems to be ok - Subject is correctly encoded Is it possible to verify if Seamonkey 1.1.2 will have the corrected version of nsMsgMdnGenerator.cpp? What is the correct CVS branch for Seamonkey 1.1.2 development?
The CVS branch for SeaMonkey 1.1.2 is the same as for Thunderbird 2.0, i.e. MOZILLA_1_8_BRANCH
Return receipts look right when sent from 188.8.131.52-pre (lt langpack).
Status: RESOLVED → VERIFIED
Oops, I guess I was too early. Here's what a "Subject" header of the Return Receipt looks like: Subject: Gavimo pažyma (laiškas atvertas) - =?windows-1257?Q?=FEi=FBrim_=FEi=FBrim=2E=2E=2E?= That is clearly RFC-incompliant. One question: am I supposed to put a QP-string in my translation for this case, or should I leave this responsibility to the application (Thunbderbird)?
Status: VERIFIED → REOPENED
Resolution: FIXED → ---
Could anyone please take a look at comment #27 ?
OS: Windows 2000 → All
Hardware: PC → All
(In reply to comment #12) > You should probably remove the localization note from > http://lxr.mozilla.org/mozilla/source/mail/locales/en-US/chrome/messenger/msgmdn.properties > as well :) This bug is pretty much the poster child for "don't comment on closed bugs unless you want to be ignored," isn't it? Pushed in http://hg.mozilla.org/comm-central/rev/442f53864c91 and thanks for catching it. (In reply to comment #27) > Here's what a "Subject" header of the Return Receipt looks like: > > Subject: Gavimo pažyma (laiškas atvertas) - > =?windows-1257?Q?=FEi=FBrim_=FEi=FBrim=2E=2E=2E?= Nice catch - you're just missing one step, where you say "here's what a Subject header looks like _unless you have already sent an email in the same session_." Filed (and to my astonishment, fixed) bug 462725 for that case. > One question: am I supposed to put a QP-string in my translation for this case, > or should I leave this responsibility to the application (Thunbderbird)? No, you shouldn't cheat your way around the bug like de and fr did, but since this is after all the poster child for "don't comment on closed bugs etc." you apparently should have filed a new bug saying "...still don't conform..."
Status: REOPENED → RESOLVED
Last Resolved: 12 years ago → 10 years ago
Resolution: --- → FIXED
(In reply to comment #30) I'll keep that in mind, thanks. ;) Also thanks for finally fixing.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Status: REOPENED → RESOLVED
Last Resolved: 10 years ago → 10 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.