HTML+plain-text (multipart) email message with wrong encoding does not upgrade to UTF-8 properly

RESOLVED FIXED in Thunderbird 27.0

Status

defect
RESOLVED FIXED
14 years ago
6 years ago

People

(Reporter: mozilla, Assigned: mkmelin)

Tracking

({intl})

Trunk
Thunderbird 27.0
Bug Flags:
in-testsuite +

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

Reporter

Description

14 years ago
User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20050513 Debian/1.7.8-1
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20050513 Debian/1.7.8-1

When I try to send an email message with the wrong encoding, I usually get a
message box saying "The message you composed contains characters not found in
the selected Character Encoding."... This saves me from sending Hebrew messages
that only appear as question marks.

When I send mail as both HTML and plain text, however, the dialog doesn't
appear. The HTML part of the mail goes out fine (sort of, it uses & escaping to
encode the characters), but the plain text part goes out as question marks.

The dialog should not if "plain text only" was chosen, but if "html only" was
not chosen.

Reproducible: Always

Steps to Reproduce:
1. Open the "compose" window
2. Write a message containing any character outside of ASCII
3. Choose an encoding for the mail that does not contain said character
4. Hit "send", and select "Send in plain text and HTML"
5. View the received message

Actual Results:  
Message-ID: <42EB3039.6090508@hamakor.org.il>
Date: Sat, 30 Jul 2005 10:46:01 +0300
From: Shachar Shemesh <removed>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20050513
Debian/1.7.8-1
X-Accept-Language: en
MIME-Version: 1.0
To:  removed
Subject: (no subject)
Content-Type: multipart/alternative;
 boundary="------------010801050603060908070401"

This is a multi-part message in MIME format.
--------------010801050603060908070401
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

??? ????? ????

-- 
Shachar Shemesh
Lingnu Open Source Consulting ltd.
http://www.lingnu.com/


--------------010801050603060908070401
Content-Type: text/html; charset=us-ascii
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
  <title></title>
</head>
<body bgcolor="#ffffff" dir="ltr" text="#000000">
&#1488;&#1495;&#1514; &#1513;&#1514;&#1497;&#1497;&#1501;
&#1513;&#1500;&#1493;&#1513;<br>

<pre class="moz-signature" cols="72">-- 
Shachar Shemesh
Lingnu Open Source Consulting ltd.
<a class="moz-txt-link-freetext"
href="http://www.lingnu.com/">http://www.lingnu.com/</a>
</pre>
</body>
</html>

--------------010801050603060908070401--


Expected Results:  
A warning message should have poped up, saying that some character are going to
come out wrong.

Comment 1

14 years ago
And a related (or dup?) TB bug:
https://bugzilla.mozilla.org/show_bug.cgi?id=295445
Summary: Sending HTML+plain-text email message with wrong encoding does not warn → Sending HTML+plain-text (multipart) email message with wrong encoding does not warn

Updated

14 years ago
Status: UNCONFIRMED → NEW
Ever confirmed: true
OS: Linux → All
Summary: Sending HTML+plain-text (multipart) email message with wrong encoding does not warn → [SeaMonkey/Suite] Sending HTML+plain-text (multipart) email message with wrong encoding does not warn

Comment 2

14 years ago
(In reply to comment #1)
> And a related (or dup?) TB bug:
> https://bugzilla.mozilla.org/show_bug.cgi?id=295445

As the submitter of bug 295445: Yes, it's a dupe, and the description here is
much better, so I wouldn't mind if my bug is marked as the dupe even though I
reported it first.

Comment 3

14 years ago
(In reply to comment #0)
> The dialog should not if "plain text only" was chosen, but if "html only" was
> not chosen.

I assume this was intended to read "The dialog should show not (just) if 'plain-
text only' was chosen, but if 'html only' was not chosen."
Assignee: mail → nobody
Component: MailNews: Main Mail Window → MailNews: Composition
Keywords: intl
Product: Mozilla Application Suite → Core
Hardware: PC → All
Summary: [SeaMonkey/Suite] Sending HTML+plain-text (multipart) email message with wrong encoding does not warn → Sending HTML+plain-text (multipart) email message with wrong encoding does not warn
Version: unspecified → Trunk

Comment 4

14 years ago
*** Bug 295445 has been marked as a duplicate of this bug. ***
Assignee

Comment 5

11 years ago
Still a problem with Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9b4pre) Gecko/2008020603 Lightning/0.6a1 Thunderbird/3.0a1pre ID:2008020603.
Assignee: nobody → smontagu
Component: MailNews: Composition → MailNews: Internationalization
QA Contact: mailnews.i18n
Product: Core → MailNews Core
Assignee

Updated

6 years ago
Summary: Sending HTML+plain-text (multipart) email message with wrong encoding does not warn → HTML+plain-text (multipart) email message with wrong encoding does not upgrade to UTF-8 properly
Assignee

Comment 6

6 years ago
Posted patch proposed fixSplinter Review
For reference: a few years back we used to have a dialog that asked if you wanted to upgrade charset when text didn't fit the given charset.
We changed that to just upgrade silently to UTF-8 a needed in bug 410333.

As can be noticed from the comments in this bug the conversion isn't always doing things right. Since there's a misfeature of turning non-fitting chars to numeric character references (NCRs) what's shown in html is (i suppose) usually correct if the font for the original charset can display it.

However, the headers all still say latin1, so when we send as multipart/alternative the html -> plaintext conversion for the HTML with NCRs try to convert it to latin1, which results in just questsionmarks in the plaintext part. (Possibly also some problem as the ncr-html would not be flagged as not fitting charset later when checking.)

There's no reason we should use NCRs, just making the text UTF-8 if needed should be just fine, and the right thing to do in any case.
Assignee: smontagu → mkmelin+mozilla
Status: NEW → ASSIGNED
Attachment #807146 - Flags: review?(irving)
Comment on attachment 807146 [details] [diff] [review]
proposed fix

Review of attachment 807146 [details] [diff] [review]:
-----------------------------------------------------------------

Sorry for the review delay; I've been trying to test this in my trunk builds but it's been weeks since I've been able to check out at exactly the right time to get a version of TB that builds and runs correctly, and I still don't have one.

The code looks fine to me, so let's get it into Trunk and see how it behaves.

::: mailnews/base/util/nsMsgI18N.cpp
@@ +403,5 @@
>    nsCOMPtr <nsISaveAsCharset> conv = do_CreateInstance(NS_SAVEASCHARSET_CONTRACTID, &res);
>    NS_ENSURE_SUCCESS(res, res);
>  
> +  // First try to transliterate, if that fails use '?' for "bad" chars.
> +  res = conv->Init(charsetName.get(), 

nit: trailing white space
Attachment #807146 - Flags: review?(irving) → review+
Assignee

Comment 9

6 years ago
https://hg.mozilla.org/comm-central/rev/8014698e4cd4 -> FIXED
Status: ASSIGNED → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 27.0
Assignee

Updated

6 years ago
Flags: in-testsuite+
You need to log in before you can comment on or make changes to this bug.