The default bug view has changed. See this FAQ.

Line breaks in internet-cited text break UTF-8 quoted-printable characters

RESOLVED FIXED in Thunderbird 10.0

Status

Thunderbird
Message Compose Window
RESOLVED FIXED
6 years ago
6 years ago

People

(Reporter: Diomidis Spinellis, Assigned: Diomidis Spinellis)

Tracking

Thunderbird 10.0
x86_64
Windows 7

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

(Assignee)

Description

6 years ago
Created attachment 558067 [details]
The generated reply with the problem

User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.16) Gecko/20101125 Thunderbird/3.0.11

In a non-HTML reply to a message containing long lines with non-ASCII characters the internet-cited (>) line of the original text has a newline splitting UTF-8 quoted-printable characters in two.  This results in two illegal characters appearing in the mail message body.  In Thunderbird these appear with a question mark (?).  When Apple Mail displays such a message it fails to decode its contents as UTF-8 displaying it as gibberish.

Reproducible: Always

Steps to Reproduce:
1. Create a plain-text UTF-8 encoded message containing a line of 1000 non-ASCII characters
(e.g. αααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααα)
2. Send message to yourself
3. Reply to the message
4. Examine the sent message in the sent folder or the Outbox
5. The message will contain lines like the following
> ααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααααα�
�α 

Here is one line of the problematic internet-cited message, with the problem marked with "<------- ERROR"

> =CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=
=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=
=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=
=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=
=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=
=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=
=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=
=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=
=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=
=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=
=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=
=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=
=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=
=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=
=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=
=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=
=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=
=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=
=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=
=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=
=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=
=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=
=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=
=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=
=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=
=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=
=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=
=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=
=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=
=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=
=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=
=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=
=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=
=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=
=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=
=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=
=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=
=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=
=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=
=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE=B1=CE  <------- ERROR
=B1=CE=B1
>
(Assignee)

Comment 1

6 years ago
Created attachment 559756 [details] [diff] [review]
Fix bug by replacing conservative line length limit with the exact value

This patch increases LINE_BREAK_MAX from 990 to 1000 - MSG_LINEBREAK_LEN i.e. to 998, which is the correct value.  The previous conservative value would break lines that were correctly crafted during preceding steps to be within the standard-imposed limit of 998 chars (plus CR/LF) by inserting unneeded line breaks.  These could be inserted at the wrong place, for instance breaking a UTF-8 pair in half.  EnsureLineBreaks doesn't have enough knowledge about the body's encoding, as it deals with plain bytes.  Therefore, it should avoid inserting line breaks unless absolutely necessary.
Attachment #559756 - Flags: review?(dbienvenu)
Assignee: nobody → dds
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true

Comment 2

6 years ago
Comment on attachment 559756 [details] [diff] [review]
Fix bug by replacing conservative line length limit with the exact value

How common is this? It seems like it would be exceedingly rare.
Attachment #559756 - Flags: review?(dbienvenu) → review+

Updated

6 years ago
Keywords: checkin-needed
(Assignee)

Comment 3

6 years ago
(In reply to David :Bienvenu from comment #2)
> How common is this? It seems like it would be exceedingly rare.

I get it relatively often (20% of the time) when replying to email composed in Greek by an Apple MUA.  This MUA merges paragraphs sent in UTF-8 quoted printable flowed format into one long line, which then triggers the bug.
Checked in: http://hg.mozilla.org/comm-central/rev/1b02fbd2f3b0
Status: ASSIGNED → RESOLVED
Last Resolved: 6 years ago
Keywords: checkin-needed
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 10.0
You need to log in before you can comment on or make changes to this bug.