593907 - Some characters get broken when importing messages with non-standard-conformant long lines

Status: RESOLVED FIXED

(MailNews Core :: Import, defect)

Description

[Mike Kaganski]

User-Agent:       Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0)
Build Identifier: 

If a message contains too long line (that violates rfc822 being longer then 1000 characters), when importing such a message (I did it from Outlook, don't know if other paths are affected), TB inserts CRLF every 990 characters. It does so without respect to where in the text it does insertions. There are many ways it modifies the original message:
1. It may insert CRLF in between characters of a word. Then, depending of the mode (plain/html) the word will become split over two lines or just divider by a space.
2. It may insert CRLF inside an HTML tag. Then the result may wary (formatting may be lost, table may become broken, etc) depending on which tag was affected which way.
3. It may insert CRLF between parts of a single surrogate pair in a multi-byte charset (like ISO-2022-JP or utf-8). Then the letter that consisted of the two bytes gets broken and instead, funny characters may appear.
The code that splits the text is in the nsMsgSend.cpp (nsMsgComposeAndSend::EnsureLineBreaks()). Although the source messages are, of course, violating the specification, they need not to be sent any further, instead, from the TB's point of view, they are being recieved. The import could be made without such a modification. Anyway, a user has such messages and wants to keep them while switching the mail agents, so it would be better to keep the archived mail intact.

Reproducible: Always

Comment 1

[Mike Kaganski]

Attachment: Outlook message having such a non-standard long line (from Microsoft)

Comment 2

[Ludovic Hirlimann [:Usul]]

Describe importing please ? did you import it thought tools -> import or you just received the email ?

Comment 3

[Mike Kaganski]

I used tools -> import. While working on the patch for Bug 207156, I had a chance to test importing of hundreds of mails in different charsets and line lengths.

Comment 4

[Makoto Kato [:m_kato]]

related to bug 355209.  This will be fixed by another bug.

Comment 5

[Ludovic Hirlimann [:Usul]]

(In reply to comment #4)
> related to bug 355209.  This will be fixed by another bug.

Which one ?

Comment 6

[Wayne Mery (:wsmwk)]

is this bug dependent on work to be done in bug 355209?
and, does fixing this block any work in Bug 207156?

(In reply to comment #5)
> (In reply to comment #4)
> > related to bug 355209.  This will be fixed by another bug.
> 
> Which one ?

I think he means bug 553526 and bug 26734.

Comment 7

[Mike Kaganski]

(In reply to comment #6)
> and, does fixing this block any work in Bug 207156?

No. I just came across this problem while working on Bug 207156, but they are entirely independent.

Comment 8

[Jorg K (CEST = GMT+2)]

This bug is no longer relevant. nsMsgComposeAndSend::EnsureLineBreaks() has been removed as part of fixing the "Asian Crisis" in bug 1225904:
https://hg.mozilla.org/comm-central/rev/7e4f38edda2ff97ad32c99d68b8c387708940911#l2.124

A work-around for the chopping was in fact implemented in bug 207156:
https://hg.mozilla.org/comm-central/rev/90c3929c5b5d#l8.1272

Comment 9

[Jorg K (CEST = GMT+2)]

This was fixed in bug 207156 as mentioned in bug 207156 comment #64. Hence setting target to Thunderbird 6.