611666 - Mail forward is broken if forwarded email body contains ^M chars(Ctrl+M char, CR, 0x0D) (If stand alone CR exists in mail data, Tb tries to forward the mail data in base64, then bug 326303 occurs and message/rfc822 part data is corrupted)

Status: NEW

(MailNews Core :: MIME, defect)

Description

[Olivier Salaun]

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; fr; rv:1.9.2.12) Gecko/20101026 Firefox/3.6.12
Build Identifier: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; fr; rv:1.9.2.12) Gecko/20101027 Lightning/1.0b2 Thunderbird/3.1.6

I noticed that some mail could not be properly forwarded (as MIME). I realized that it happens when mail body contains ^M characters.

Reproducible: Always

Steps to Reproduce:
1.open a mail with body including ^M characters
2.hit the "forward" button ("transférer" in my French speaking thunderbird)


Actual Results:  
the forwarded mail body is corrupted ; it seems to be Base64 encoded, though the content-transfert-encoding is 8bit.

Expected Results:  
the forwarded mail body should not be corrupted

note that my thunderbird is configured to forward mail as attachment (as MIME)

Comment 1

[Olivier Salaun]

Attachment: sample mail with ^M chars

Comment 2

[Olivier Salaun]

Attachment: forwarded email with broken body

Comment 3

[WADA:World Anti-bad-Duping Agency]

What do you call by "^M char"? Binary called "Ctrl+M char" in Linux = CR(0x0D)?
> http://www.unix.com/unix-dummies-questions-answers/92034-how-find-m-control-m-character-unix-file.html
If so, phenomenon you saw is Bug 523796 on different but similar mail data.

(1) Tb tries to send message/rfc822 part in base64 encoded, when the data for the message/rfc822 is not "pure text"(e.g. 0x01 in mail data stream). "No CRLF, or mixture of CRLF, LF, CR" is treated as "not pure text", in order to avoid loss of some newline character or different newline character at mail recipient side, or to avoid RFC violation by Tb. IIRC, base64 encoding for "no CRLF, or mixture of CRLF, LF, CR" is done from Tb 3.0. Bug 523796 is report for "no CRLF in .eml file" case.
(2) When Tb tries to send message/rfc822 in base64, Tb doesn't encode message headers portion of mail data. Tb encodes mail only payload portion of mail data. This is known bug 326303.

Simplest workaround.
(1) Save mail in .eml file
(2) Convert stand alone CR and stand alone LF to CRLF
> http://www.unix.com/unix-dummies-questions-answers/92034-how-find-m-control-m-character-unix-file.html
(3) Drag&Drop the .eml file to thread pane of a local mail folder, and forward

Comment 4

[Olivier Salaun]

I confirm that the ^M character is a Line Feed (0xOD). 
The issue seems related to the ones you mention.

However the workaround you'reproposing is not realistic: when I forward a mail to somebody I don't check the CRLF in it before. It looks like a rather serious issue in Tb that breaks the forward feature.

Thanks for tha fast response.

Comment 5

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #4)
> However the workaround you'reproposing is not realistic:
> when I forward a mail to somebody I don't check the CRLF in it before.

You are right. My workaround is for "AFTER you notice problem will happen".
If you receive many mails of mixed CRLF/LF/CR or mail of non-text binary in mail data, and you need to forward them daily, a possible protection after you noticed Tb's bug is one of next.
(a) Save As Draft, Edit Draft, Send without any modification, if no problem.
(b) Send Later, View Message Source of generated mail,
    Send Unsent Message if no problem.
(c) Combination of (a) and (b).
(d) Forward mail as usual, without (a)/(b), and resend with my workaround
    when mail reipient said "received mail is corrupted".
(e) Use other mailer than Tb for mail forwarding or any sending.

(a)/(b) is also a protection from mail data corruption like next(also a Tb's bug).
  - User sets wrap line length=0, to avoid newline at around 76 bytes.
    As format=flowed is killed for some DBCS charset such as ISO-2022-JP,
    EUC-KR, it's still used by users of such charset.
  - User usually types very many characters without newline, without space
    (it easily happens on DBCS char world), or paste very long characters
    without newline/space.
  - In such case, mailer splits very long line to multiple lines(insert CRLF),
    because line length is limited by mail related RFCs.
  - Upon such splitting, Tb seems to split mail data at wrong position,
    if 3bytes code exists at special position(e.g. buffer boundary).
    3bytes code seems split by [CRLF] at mid of 3bytes code to two lines.
    Many 3bytes code is used if iso-2022-jp, because escape sequence is 3bytes
    code. 3bytes code is also used by utf-8.
This kind of issue produces corrupted/garbled mail data, so protection like (a)/(b) by user is needed, if such problem very frequently occurs in his environment.

If IMAP, (e) is a reasonable choice, because you can concurrently use multiple mailers on a PC for single IMAP mail box. I use SeaMonkey(for daily use), Thunderbird(mainly for testing), and Outlook Express(test only) for single Gmail IMAP mail box.

Comment 6

[Olivier Salaun]

Thanks a lot for your detailed propositions

Comment 8

[Ludovic Hirlimann [:Usul]]

Removing myslef on all the bugs I'm cced on. Please NI me if you need something on MailNews Core bugs from me.