463129 - Message opened from .eml file and forwarded as attachment is improperly base64 encoded. (mail in .eml file has no CRLF after close boundary of multipart mail or after last line of single part mail)

Status: NEW

(Thunderbird :: Message Compose Window, defect)

Description

[Mark Sapiro]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3
Build Identifier: version 2.0.0.17 (20080914)

I have a message with the following structure in a .eml file

Mime-Version: 1.0
Content-Type: multipart/mixed; boundary="============_-986412674==_============"

--============_-986412674==_============
Content-Type: text/plain; charset="us-ascii" ; format="flowed"


[plain text content]
--============_-986412674==_============

Content-Type: application/octet-stream; name="name.pdf"
Content-Disposition: attachment; filename="name.pdf"
Content-Transfer-Encoding: base64

[base64 encoded content]
--============_-986412674==_============--

I open this message in Thunderbird and forward it as an attachment. The forwarded message has the following structure:

MIME-Version: 1.0
Content-Type: multipart/mixed;
 boundary="------------050509080504090705000101"

This is a multi-part message in MIME format.
--------------050509080504090705000101
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

[My text describing the forwarded message]


--------------050509080504090705000101
Content-Type: message/rfc822;
 name="Fwd: file name.eml"
Content-Transfer-Encoding: base64
Content-Disposition: inline;
 filename="Fwd: file name.eml"

[headers of the original message including the following]
Mime-Version: 1.0
Content-Type: multipart/mixed; boundary="============_-986412674==_============"

[base64 encoded data which decodes to the part headers and part contents of the original message]
--------------050509080504090705000101--


Thus, the issue is the subparts of the message/rfc822 message content have been base64 encoded together, headers and bodies as one with nothing anywhere indicating this is even base64 encoded content.

Reproducible: Always

Steps to Reproduce:
1.Open a .eml file containing a multipart message
2.Forward the message as attachment
3 [review].
Actual Results:  
Defective message

Expected Results:  
Valid message

I will attach files x2.eml containing a simplified version of the original message and Sent.eml containing the result of opening and forwarding x2.eml

Comment 1

[Mark Sapiro]

Attachment: A minimal, multipart message in a .eml file

Comment 2

[Mark Sapiro]

Attachment: The resultant message when x2.eml is opened and forwarded as attachment.

Comment 3

[Paul Bryson]

I can also confirm experiencing this specific bug where the headers for the attached rfc822 message are not base64 encoded, but the rest of the message is.  The result is that Thunderbird will show the attached email message with the name and proper headers, but no body.

Comment 4

[Paul Bryson]

The bug does not happen with all messages, and I have not been able to figure out what triggers the bug and what doesn't.  I do know that if it happens when attaching a message, it will always happen when attaching that message.  The body of the message is properly encoded as base64, simply missing the headers.

Comment 5

[WADA:World Anti-bad-Duping Agency]

Do you set mail.file_attach_binary to true?
If yes, same problem as Bug 326303 in different situation?

Comment 6

[Mark Sapiro]

(In reply to comment #5)
> Do you set mail.file_attach_binary to true?

No.

> If yes, same problem as Bug 326303 in different situation?

It appears to be the same symptom, and quite likely from the same underlying cause, but even with mail.file_attach_binary False, this issue occurs.

Comment 7

[WADA:World Anti-bad-Duping Agency]

Do you use add-on? If yes, can you reproduce problem with new profile for test?
 1. Create new profile, add dummy POP3 account(with no Global Inbox use)
 2. Execute same operations with same .eml, and "Send Later"
How about Tb nightly latest-trunk build?
> http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/latest-trunk/
> You can test by Unzip of win32.zip build only.
> Please never test with your daily use profile. Test with the new profile for test.

Comment 8

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #1)
> A minimal, multipart message in a .eml file

No [CRLF] after close boundary string in your test mail.
> --============_-986412674==_============--[End_Of_File]
I think it should be next;
> --============_-986412674==_============--[CRLF][End_Of_File]
Can you produce your problem with newlines at bottom of the .eml file?

Comment 9

[Mark Sapiro]

(In reply to comment #8)

> Can you produce your problem with newlines at bottom of the .eml file?

No. I don't think I can. I have tried with a couple of messages, both with and without a base64 encoded part in the original, forwarded message, and as long as the final boundary is terminated with <CRLF>, I don't see the problem.

This is good to know as far as T'Bird's behavior is concerned, but I note that the definition of multipart syntax in section 5.1.1 of RFC 2046 shows that the CRLF and any epilogue following the terminating boundary are optional.

Comment 10

[WADA:World Anti-bad-Duping Agency]

Adding "No CRLF after close boundary" to summary based on Comment #9, for ease of search.

Comment 11

[Michael A. Pasek]

(In reply to comment #9)
> This is good to know as far as T'Bird's behavior is concerned, but I note that
> the definition of multipart syntax in section 5.1.1 of RFC 2046 shows that the
> CRLF and any epilogue following the terminating boundary are optional.

Hmm...the RFC seems ambiguous.  In one place it states that the boundary line consists of "a line consisting entirely of two hyphen characters ("-", decimal value 45) followed by the boundary parameter value from the Content-Type header field, optional linear whitespace, and a terminating CRLF."  

RFC1521 (RFC2046's predecessor) defines the body and close-delimiter:
   multipart-body := preamble 1*encapsulation
                  close-delimiter epilogue

   close-delimiter := "--" boundary "--" CRLF ; Again, no space by "--",

RFC2049's Appendix B, which summarizes the changes from previous specifications says:
    (7)   The BNF for the multipart media type has been
          rearranged to make it clear that the CRLF preceeding
          the boundary marker is actually part of the marker
          itself rather than the preceeding body part.

Yet, it makes no mention of the (probably unintended) change in that rearrangement that eliminated the (previously required) CRLF in the close-delimiter.

Comment 12

[Mark Sapiro]

I have just observed that this bug does not depend on the attached message being multipart. It occurs in TBird 3.1.10 with a simple text/plain message in the .eml file if the last line of the file is not terminated with a <CRLF>.

I have edited summary and version to reflect this.

Comment 13

[WADA:World Anti-bad-Duping Agency]

If mail is sent or transferred via SMTP, or mail is received via POP3, "no new line char at end of mail data stream" can't occur, because "new line at end" is mandatory in SMTP and POP3.
It occurs only when;
  A mail is generated by Web mailer.
  The Web mailer doesn't put new-line at end of mail data.(Gmail is an example)
  The mail data is passed to IMAP server without SMTP data transmission. 
  The mail is passed to Tb(and to other mailer) via IMAP. (Gmail supports IMAP)
  The mailer of mail recipient saves mail data in .eml file as-is.
If "mailer of mail recipient" is Thunderbird, Tb perhaps adds "new line at end of mail" when IMAP folder of Offline-use=On, because Unix Mbox format is used for offline-store file.
However, when Offline-use=Off, Tb uses mail data passed from IMAP server as-is. So, such mail is held in IMAP Offline-use=Off folder of Tb, this bug can happen.

Confirming.

Other than Content-transfer-encoding:7bits(or 8bits) is prohibited for message/rfc822 part by RFC.
However, if a mailer wants to send mail data as message/rfc822 part without any alteration from original mail data stream, base64 encoding is needed when not-well-formed mail data.

Following may be a solution.
  Content-Type: application/octet-stream
  Content-disposition: inline(or attachment) filename="xxx.eml"
  Content-Transfer-Encoding: base64

Note:
When Tb tries to send message/rfc822 part in base64, bug 326303 still occurs.