500479 - some .eml files can't be dragged to imap folders

Status: NEW

(MailNews Core :: Backend, defect)

Description

[Magnus Melin [:mkmelin]]

Attachment: testcase, bug370077.nodrop.eml -> message contains bare newlines

Follow-up to bug 171907.

Some messages can't be dragged to imap folders. 
These messages are from possibly now quite correct (it's from stuff i've triaged and whatnot). Either way, it's obviously available in the wild.

Comment 1

[Magnus Melin [:mkmelin]]

Attachment: testcase, bug399278.nodrop.eml -> Message contains invalid header

Message contains invalid header
.

Because a header value is empty?

Comment 2

[Phil Lacy]

I can drag and drop these to gmail.  What error am I looking for.

Comment 3

[WADA:World Anti-bad-Duping Agency]

Line-Break of the two .eml file is [LF] (0x0A), instead of [CRLF] (0x0D0A), and both has some [LF][LF] (some consecutive [LF]). Fisrt one is;
> last_mail_header: ...[LF]
> [LF]     <= corresponds to delimiter of mail headers and mail payload.
> first_mail_payload_line[LF]

Data in Tb's mail folder file is "data stream from POP3/IMAP/NNTP" or "data stream to SMTP/NNTP" (never simple text file). So, if these .eml file is copied in Tb's mail folder file, it's passed to IMAP server as-is by Tb.
I think "data in .eml file" is "data stream defined by RFC 2822" too, but I don't know spec of ".eml file". What is done for Line-Break-Character in import code?

If mail with digital signature, I think change from [CRLF] to [LF] by .eml file creator means "corruption of mail data". If mail with digital signature, and if [LF] is correct data, conversion from [LF] to [CRLF] by Tb means "corruption of mail data".
What should be done for .eml file with [LF](or [CR]) instead of [CRLF]?

Comment 4

[Magnus Melin [:mkmelin]]

On drop you get an alert
"The current command did not succeed.  The mail server responded:Message contains bare newlines"

Gmail doesn't care, but at least 2 of my other imap accounts do complain.

Comment 5

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #4)
> Gmail doesn't care, but at least 2 of my other imap accounts do complain.

As seen in Bug 83396, Bug 301010, Bug 402759, Cyrus is strict about RFC. Cyrus?

Comment 6

[Phil Lacy]

Should this bug be duped and the examples moved to the other bugs or those bugs consolidated (duped) to this one?

Comment 7

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #6)

(A) & (B) is cause of all "bare newline" and is common, but (C) / (D) depends on case/bug. I think bugs are better to be kept separately.
 
> (A) Cyrus(or some IMAP servers) doesn't accept [LF] only data line.
> (B) Tb passes thru such "[LF] only data line" to IMAP server.
      Sending "[LF] only data line" to IMAP server itslef is Tb's RFC violation.
> (C) Who passed the illegal "[LF] only data line".
>     Bug 301010 : MS Exchange Server
>     Bug 402759 : Multiple cases are reported.
>         - POP3 server, IMAP server
>         - User, by placing [LF] in cutomized mail header data of prefs.js
>     This bug   : Tb's "Drag&Drop of .eml" code 
> (D) Who generated the illegal "[LF] only data line" initially.
>     Bug 301010 / Bug 402759 : Mail sender
>     This bug :  I think mailer who saved the .eml file.
>                 Tb on Linux?

Comment 8

[Magnus Melin [:mkmelin]]

On of the servers was iPlanet Messaging Server i think. Not sure about the other one.

Thinking some more about this, i'm not sure these errors are fixable. For instance if we modify the line endings signed messages will not be valid i suppose. Fixing headers should be ok though.

I'm also unsure if it's worth duping since in this case the data is from the .eml and is what it is, we didn't necessarily have anything to do with the creation of the message.

Comment 9

[WADA:World Anti-bad-Duping Agency]

Bug 503271 exists. i.e. Tb on Linux won't convert [CRLF] to [LF] upon save to .eml.
If user has .eml file with [LF] by other program, and if IMAP server rejects it, user should convert [LF] to [CRLF] before pass the .eml file to Thunderbird.
  sed -e 's/$/\r/' inputfile > outputfile               # UNIX to DOS  (adding CRs)
  perl -pe 's/\r\n|\n|\r/\r\n/g' inputfile > outputfile # Convert to DOS
  ( see http://en.wikipedia.org/wiki/Newline )

Comment 10

[Wayne Mery (:wsmwk)]

(In reply to comment #9)
> Bug 503271 exists. i.e. Tb on Linux won't convert [CRLF] to [LF] upon save to
> .eml. ...

bug 503271 was closed invalid. so is this bug invalid as well?

Comment 11

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #10)

Bug 503271 is an evidence that culprit of LF in the .eml file is never Thunderbird.
Server who rejects bare LF never violates RFC. From perspective of IMAP, who violates RFC is Thunderbird. Problem is;
 When other software generated .eml file in which newline is LF instead of CRLF,
 what Tb should do upon upload of mail data in the .eml file to IMAP server.
My opinion is comment #9. (i.e. WONTFIX for .eml file case)