310583 - Invalid header created transferring messages from local to IMAP folders because "From" preceeded by ">"

Status: RESOLVED WORKSFORME

(MailNews Core :: Backend, defect)

Description

[cpl1]

User-Agent:       Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.7.8) Gecko/20050512 (No IDN) Firefox/1.0.4
Build Identifier: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.7.8) Gecko/20050512 (No IDN) Firefox/1.0.4

If you create a standard mbox-format mailbox in another mail client -- e.g. pine
or mutt -- and drop it in the Local Folders folder of a Thunderbird
installation, Thunderbird will successfully open it and treat it just like any
other Local Folder.  However, if you try to transfer a message up to an IMAP
store, you may (depending on your server) get an "invalid header" error from the
server.  (This happens with the Mirapoint IMAP server, but not with UWash;
haven't tested others.)

The problem appears to be that other clients create mbox files with "From "
lines in the format

From [e-mail] timestamp

as opposed to Thunderbird's

From - timestamp

When Thunderbird encounters an email address in the "From " line, it quotes it
with ">".  Since this is, in fact, an invalid header, IMAP servers which are
sensitive to this will reject the transfer.  This is obvious if you transfer 
the message to a UWash server which uses mbox format to store messages -- you
can see that the headers now read

From email@address timestamp
>From email@address timestamp
Received: ...

I've verified this behavior under Thunderbird 1.0.6 on both Windows and Linux.

Reproducible: Always

Steps to Reproduce:
1.Create mbox-format mailbox in mutt or pine
2.Transfer mailbox to "Local Folders" folder in a Thunderbird profile
3.Transfer message from mailbox up to an IMAP store

Actual Results:  
Using a server tolerant of invalid headers (e.g. UWash), the message is
transferred up, but now includes a bad header in the format ">From ".

Using a server intolerant of invalid headers (e.g. Mirapoint), the transfer
fails with an "invalid header" error.

Expected Results:  
"From " line should be discarded and a new envelope generated during the
transfer.  This is standard behavior in other mail clients that can read
mbox-format mailboxes.

Comment 1

[Szókovács Róbert]

cyrus imap server barfs on this error too.

Comment 2

[cpl1]

New data.  Under Thunderbird 1.5, this bug happens with Thunderbird-created mailboxes -- if you transfer a message down from an IMAP mailbox to a local mailbox, then transfer it back up, the transfer fails (if your IMAP server is sensitive to bad headers).  The cause is identical: Thunderbird quotes the "From " line, rather than deleting it.  This has the potential to create some serious problems for us.

Comment 3

[Jaime Velasco]

(In reply to comment #2)
> New data.  Under Thunderbird 1.5, this bug happens with Thunderbird-created
> mailboxes -- if you transfer a message down from an IMAP mailbox to a local
> mailbox, then transfer it back up, the transfer fails (if your IMAP server is
> sensitive to bad headers).  The cause is identical: Thunderbird quotes the
> "From " line, rather than deleting it.  This has the potential to create some
> serious problems for us.

Also happens with 1.0

Comment 4

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

Reporters, could you please test with the latest supported 2.0.0.x / trunk nightlies and see if the issue still occurs?

Comment 5

[cpl1]

As of 2.0.0.14, messages in Thunderbird-created mailboxes can be transferred back up to IMAP with no errors.  However, an mbox-format file dropped into the Thunderbird Local Folders still produces the same error.  Comparing the two, it appears that Thunderbird is successfully removing envelopes in the format

From - datestamp

it uses for its own mbox delimiter, but still quotes

From [e-mail] datestamp

when trying to transfer messages from an imported mbox.

Comment 6

[cpl1]

Additional info: Still broken under the 3.0.0.a1 trunk build.

Comment 7

[Nikolay Shopik]

version 3.0a2pre (2008062603)

Comment 8

[Wayne Mery (:wsmwk)]

cpl writes:
I can verify that the current version of Thunderbird for Linux does not show this issue against either Exchange or whatever pobox.com/fastmail.com is using.  Unfortunately I don't have access to Linux-based IMAP servers any more, so I can't test this fully, but it looks promising -- Exchange probably doesn't test anything, but I'd expect pobox to be reasonably strict.