416166 - Local folder email headers poisoned with ">From -" line

Status: RESOLVED DUPLICATE of bug 147191

(SeaMonkey :: MailNews: Backend, defect)

Description

[Michael Baffoni]

At some point, some of my SeaMonkey clients started inserting a line(s) starting with ">From - <dates>" into the headers of the local saved folders (and probably extra x-mozilla-status lines), e.g.:

From - Sun Jan 20 08:27:18 2008
X-Mozilla-Status: 1011
X-Mozilla-Status2: 00000000
>From - Wed Jan 16 11:04:38 2008
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000

Now when we attempt to copy those email messages up to our IMAP server, it won't accept them because it thinks the headers are corrupt, although the SeaMonkey email client can read them fine.  Manually deleting the line ">From - Wed Jan 16 11:04:38 2008" allows me to copy the email message up normally, but then I have to do this individually to each message within the mail folder (times however many mail folders are affected).

How do I get SeaMonkey (or Thunderbird!) to remove these lines prior to uploading to the IMAP folder?  Does someone have a simple script I can use to sanitize the local mail folders so that I can then upload them normally?

We are trying to move forward on a move of all critical data to a network centric storage, but our highest-profile users can't move their mail because of this bug!

The corrupt folders were noticed on SeaMonkey clients (1.1 and later), however, upgrading to latest SeaMonkey (SeaMonkey 1.1.7, Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1.11) Gecko/20071128 SeaMonkey/1.1.7) or migrating to Thunderbird (version 2.0.0.9 (20071031)) did not enable the uploading of the messages.  Only the manual modification of the local mail file seems to have any affect.

Comment 1

[WADA:World Anti-bad-Duping Agency]

">From -" line and dup'ed X-Mozilla-Status/Status2: is probably DUP of Bug 381472.
"IMAP server won't accept >From -" is DUP of Bug 373941. 

Comment 2

[WADA:World Anti-bad-Duping Agency]

First report of "IMAP server won't accept >From -" was Bug 147191.
See dependency tree for Bug 381472.

Comment 3

[Serge Gautherie (:sgautherie)]

Can you reproduce with SeaMonkey v2.0a2pre ?

Comment 4

[Michael Baffoni]

Depends on what you want me to reproduce.  Do you want me to see if I can upload a poisoned local file email to the IMAP server?  If you are referring to the creation of the extra ">From -" line, I don't know how to stimulate that error.

Comment 5

[Serge Gautherie (:sgautherie)]

[Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9a5pre) Gecko/20070515 SeaMonkey/1.5a] (nightly) (W2Ksp4)

Got one in my Junk folder today:
{
From - Thu Sep 25 21:14:38 2008
X-Mozilla-Status: 0008
X-Mozilla-Status2: 00000000
>From - Thu Sep 25 21:14:38 2008
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
}

*****

(In reply to comment #4)
> If you are referring to the creation of the extra ">From -" line,
> I don't know how to stimulate that error.

Sure, I only would like to know if it happens with current 2.0a2pre too.

Comment 6

[Tony Mechelynck [:tonymec]]

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b1pre) Gecko/20080925001738 SeaMonkey/2.0a2pre

I just checked all files in the Mail folder of my profile and all its subfolders, searching for the string >From in column 1 of a line. There were 225 hits, but all of them false alerts, corresponding to the word From at the start of a line somewhere in the body of an email -- none of them were in the headers part of an email in an mbox file.

IIRC, some of the emails stored there were fetched by Thunderbird 1.5 then transmitted to Thunderbird 2.0 and SeaMonkey 2.0a1pre (and, today, 2.0a2pre). Either I'm lucky or I didn't do whatever it takes to get the particular kind of mbox corruption you guys got.

After closing SeaMonkey, you might try the following:
1. backup the Mail subdirectory of your profile (see http://kb.mozillazine.org/Profile_folder_-_SeaMonkey if you don't know how to locate your profile).
2. use some external tool, if you can locate one, to remove any sequence of full lines (as many as possible, but not including any empty line) from "From " in column 1 (excluded) to ">From " in column 1 (included), in any file in that Mail directory which has no dot in its name.
3. Then delete all *.msf files (they are indexes of the corresponding extension-less files, we just made them invalid, but the mailer can rebuild them, and will if it doesn't find them).
4. Reload SeaMonkey and see if it can read its stored mail.
5. Try to upload the "problematic" emails to your IMAP server using drag-and-drop.
6. Does it complain for any >From lines?

If the above doesn't work, you can restore the backup from step 1 and be no worse than before (except that any _new_ mail stored _locally_ will be lost).

I suspect the corrupt mboxes were created by bug 381472, which would mean that:
- it's normal that I don't get the bug, since all my mail accounts use POP.
- you might avoid the bug (until it is fixed) by making sure that none of your IMAP servers is set "for offline use".

I suspect this is a dupe of bug 147191 or at least blocked by bug 381472.

Comment 7

[Wayne Mery (:wsmwk)]

Michael, do you agree this is a duplicate?

Comment 8

[Michael Baffoni]

I agree (whew! took some reading, lots of potential bugs in there).

BTW, I don't know how I missed the earlier responses: yes, by manually deleting the lines containing the >From line in them, the messages upload normally, so that line is a minimum and necessary issue to resolve to fix this bug.