275032 - Mailbox/file reads incorrectly and splits a mail, when a line in body begins with "From "

Status: RESOLVED EXPIRED

(Thunderbird :: General, defect)

Description

[Nikolaj Jørgensen]

When having a bunch of mails in a mailbox, and Thunderbird reads/creates a 
summary, it somehow fails to read it correctly.

Have a mail, containing/begins From inside the body of the message, which makes 
Thunderbird "split" up the mail into 2 pieces. The mail is complete, and the 
mailbox-file is correctly, so the mails are imported just fine.

Sample:
...
<p><font size=3><b>Ricochet Recharged:  Holiday fun with this "smashing" award-
winning game</b></font><p>
From the incredibly talented ...
<p>
...

It looks like a minor problem inside the summary-generation.

Comment 1

[Nikolaj Jørgensen]

Tried to change the "Target" to Thunderbird 1.0 (since the selectbox was empty 
on creation-time), bug the system doesn't let me do that either.

Comment 2

[WADA:World Anti-bad-Duping Agency]

Splitting occurs on mail receive?
If no, will split occur on "compact folder"? 
Or only when ".msf" recreation on restart after deletion of ".msf"?
(My guess is after ".msf" deletion only)

"Importing fine" is when import from Outlook family(read pst)?
I think it is false when import from Unix Mbox(Netscape 4.x, Mozilla including
NS 6.x/7.x, Eudra, Opera etc.), because ".msf" recration from Unix Mbox is required.

Mozilla seems to use "From " as key for the separater.
So this problem can be avoided in many cases if separater detection key is
changed from "From " to "From - ", since " - " just after "From" is rare in
usual data, although this change may be impossible when IMAP or import from Unix
Mbox of other mailers.
(Key of "From - " is valid since format definition says "-" after "From " and
date following it should appear.)
I cannnot imagine workaround other than editing folder file - insert a space
before "From", change "From" to "from", move "<p>" before "From" and so on. 
 
Anyway, this is the weakest point of Unix Mbox file.
If the separater was defined something like "tab"+"From"+"tab"+"-"+"tab"+... ,
problem relates to separater would hardly occur, and no escaping would be
required for almost all mails.

Comment 3

[WADA:World Anti-bad-Duping Agency]

(Additional description)
"-" after "From" is mozilla family only convention.
This field is non-space character(s) in format definition.
I do not know whether null is permitted as this non-space filed or not.
But at least Opera 6.x generated null as this non-space field when draft mail.
Opera's convention for this field is timestamp of mail, but this data is null
when draft, then became null.
I do not know whether this was improved on Opera 7.x or not.

Comment 4

[Nikolaj Jørgensen]

Splitting (or at least visually) occurs on creation of the summary file.
When looking in the folder, the real mail exists and can be read, but only 
until the part where "From" begins.
At the top of the list of mails (without any real/correct date), exists 
a "unknown" mail, which contents (of at least mailsource) is the rest, 
including "From". But it is only a "blank" mail that is shown in the Read Mail 
window.

In the mailbox-file, the mail looks just fine - without any split, or what-so-
ever. But as said, I think the error occurs, when creating the .msf/summary-
file, which then specify wrong "positions", 'cause of the From.

As for importing, I meant Outlook Express. It worked perfectly, as I also can 
see directly, by opening the mailbox-file through e.g. Notepad.

About the "From ", I thought that Thunderbird would look after the "From -" 
separater, but it seems like it doesn't do so. When it has reached to the end 
of the header, it starts to look after the next "From" (without " -").

I know I can edit the mailbox-file myself, but I cannot do so when receiving a 
file (through POP3 or IMAP).

To conclude up, Thunderbird doesn't look after it's own "From - ", but rather 
after "From". At least, so it seems at the moment.

I don't know if you are able to forge the error to occur.

Thanks in advance.

Comment 5

[David :Bienvenu]

how did the mail get into the mailbox? Did thunderbird download it? Lines
starting with "From " are supposed to be escaped by the message downloading code
so they look like "> From "

Comment 6

[Nikolaj Jørgensen]

This mail I'm talking about right now, got into the mailbox through the
OE-import-function.

But you are then saying, that it normally should convert it to > From?! I tried
prepending a ">", and actually saw that Thunderbird had added a ">" to the
notice-mail from bugzilla-daemon@, but then the > is shown in the Mail
View-window instead :(
It is the same, about the test I did, by adding a > myself, and for a couple of
other mails, that I found out *sighs*

I don't know why one of the mails are left out, since all is imported - accept
for those from bugzilla-daemon@.

Now I just have to either just get used to having > in front of all "From" in
the beginning of lines in the mails - or - get a lot of "splitted" mails.
It's like being between the devil and the deep blue sea.

Comment 7

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #5)
It maybe occurs on importing from Outlook family(and/or Opera) only.
(Mozilla believes "From " is escaped when import...)
Since Opera has binary control data after each mail(before separator) in his
Unix Mbox format mail file, there is no need to escape "From " data in his local
file use(this was true when I tested with Opera 6.0), although I do not know
whether Opera currently escapes "From " in data or not. 
This may be applicable to Outlook family.

Comment 8

[Nikolaj Jørgensen]

I am not sure why it has had errors during import, cause most of the mails have 
been converted... Unfortunately :(
So now there is a lot of >From, even when viewing the mail through Thunderbird. 
This includes received mails (like the first mail that was sent from bugzilla-
daemon@, when I created this bug-item). E.g.:
"...<p>
>From the incredibly talented ...
<p>...").

So it's not only Outlook, after what I can figure out (since it's not Outlook 
that receives/downloads new mails, but Thunderbird).

Anyway, I'll try make some more tests...

Comment 9

[Nikolaj Jørgensen]

Besides... why is Thunderbird showing it's own headers, like X-Mozilla-Status 
and so on? Don't think the "user" needs that, but rather the original 
mailbody/source.

I just got curious about it :-s

Comment 10

[Nikolaj Jørgensen]

Small update...

Seems like the error only exists in mails without multipart.

E.g. if there is only one body (=no boundaries), the error occurs.
If inside a multipart (=with boundaries), the mail is shown just fine - and 
with no need to put ">" in front of "From".

But as for the >, it probably should be left out entirely :-s

Comment 11

[John Bonnen]

Let me tip my hat into the fray on this one.  I use fetchmail combined with
movemail on my solaris 8 box to get my mail off of my company's outlook server.
 I get mail splitting and/or loss whenever there is a "From "  leading off a
paragraph off text.  I can reproduce this error locally in thunderbird from
within my own system by using  /bin/mail and a tty.  

I'm going to ask a question of the depths of my ignorance.  Can the meaning of
the "From " be inferred from the content length?

Comment 12

[Gervase Markham [:gerv]]

This is an automated message, with ID "auto-resolve01".

This bug has had no comments for a long time. Statistically, we have found that
bug reports that have not been confirmed by a second user after three months are
highly unlikely to be the source of a fix to the code.

While your input is very important to us, our resources are limited and so we
are asking for your help in focussing our efforts. If you can still reproduce
this problem in the latest version of the product (see below for how to obtain a
copy) or, for feature requests, if it's not present in the latest version and
you still believe we should implement it, please visit the URL of this bug
(given at the top of this mail) and add a comment to that effect, giving more
reproduction information if you have it.

If it is not a problem any longer, you need take no action. If this bug is not
changed in any way in the next two weeks, it will be automatically resolved.
Thank you for your help in this matter.

The latest beta releases can be obtained from:
Firefox:     http://www.mozilla.org/projects/firefox/
Thunderbird: http://www.mozilla.org/products/thunderbird/releases/1.5beta1.html
Seamonkey:   http://www.mozilla.org/projects/seamonkey/

Comment 13

[Gervase Markham [:gerv]]

This bug has been automatically resolved after a period of inactivity (see above
comment). If anyone thinks this is incorrect, they should feel free to reopen it.