mbox parser gets confused by "From " at beginning of line
Categories
(MailNews Core :: Movemail, defect)
Tracking
(Not tracked)
People
(Reporter: g.s, Assigned: aceman)
References
Details
Attachments
(3 files)
Comment 1•20 years ago
|
||
Comment 2•20 years ago
|
||
Comment 3•19 years ago
|
||
Comment 4•19 years ago
|
||
Comment 5•19 years ago
|
||
Comment 6•19 years ago
|
||
Comment 8•18 years ago
|
||
Updated•18 years ago
|
Comment 9•18 years ago
|
||
Comment 10•17 years ago
|
||
| Assignee | ||
Comment 12•13 years ago
|
||
| Assignee | ||
Comment 13•12 years ago
|
||
Comment 14•12 years ago
|
||
| Assignee | ||
Comment 15•12 years ago
|
||
Comment 16•12 years ago
|
||
Comment 17•12 years ago
|
||
Comment 18•12 years ago
|
||
| Assignee | ||
Comment 19•12 years ago
|
||
Updated•9 years ago
|
Comment 21•6 years ago
|
||
Interesting bug. I tried to put some old mbox files under Local Folders that had been saved long ago from a tb ImapMail mbox file. It worked OK but several emails had today's time/date and appeared truncated and had no subject or other header info shown. I didn't know about this bug and traced it to several the leading From's in the message bodies.
Adding a space in front of the leading "From" fixed the problem.
I don't know if this is a Local Folder only problem since I probably would have noticed the problem back when the files were used in Imap.
Also, this does seem to be case sensitive since a leading "from" didn't cause a problem.
Comment 22•6 years ago
•
|
||
There seems to be a fix for the problem by defining "STRICT_ENVELOPE" here and rebuilding:
https://searchfox.org/comm-central/rev/551ff478566e07e5854ade095b6d7ac74cb8a0d4/mailnews/local/src/nsParseMailbox.cpp#659
Now for mbox files placed in Local Folders the parsing of the file ignores the "From " at the beginning of lines of the message body instead of erroneously starting a new message. One of my mbox files had about 20k messages and over 300 messages got fragmented and had to be manually edited. With STRICT_ENVELOPE defined there were zero problems and no manual cleanup.
This may also fix problems that originated from a "movemail" setup, but not sure. Maybe aceman's patch above is also needed for movemail even with STRICT_ENVELOPE defined? Looking some at his patch, it appears it is fixing up the mbox file created by movemail so a non-strict envelope parse of mbox file will work. So his fix will probably fix the problem for movemail but it will not fix a legacy mbox file manually placed into local folders.
Don't know why STRICT_ENVELOPE code is not enabled. Maybe it's too strict as per comment 18 that links to this: http://www.jwz.org/doc/content-length.html. Or maybe with it some 3rd party mbox imports would fail (but it does seem to be strict but somewhat tolerant)?
Edit: Well, I see why. I just ran into a 3rd party mbox file that separates messages with just "From " (no user or timestamp following). With strict envelope defined, only the 1st message in the mbox is visible in tb. Also, tb deletes all the other stuff from the file so that an mbox file that started out as multi-MB in size becomes a few KB in size. (The 3rd party file, is in mboxo format since lines beginning with "From" are ">From". TB just displays them as ">From" which is another issue. TB should strip off the leading ">" before displaying the email.)
FWIW this link provides the best explanation I've found of the various mbox* formats: https://jdebp.eu/FGA/mail-mbox-formats.html.
Comment 23•5 years ago
|
||
I see the same bug also when copying Imap mbox files to Local that were created by the same version of TB (78.4.3 in my case).
I recently added and synchronized an imap profile, then deleted the account and manually imported/moved the mboxes in the Local folder. Emails beginning with "From" are parsed incorrectly and added as unread fragments.
Is there a difference between imap mbox parsing and local folder mboxes?
Comment 24•5 years ago
|
||
Support for Movemail has been removed as of version 87 per Bug 1625741
Comment 25•5 years ago
|
||
I don't remember any details on this even though I researched and wrote about it 8 months ago. But I think my conclusion was, and seems to be supported by comment 23, that a problem exists independent of movemail (now removed). Not sure how big a problem it is or if it really needs to be fixed, so maybe WONTFIX is still appropriate.
Comment 26•5 years ago
|
||
(In reply to gene smith from comment #25)
I think my conclusion was [comment 22?], and seems to be supported by comment 23, that a problem exists independent of movemail (now removed). Not sure how big a problem it is or if it really needs to be fixed, so maybe WONTFIX is still appropriate.
I have no qualms about reopening. Let's see what others think.
Comment 27•5 years ago
|
||
The symptoms still apply, so if you have a mbox with such data it will display slightly wrong.
But the point is, you will not have such a file, because Thunderbird will escape the From before adding it to the mbox.
Description
•