Closed Bug 50155 Opened 24 years ago Closed 16 years ago

.msf's get corrupted when moz crashes

Categories

(MailNews Core :: Backend, defect, P3)

defect

Tracking

(Not tracked)

RESOLVED INCOMPLETE
Future

People

(Reporter: rmosher, Unassigned)

References

Details

(Keywords: dataloss, Whiteboard: closeme 2008-12-18)

after mozilla has crashed, all msf's seem to be messed up.  I have to rm *.msf
and reenter mozilla. if i dont remove all the msf's, all mail gets
sorted/filtered to inbox and they dont show.  the only way to make them show is
to remove the msf's and restart mozilla
I have been unable to reproduce this bug, but it seems like a critical issue if
it really does occur.  So, I'm confirming it so that the mail team can triage as
appropriate.
Status: UNCONFIRMED → NEW
Ever confirmed: true
i don't see this....
Target Milestone: --- → Future
Confirmed.  I've had my newsgroup .msfs corrupted three times in the past two 
weeks, each time immediately following a hard lock-up of my system while 
Mozilla was open.
cc'ing bienvenu
No, I don't think crashing is corrupting .msf files, unless the crash happens
during the writing of the .msf files. What's more likely is that the filtering
problem occurs because the INBOX.msf database is out of sync with the INBOX
berkeley mailbox, so that if we have to rebuild the INBOX.msf file on startup
(you're using POP3, right?) while we're getting new mail, bad things happen.
Is this still occuring in the latest nightlies?
i had a serious problem with this recently too on imap (see bug 73569).

Even today (build 2001-04-10, winNT), i had a message that couldn't be viewed
after deleting a message (mark as deleted) and then quickly (1-2 secs) selecting
the next msg. Related?
Blocks: 74644
I have a corrupted msf file to. 

Moreover, each time Mozilla tries to read it, it crashes. 100% reproducible.

I can send the msf file if anyone is interested.
I have a corrupted msf file to. 

Moreover, each time Mozilla tries to read it, it crashes. 100% reproducible.

I can send the msf file if anyone is interested.
I'm running mozilla 0.9.2 on Windows 98 release with the same problem.  The
email could not be seen or found in the Inbox.  The Inbox file was recieving the
data an d after renaming the inbox.msf file was rebuilt and I could again access
my email. The recreated .msf file was 85k smaller (395k vs 480k).  In addition
the old .msf file had not been updated in 5 days though email had been received
and was available after the rebuild.  Mozilla would not crash but this happened
after I attempted to compact the mail folders.  I have the before and after .msf
file if someone would like to look at them.
ok. I reported and saw fixed http://bugzilla.mozilla.org/show_bug.cgi?id=85632 ,
which was very similar, but had clearly identifiable causes. I'm still seeing
this sometimes. 

I propose dataloss (since it loses all my filtered data, and I have to delete
and rebuild all my .msfs, a real pain), and
cc'ing naving since they seem to have a lot of clue about this.

Next time it happens I'll try and pin down the steps more clearly. Is there any
kind of logging I could usefully turn on?

I'm probably seeing this about once a week

Mozilla/5.0 (Windows; U; Win 9x 4.90; en-US; rv:0.9.5+) Gecko/20011112


This issue appears also with M0.9.7 {2001122106} on Win98. And this is really
annoying. (Change OS to All?)
Based on comments, setting OS/Platform = All/All, Severity = critical, keywords
+= dataloss.
Severity: normal → critical
Keywords: dataloss
OS: Linux → All
Hardware: PC → All
can anyone reproduce this bug? It is almost 1.5 years old.
This is still in Build #20020117.
When mozilla crashes, some of news thread will never set to be read.
Therefore, some thread will always displyed unless I delete msf file.

This is very annoying...
In fact with 0.9.7 I had a situation where attempting to open a particular
newsgroup would lock the newsreader forever. Had to kill it. Eventually hit on
deleting the .msf file, which fixed things up.
This is a bad bug.  Just last weekend, I lost some very important emails when I 
lost connection during download of about 30 emails.  I reconnected and 
continued the download only to find that all the message bodys no longer 
matched the subjects.  Before I realized this, I deleted (what I thought) was 
the typical spam messages, only to find out that their bodys were my 
*important* emails!  This was with Netscape 6.2, but I come to find out that 
Mozilla STILL has this problem.

Data Loss.  This is no good folks!

I report this as a regular user would, having (at the time) NO idea about 
the .msf files and their flaky-ness.  You can't leave this bug out there on an 
unsuspecting public, and no company in their right mind would use this for 
their Mail application.
What he said! I had this happen to me again yesterday on 2002021203 on XP

it doesn't even seem to be getting any attention at all. 

dataloss, dataloss, dataloss.
There are about 10 different problems in this bug. Completely different
problems. It's helpful to us if people don't start throwing their unrelated
complaints into bugs, but rather, open new bugs.

Now, the bug that cmorley describes seems to have to do with our handling of
getting disconnected from the server while doing a pop3 download, which is the
first mention of that bug that I can see in this bug report. I assume cmorley is
doing a pop3 download, though he doesn't specify imap or pop3. Navin, could you
try that? Also, cmorley and stefan, it might be useful to Navin to know things
like whether you've got leave on server set, and how you were disconnected from
the server. Was it an async connection that just got dropped?
bug 125503 filed for interruption of pop3 download leaving summary file invalid
wrt the local mailbox.
ok, see my comment #11, 
and
http://bugzilla.mozilla.org/show_bug.cgi?id=85632

There is some unrelated stuff in this bug, but mostly the comments accurately
describe the issue in the summary. I don't see it as often as I used to (every
couple of days), but I still see it (bi-weekly). I've trying hard to identify a
reproducible condition, but I can't.

All that I can say is: sometimes my .msfs get corrupted, and I get dataloss
results as described by cmorley. I can't confidently say however that this only
occurs when a pop3 download is interrupted.

me: 2 pop3 accounts, always download from server, 30 folders, several thousand
messages per folder. 20 filters.

I'll try and investigate the sudden disconnection theory, although it's painful
to do so. maybe I'll set up a dummy acct for testing against, since my mail is
important. any logging I could switch on to help?
Workaround: If you see that subject lines and text bodys don't match any more
(or any other signs of .msf corruption), do the following:
1. Close Mozilla.
2. Delete the corresponding .msf file.
3. Restart Mozilla.
4. Do this before you fetch any new mail: Select the corresponding folder in
MailNews. Mozilla will correctly rebuild the .msf file.
In the local mail folder backend, I would suggest that when displaying or
deleting local messages, we should do a quick sanity check that the .msf db
header has the correct offset for the message, by checking that there's a "From
" at the offset in the local mail folder. If not, then we should abort the
message load/delete operation, and invalidate the summary file. I'm not sure if
that would catch these corruptions or not, since I don't know if the header has
a correct offset into the folder, but there's just a different message there, or
if the header offset is completely off. If the header offset is completely off,
I would expect not only the wrong message to get displayed, but often the
message to be missing headers, or otherwise display incorrectly. For example,
you could see message headers displayed as message text. Does anyone see that?
Mozilla 0.9.9 {Build ID: 2002031104}

Win98 crashed -> Mozilla rebuilt the msf for each folder to which mails were
moved by filters. Annoying.
Sascha, are you using pop or imap? The .msf files weren't corrupt in that case;
they were out of date (there's a time stamp in the .msf file that has to
correspond to the time stamp of the mailbox file, in order for us to make sure
no one has changed the mailbox file, and that our .msf file is in sync with the
mailbox file).
I used pop3. Is there a bug# for this?
I use IMAP and today my inbox got corrupted (of course 
only the local copy, not on the server): Some messages
don't have a body at all, some consist of more than one.
The message subjects don't match the bodies any more.

I don't now if it happened yesterday when mozilla hung
and i had to force end the process _or_ today when I 
cleaned up the inbox.
David, 
I habe _some_ folders configured for offline use, the inbox is one of them.
That's what I meant with "local copy".

I have "clean up inbox" set (if that's what you to refer to as "auto-compact").
I have also set "sync before going offline" but I can't find that switch
any more :-). Saying that, I rarely ever switch to offline before hanging
up the isdn line because I am too lazy to tell Mozilla that I am going offline.
In my opinion, it should figure that out by itself.

I managed to fix the broken inbox folder (local copy) by searching the .msf file,
deleting it and starting MozillaMail again. It downloaded the folder and
all is sunshine now.

Cheers
Daniel
Confirmed with all versions of Mozilla (with Windoze 2000 operating system). I
had data corruptions of my INBOX **several** times after Mozilla crashed or
(more frequent) Windows crashed. It turned out that the only way to get MailNews
working is to delete *.msf files. In addition it was somtimes necessary to
remove the INBOX file (!).
It seems that Mozilla also has a 2GB (31 bit) limitation for the maximum file
size of mail files (Windows2000, NTFS filesystem).
See also my bug report of today.  
This shouldn't be a problem since around 1.6, thanks to fix for bug 221797. Do
anybody still see it? Of course, if the crash occurs in the middle of a physical
write to the disk, nothing can help you.
Product: MailNews → Core
Assignee: mscott → nobody
QA Contact: lchiang → backend
Product: Core → MailNews Core
anyone still see this on a *regular* basis?
Whiteboard: closeme 2008-12-18
RESO INCO due to lack of response to last question. If you feel this change was made in error, please respond to this bug with your reasons why.
Status: NEW → RESOLVED
Closed: 16 years ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.