Open Bug 339805 Opened 18 years ago Updated 6 years ago

inboxes, filters and messages get corrupted and sometimes completly lost when disk space is low

Categories

(MailNews Core :: Backend, defect)

x86
Windows XP
defect
Not set
critical

Tracking

(Not tracked)

People

(Reporter: simone.avogadro, Unassigned)

References

(Blocks 1 open bug)

Details

(Keywords: dataloss)

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3
Build Identifier: Thunderbird 1.5.0.2 (20060308) - Italian

when disk space is low and message folders are big sometimed the message folders will get damaged, messages will be permanently lost and message folders will require lots of time to be placed back at work

Reproducible: Sometimes

Steps to Reproduce:
Thunderbird configuration:
- I got multiple mail accounts
- all of them have filters set
- all but one use the global inbox
- 3 of them read data simultaneously fomr the same server
- antivirus: AVG 7.1 network edition
- junk mail controls globally enabled
- multiple filters from different accoutns will redirect messages to the same folder
- some filters will manually redirect content to the junk folder
- some folders are very big (500-700+ Mb)
- HD space is quite low (800-900Mb)
- profile is located in a custom folder and does NOT contain spaces in the path name

Steps to reproduce:
- open thunderbird
- download all
- many (20-50) messages on each of the 7-8 accounts, with a total size about 20Mb
- the disk begins flickering desperately
- PARTIAL PROBLEM (which happen somewitmes): an error pops up saying something like "cannot process messages, folder is busy..."



Actual Results:  
PROBLEM:
- much disk space is lost (200-300Mb)
- BIG PROBLEM: afterwards _all_ the _new_ messages on some of the accounts are completely useless: they don't show any body, some of them will show completly garbled text, some will show random text from different messages (including spam which has been automatically discarded)
- note: message subjects are good in the summary
- SMALL PROBLEM: filters on the problematic account become completly disabled and afterward message classification must be done by hand
- 'damaged' message folders become unable to correctly receive messages from any possible account


Expected Results:  
messages should arrive correctly to the folders, otherwise thnunderbird should signal problems but should _NOT_ under any condition remove messages form the server unless it's absolutely sure that they have been received correctly

THINGS DISCOVERED:
- sometime it's possible to re-download all the messages, more rarely messages are lost forever
- most of the times removing all the foldes indexes and compacting the folders will 'sanitize' the folders, allowing to receive further messages: wrongly received messages will not be recovered.
Summary: inboxes, filters and messages get corrupted and sometimes completly lost → inboxes, filters and messages get corrupted and sometimes completly lost when disk space is low
Version: unspecified → 1.5
more testing showed that the following might be part of the problem:
- there is a filter which explicitly sends messages directly to the junk mail folder (I wonder if this might interfere with the spam filters being run on other accounts)
- message download period varies from 1 to 10 minutes on different accounts
- the problem usually happens when I explicitly ask to download all the new messages with the upper left button in the toolbar (might this interfere with downloads ongoing in the background?)

other symptoms which might be helpful in tracing down the problem:
- a message saying "the folder is begin accessed, try later"
- a message daying "...try to remove INFOX.msf ..."

p.s.: if needed (and someone explains me how to do it...) I might send my filters/inboxes configuration
Have you set up any accounts to use the same local directory, by changing the local directory setting (as opposed to using the global inbox)? Are you using virus checker quaranting (tools | options | privacy | anti-virus)?

Attaching (or e-mailing me) your prefs.js from your user profile directory might be helpful
(In reply to comment #2)
> Have you set up any accounts to use the same local directory, by changing the
> local directory setting (as opposed to using the global inbox)?
No, I did not, I configured all the accounts but one to use the global inbox. The only exception being the send mail folder, for which I manually selected to use always the global one (also for the 'external' account)
> Are you using
> virus checker quaranting (tools | options | privacy | anti-virus)?
No, I'm not: I'll turn it on to see if this improves the situation
 
> Attaching (or e-mailing me) your prefs.js from your user profile directory
> might be helpful
sent


After upgrade all emails were lost
I'll confirm, there's enough similar bugs that all seem to agree that there are data loss / truncation / corruption issues possible with an out-of-disk condition in Thunderbird and other products. Not sure if this is an exact dupe of any of them, but it shares elements with bug 174818, bug 318959, bug 102030 and others. 
Status: UNCONFIRMED → NEW
Ever confirmed: true
Whiteboard: dupeme
Assignee: mscott → nobody
Whiteboard: dupeme
Version: 1.5 → Trunk
Blocks: 195077
Blocks: 699963
You need to log in before you can comment on or make changes to this bug.