Open Bug 1545725 Opened 2 years ago Updated 7 months ago

loss of emails when disk is full

Categories

(MailNews Core :: Database, defect)

x86
Windows 10
defect
Not set
critical

Tracking

(Not tracked)

People

(Reporter: buecher, Unassigned)

References

Details

(Keywords: dataloss)

User Agent: Mozilla/5.0 (Windows NT 10.0; rv:66.0) Gecko/20100101 Firefox/66.0

Steps to reproduce:

total loss of inbox by the following events on Win 10. Happened three times in 2018/2019, so it is reproducible:

Windows downloads something (?? updates??) and brings C:drive to exactly 0 bytes. (This is new on Win 10, before, it always left a few MB).

Actual results:

inbox.msf is gone. Inbox no longer displayed. Emails are on imap server, but without tags (not stored on this specific imap server). My addon to backup tags no longer works since some time due to TB code changes, so I have no recent backup of tags.

Inbox file can be loaded as local folder. But all attempts to put it into the imapmail folder result in TB deleting it on next start. See one of the recent posts of others in TB forum which reports the same, it is impossible to get inbox file back for imap account.

Expected results:

This is orignated by Windows, but if it happens to others, their anger will go against TB. TB should not write anything if harddisk space = 0.

Actually, I noticed C being 0, emptied some space, but apparently TB was faster with some autosave.

My special worry is that it propagates to the server by TB reporting inbox is empty and server deleting all emails and then propagating to all other TB's logged in to that account.

That recently happened to me on another subfolder where TB suddenly reported in status line that it is deleting the folder and it is now gone everywhere (including server). I wasn't fast enough to pull network cable on imap server and other PC's. But that might be another bug - I did not press any delete.

OS: Unspecified → Windows 10
Hardware: Unspecified → x86

(In reply to klaus from comment #0)

User Agent: Mozilla/5.0 (Windows NT 10.0; rv:66.0) Gecko/20100101 Firefox/66.0

Steps to reproduce:

total loss of inbox by the following events on Win 10. Happened three times in 2018/2019, so it is reproducible:

Windows downloads something (?? updates??) and brings C:drive to exactly 0 bytes. (This is new on Win 10, before, it always left a few MB).

Why would you contniue operating under this situation? (I guess that's a different issue)

Actual results:

inbox.msf is gone. Inbox no longer displayed. Emails are on imap server, but without tags (not stored on this specific imap server). My addon to backup tags no longer works since some time due to TB code changes, so I have no recent backup of tags.

Inbox file can be loaded as local folder. But all attempts to put it into the imapmail folder result in TB deleting it on next start. See one of the recent posts of others in TB forum which reports the same, it is impossible to get inbox file back for imap account.

Several bugs in the query https://mzl.la/2UoEjtK - bug 1169252 is a good match

Severity: normal → critical
Status: UNCONFIRMED → RESOLVED
Closed: 2 years ago
Component: Untriaged → Backend
Keywords: dataloss
Product: Thunderbird → MailNews Core
Resolution: --- → DUPLICATE
Summary: loss of emails by disk overflow → loss of emails when disk is full
Duplicate of bug: 1169252

This is not bug 1169252, which is about some simpler stuff in nsMsgMailNewsUrl.cpp.

Comment 0 specifically mentioned databases (albeit only sql) which should be handled differently. I have fixed that description to also mention mork (msf), which also needs a completely different fix (it can't be fixed with MsgNewSafeBufferedFileOutputStream).

In case of .msf and .sqlite files being transactional, we probably just need to stop appending further transactions to the file if disk is full, losing new updates, but must not lose the whole file, as described here.

For example for POP3 downloading into Inbox (not the msf, but the mbox file), we always keep several MB of free space and refuse to download new message until space is freed.

Status: RESOLVED → REOPENED
Component: Backend → Database
Ever confirmed: true
Resolution: DUPLICATE → ---

You've lost me about comment 0 mentioning sql, and any change to the description (or summary).

But indeed we shouldn't lose the whole file, and have that somehow reflected back to the server. AND something should give the user simple, non-technical feedback.

I meant bug 1169252 comment 0 :)

We should somehow stop syncing/downloading new messages from IMAP if disk is almost full. We do it for POP3, so should do for IMAP and other server types too, but of course it is more complicated.

this is more serious for Win 10 than before at Win 7.
Win 10 downloads updates or reorganises files to make C drive exactly 0 bytes. Win 7 always left a few MB. After restart, Win 10 is back to 2GB, but that does not help, files are corrupted already.

If a warning is given to the user at full disk, he/she can free up space. For example, delete c:\windows\temp and c:\windows\logs, maybe even c:\users\<userid>\appdata\local\temp or delete something in downloads that can be recovered. Usually, some 10-200MB are sufficient to rescue TB if the msf has not been written already.

See Also: → 1169252

this is also related to the fact that TB cannot move large numbers of emails inside imap.

Even if I have a backup, I cannot get it back into the server. If I put backup into the imap mailfolder, it is deleted by TB.
If I put backup into local folder, I have to copy/move mails to imap, but that fails if large number of emails (>9000) - there is another bug for that.

So for the end user, both bugs are related. If this here happens and msf is deleted, even having a backup cannot help due to the other bug.

this is the bug on failing to copy large numbers of emails bug #538375.

Status: REOPENED → NEW

as of today, the same happened while C at 3GB, and on another PC with C at 170 MB. See bug 1575661. So the interpretation of this bug: 'when disk is full' does not seem to hit the reason. So probably, this bug should be closed due to wrong interpretation of events, and the other left open.

another total loss of emails: TB inresponsive after low RAM (FF took 3 GB), stays inresponsive after FF killed and RAM again available. TB killed by taskmanager. Restart of Win.

After restart of TB, inbox subfolders change their names from xxx to INBOX.xxx etc. Top inbox is empty, no inbox file, only the msf.

After another restart of TB, all inbox and subfolders are gone. In file system. INBOX-3.sbd still holds all subfolders, but they are not displayed.
inbox3.msf exists (10 MB), also inbox4.msf, which is empty.

Renaming inbox3.msf to inbox4.msf brings back the emails in subfolder, but no inbox emails on top level. Folders are again named INBOX3.xxx. INBOX3.yyy etc.

Win10 with 4GB of RAM. Owning two PCs, my experience is total loss of emails ca. >twice a year (counted over both PCs).

this helps:

start TB, inbox-5 is created. Load messages, this also recreates the subfolder list. Immediately set offline.

Rename old inbox-3.sbd to inbox5-sbd. Start TB, load messages. Now, subfolders are already there. It seems they need not be downloaded, meaning their tags are not lost (redownloading removes the tags, because they are local).

loosing mailbox at TB startup.

C was down to ca. 170MB. TB crashed, then I noticed space on C to be low. After restarting Win 10 (C: back to 2.7GB) and starting TB, inbox was gone and had to be re-downloded from imap

Is there a memory leak in FF? After displaying 4-5 websites, it eats all RAM out of my 2 GB and creates 2.5 GB pagefile. Even after closing FF, the pagefile still covers all C drive.

Still, TB should not crash and kill the inbox, no matter why C is low.

You need to log in before you can comment on or make changes to this bug.