797215 - When a folder exceeds the available disk space and you close thunderbird, the entire folder is erased without a warning message.

Status: RESOLVED INVALID

(MailNews Core :: Backend, defect)

Description

[Terry Wiles]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0
Build ID: 20120420145725

Steps to reproduce:

Get mail, moved some mail to a local folder (Inbox 2012), then closed thunderbird.
There were no warning messages.


Actual results:

Upon reopening thunderbird my entire local folder (Inbox 2012) was gone. The file had been deleted.


Expected results:

Some warning about the fact that there was insufficient disk space to save the folder (Inbox 2012) and provide an alternative option, i.e. delete some mail, save somewhere else...

Comment 1

[:aceman]

What is your TB version?

Comment 2

[Terry Wiles]

I always keep TB up to date.
This problem occurred on 17 April, then again sometime in August (I can't remember exactly when), so whatever was the current version at those times.

Comment 3

[WADA:World Anti-bad-Duping Agency]

(i) If (a) auto-compact is enabled, and if (b) you request Tb to do it silently, Compact of folders can occur after "delete mail" operation automatically with no confirmation.
  (a) mail.prompt_purge_threshhold = true
      mail.purge_threshhold_mb = NN (in MB)
  (b) mail.purge.ask=true is not set
(ii) As for mail store file(if "Inbox 2010" folder, "...\Inbox 2010" instead of "...\Inbox 2010.msf"), mail data is simply appended to the file by your archive operation(==move from Inbox to "Inbox 2010"). Tb never deletes or renames file of "...\Inbox 2010" by "mail move operation".
(iii) Because archive = "mail move" == "copy mail from Inbox to Inbox 2010" + "delete mail from Inbox", if both (a) & (b) is True in your environment, Compact on "Inbox 2010" can be invoked silently.
(iv) As seen in bug 674742, current Compact logic has hole of "delete mail store file" in special situation.

Setting dependency to bug 674742.

Comment 4

[Andrew]

I do not understand why WADA mentions "...\Inbox 2010", as opposed to "Inbox 2010".

I agree that it may depend on bug 674742. Though does not seem as a complete duplicate.
It seems that there's one more bug, which triggers the bug 674742.

The folder "Inbox 2012" may be gone not because you archived into it.
But because you closed Mozilla, and compaction started on closure. Or perhaps a time for automatic compaction came.

If disk was full, compaction might have stopped "half way", but failed to report the error.
This would mean that some other bug may exist.
Then bug 674742 destroys original file. Then it renames the compact copy, which may contain very few files, or none at all (if disk is very full).

These are only my presumptions. You need to investigate the code in the entire sequence of calls. maybe you find this one more bug. You find the starting point in bug 674742.

Terry, did you really check the file on disk? Or do you mean only the folder in window of Mozilla?
Is your disk local, or remote over a network?

I agree with Wayne that this bug looks very similar to bug 398151. Perhaps a duplicate.
Although bug 398151 seems to report that some messages may not be gone. While your data is gone completely.
On the other hand, that would be consistent with my theory that compaction can stop "half way". This would clearly indicate a new bug.

If not for the bug 674742, which destroys the original mail afterwards, this new bug would be not critical.

Now, do I look at code this time too?
Or is anyone of you going to check it?

Comment 5

[Terry Wiles]

I do not use automatic compaction - I do this manually.
I did not notice if the file was gone before I closed TB.
I did not notice if the file was gone after compaction.
But it was definately gone when I reopened the next day.
Yes I actually looked for the mail file on the disk and it was gone.
This is a local disk.

Comment 6

[Andrew]

Maybe automatic compaction is still not disabled.

I have seen some preferences in Mozilla, which reemerge as set, although you unset them. One such is "Send crash reports".
An "JS error console" (in tools menu) displays an error, if you try to change such particular setting.

You may try enabling, and disabling compaction, and watch if this produces an error in the list of that console.
Strange it is that such errors do not display to the user. Instead they hide in the list in the system.

Please check the "about:config" page for the actual settings, which WADA suggested.
Or better check all setting filtered by "purge".
In TB you find that page through a button among advanced options.

Compaction operates only on folders, which have space to save. If you never deleted a mail from the folder, compaction would not operate on it, and would not destroy it.
Did you ever delete messages in the missing folder?

If compaction is not to blame, this would mean that an twin of bug 674742 exists in archival as well.
Very likely, if the same person made both.

> I did not notice if the file was gone before I closed TB.

Do you mean that you were able to work with mail, which you put in archive right until you closed TB?
Perhaps you never looked at the archive after you archived messages, and before you closed TB.

Comment 7

[Terry Wiles]

Answers to your questions:
1) Did you ever delete messages in the missing folder?
 - Very rarely.
2) Do you mean that you were able to work with mail, which you put in archive right until you closed TB?
 - Generally I move message from my Inbox to the archive, compact then close TB.
3) Perhaps you never looked at the archive after you archived messages, and before you closed TB.
 - Correct, I would not look in the archive prior to closing, nor delete any messages from it at this time.

P.S. I tried to replicate the problem by progressively filling my drive to capacity, I was unable to get TB to make any mistakes. I can only comment that previously, both times I had this problem the disk partition used for TB was at capacity.

Comment 8

[Andrew]

Which folder is missing? Inbox? Or archive?
First I thought - Inbox.
Then WADA led me to believe it is archive.
Now I realise that it is Inbox.

You received messages from a mail server.
You read them. They were not missing.
Then you moved some to archive by hand. It was not automatic archival.
Restarted.
Archive is there.
But Inbox with new messages is missing.
Correct?

Then it could be that compaction started automatically, because you moved mail (Inbox to archive). You moved more than a threshold for compaction in your preferences.

You mentioned that you do not use automatic compaction.
Is it turned off?

When you try to replicate the data loss, fill the disk so, that it has less space than there are messages, which are not deleted in a folder. Then compact it by hand.
Compaction works by copying not deleted messages into a new temporary file, named "nstmp". you may read more about this operation in attachment, which I mention below.

It is not sufficient to simply fill the disk, and restart.
Compaction must start.
One more condition: at least one message must be deleted in the folder.
After compaction you will immediately see that messages are gone. No need to restart.
The list of messages of the folder will become empty. The folder will remain in the tree. But you won't be able to use the particular folder. Because the file for it would be gone.

To recover the missing file, look around your system.
A copy may sit somewhere.
Find tips on recovery through bug 674742. There see my attachment, named "Description ...".

Comment 9

[Terry Wiles]

Hi Wayne, I now believe that this is not a bug at all.
What was happening was my virus software "Total Defense" was quarantining my entire folder ever time it found something suspicious in an email - hence the folder disappearing.
After I removed Total Defense the problem has never reoccurred.
My apologies for the effort you went to on this matter.

Comment 10

[Rainer Bielefeld]

Due to Comment 9 I close this one for now.
@Reporter: Please feel free to reopen this Bug if you still can reproduce the problem with a current Thunderbird  version and a current OS.

Comment 11

[Wayne Mery (:wsmwk)]

We use INVALID when the cause of the report is clearly not caused by Thunderbird :)