542708 - entire inbox deletes when deleting single email in pop3 account. inbox and inbox.msf files are deleted too. low value (50k) for "compact folders when it will save over" causes folder compact on every message delete

Status: RESOLVED INCOMPLETE

(Thunderbird :: Folder and Message Lists, defect)

Description

[news]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 (.NET CLR 3.5.30729)
Build Identifier: tbird 3.0.1 Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20100111 Thunderbird/3.0.1

Has happened dozens of times in latest build. In pop3 account periodically if I delete one email the entire mailbox disappears, the top tab says "drafts" instead of "inbox", and the bottom left of t-bird says "compacting". The actual inbox will still be bolded with the number of unread messages listed and the hourglass icon shows as if it's waiting for something.

In windows explorer when I go to the mail's folder both inbox and inbox.msf are missing.

This has only started happening with the t-bird 3. In has not yet happened with an imap account, only pop3.

Reproducible: Sometimes

Steps to Reproduce:
1. delete an email via keyboard delete key or t-bird delete button
2. happens almost every time if tbird has just been opened
3. seems to possibly coincide with message at bottom of tbird about "deciding which messages to index"
Actual Results:  
all messages in inbox are gone, inbox and inbox.msf files are gone too

Expected Results:  
just deleted the one email I selected

After this happens the top tab says "drafts" instead of "inbox", and the bottom left of t-bird says "compacting". The actual inbox will still be bolded with the number of unread messages listed and the hourglass icon shows as if it's waiting for something.

Seems related to compacting or messaging indexing. Happens every time if I delete an email soon after opening tbird (possibly as it's connecting to the server.) Also deletes all mail from server when this happens. Happens about 1/4 of the time when deleting emails at other times.

Comment 1

[WADA:World Anti-bad-Duping Agency]

Tb never deletes file of Inbox by delete of mail in local mail folder. "delete of a mail" simply adds flag of "mail is deleted" in X-Mozilla-Status: header line in local file named Inbox for local mail folder named Inbox.
"Delete of file named Inbox" by Tb happens upon compact folder only.
  1. copy non-deleted mail data in Inbox to file of nstmp(-N).
  2. (2-a) Delete Inbox, (2-b) rename nstmp(-N) to Inbox
  3. update(or replace) Inbox.msf.
Power failure between (2-a) and (2-b) can produce "lost Inbox file", but possibility of it is very very low, althogh it's never ZERO.
And, if it happens, mail folder named nstmp(-N) appears after restrt of Tb, when POP3 accout or "Local olders". 

Culprit of "Delete of file named Inbox" is file scanner of anti-virus software in many many cases. Do you set up anti-virus software apprpriately? Is there any log of anti-virus software?

Comment 2

[Wayne Mery (:wsmwk)]

news, https://wiki.mozilla.org/Thunderbird:Testing:Antivirus_Related_Performance_Issues may help you with the questions in comment 1

Comment 3

[news]

I am pretty sure it's nothing to do with the AV. There's nothing in the AV log about it and the way it happens seems 100% like a tb issue.

I think I have found the reason though...it's the folder compacting. What is happening is that when I delete an email it will immediately/concurrently compact the folder which apparently corrupts/deletes the inbox. (Again, inbox and inbox.msf are deleted when this happens).

This may be happening so much to me because I had the "compact folders when it will save over...x" setting at 50kb. So in that case it was auto-compacting every email or two. This repeated compacting was the problem I think. I've upped the setting to 500kb and haven't had the problem so far but it could be just luck.

This same issue will potentially happen anytime anyone deletes an email. It's just that I was seeing it so often because I had the setting low. I saw another bug report about folders compacting immediately causing other issues too.

Seems to me that a slight delay after deleting an email and before starting compacting would be a good and simple solution.

Comment 4

[WADA:World Anti-bad-Duping Agency]

You look to be able to reproduce problem like bug 493065, bug 492254 easliy and repeatedly... See bug 498814 comment #4 for possibilty of Inbox file size=ZERO.
See also bugs listed in dependency tree for meta bug 498274.

Read bug 498274 comment #2, and check difference between "CANCEL or No to dialog" case and "OK or Yes to dialog" case, please.

Comment 5

[WADA:World Anti-bad-Duping Agency]

If possible, do next, please.
 - change back to "compact folders when it will save over..." = 50kb.
 - disable dialog before auto-compact.
 - remove Tb's mail directory(or profile directory) from file scan target
   directory of anti-virus software.
   (file-scan for mail data file of Tb is needless,)
   (if mail scanner for inbound/outbound mail data is enabled.)
 - if external program such as auto-backup software is used for Tb's mail
   directory, disable it.
 - Use Tb daily as usual.
Does you problem still occur so frequently?

Comment 6

[Wayne Mery (:wsmwk)]

news@,
Did you see this problem with version 2?

Comment 7

[news]

Seems these are likely suspects bug 493065, bug 492254, although I wasn't doing anything additional to the mail while compaction was occurring. However, since the auto compact setting was so low (50k) it was compacting after deleting 1-3 emails. Deleting an email that instantly starts the auto compaction could possibly be like doing something during compaction as in the 2 bugs above.

Since raising the auto compaction setting to 500kb a few weeks ago I haven't had the problem. It seems to me the issue is that auto compacting starts instantly. The email probably hasn't completed deleting before the compacting starts which destroys the inbox.

Comment 8

[Wayne Mery (:wsmwk)]

(I hate to repeat simple questions but ...)

news,
Did you see this problem with version 2?
Can you *Please* do comment 5?  (this includes disabling AV)

Your engagement is great. But the answers to these questions are essential to make progress. Hypothesizing about other bugs, especially when they are UNCOnfirmed without a diagnosed cause, without sufficient facts to solidly match them up isn't terribly useful.

Comment 9

[news]

Wayne,
RE: version 2
Assuming you mean tbird 2?
I had the same issue with version 2. It seems like it started to happen after an update in early dec 2009.

I did disable AV and had the same issues.

I cannot do the rest of comment 5. I cannot afford to risk losing more business emails and I honestly do not have the time to take away from work to continue debugging this. The problem is not happening anymore and that's good enough for me.

Obviously I'm not a coder or debugger. I'm just a guy pointing out an issue I was having repeatedly. If I find anything else useful to add to this I will, but for now I've got to get back to work.

Best regards

Comment 10

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #7)
> It seems to me the issue is that auto compacting starts instantly.

Why still "SEEMS"? You still disable dialog just before start of auto-compact? You haven't read bug 498274 comment #2 which I pointed in comment #4?

Comment 11

[[e-mail]]

i have experienced this identical issue with tb 3.1.6, on two separate machines (my work laptop and home PC). open tb, download messages (pop3), select a message, click 'delete', and all contents of inbox disappear - and the Inbox and Inbox.msf are no longer on disk. a very painful bug. never had this occur with any previous version. i am running pc tool spyware doctor with AV 2011 - but i don't know if it correlates to this. having the actual inbox files disappear from disk is disheartening to say the least. my OS and apps are on SSD, so i disabled system/file restore based on recommendations to do so. guess that's not an option any more.

Comment 12

[Wayne Mery (:wsmwk)]

reporter isn't in a position for us to determine what bug this might be duplicate to, so => incomplete (which is not to say there is not a problem - but we do not have enough information)

paul, your loss may be a result of AV/spyware software. see comment 2. If you continue to have a problem please check http://www.mozillamessaging.com/support and post a message in the support forum if needed.

Comment 13

[[e-mail]]

(In reply to comment #12)

at this point, since i've remediated the problem, i'm reasonably confident i know what the problem is: too low a threshold on folder compaction. since turning off automatic folder compaction, this has not recurred. unfortunately, this is only correlation, so i can't say definitively - but it's been a month and half since i turned off folder compaction on both machines, and not a single recurrence. it appears to be an edge case where deleting a message triggers the compaction somehow, and the two manage to clobber the entire inbox. 

but absent proof i guess it will have to remain incomplete.

Comment 14

[Wayne Mery (:wsmwk)]

then bug 498814 is a good issue to be aware of.

note, compact value in the 1-20MB range is far better than the default. and you can turn on "prompt" so it doesn't go off in the background on you.