337554 - automatic compact folders to save space interferes with get new messages

Status: RESOLVED DUPLICATE of bug 101584

(Thunderbird :: General, defect)

Description

[Andrei Boros]

User-Agent:       Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3
Build Identifier: Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3

mbox format still uses up disk space after deleting messages. Hence the "compact folders" necessity exists. 

By default, TB comes with "automatically compact folders if it saves XX kb disk space" UNCHECKED, and XX=100kb.

Reproducible: Sometimes

Steps to Reproduce:
1.have a reasonably large inbox, deleted messages, and not performing compact folders in a while
2.set "compact folders if it saves xx disk space
3.next time you start TB, you are asked if "proceed with compact folders". say YES
4.at the same time, TB should be doing a get new messages at startup

Actual Results:  
sometimes, not always, i get an error, mailbox folder in use. 
TB seems to be trying to compact a folder (usually Inbox) and at the same time to write a new message that it downloads from the server. 
Result : write lock problems. 

since TB first creates a temporary file where it writes the compacted version of the folder, then deletes the old folder and rename the temporary file to the folder's name, this happens only sometimes, not always, as most likely, it happens when it tries to delete the old folder and rename the tempfile to the folder name while downloading a message.

Expected Results:  
There are several aspects to the expected results with this. 

1. "automatically compact folders if saving XXkb space" should be enabled by default at install time
many, many users are not aware of this drawback of the mbox format and i've seen people with inbox file being hundreds of megs, while their inbox actually holds only a few messages. 

2. disk space limit should be set to a more reasonable value for today, smth like 10-20MB, the current default being 100k, often barely means a few messages

3. if this setting is ENABLED, TB should NOT ask confirmation everytime it starts this operation. another bug covers this aspect, but yet unresolved

4. it is debatable if this operation should happen at startup or shutdown of TB app. startup is fine with me.

5. if done at startup, TB should delay the "get messages at startup" or any user requeted "get messages" until the compacting operation has ended. 

6. also, progress indicator and window status aren't always consistent with the actual operation in progress. while compact folders in progress, regardless if manual or automatically requested, at startup or at any other time, a proper window status and progress should be displayed.

7. not directly related, covered in another bug as a reply, but i'll say it again here : 
"empty trash" should also perform compact folders automatically (NS4 behaviour). 
This command is obviously a manual cleanup request.

Comment 1

[Magnus Melin [:mkmelin]]

This seems like a duplicate of one or many bugs. See e.g. bug 205756, bug 152675, (suite) bug 188728. Also bug 324576.

Comment 2

[Andrei Boros]

For sure. I knew some other bugs, related to some of the suggestions i made. 

#152675 seems close and covers the Core (all mozilla products), but discussion in it seems to have veered away from the issue of simultaneous operations and file locking in mail client to POP issues and malformed messages.

#188728 also close and more to the point. i could be a dupe of this. but would need to extend it's scope to cover TB as well as Moz suite.

Comment 3

[John David Galt]

Item 3 is a duplicate of bug 198936.

Comment 4

[Aaron Whitehouse]

This happens to me as well. I find it very frustrating as it means that I have to manually "Get all Mail" once the compacting has finished.

Comment 5

[Michal Siegel]

Yeah, I've found it slightly annoying.  I think I've finally become conditioned to click on "No" and then just try remember to select the compact-folders command after mail retrieval has completed.

(In reply to comment #4)
> This happens to me as well. I find it very frustrating as it means that I have
> to manually "Get all Mail" once the compacting has finished.
> 

Comment 6

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

Reporter, does the issue still occur in the latest supported 2.0.0.x / Shredder trunk nightlies?

(1.5.0.x is now end-of-life and the latest supported 2.0.0.x is 2.0.0.16)

Comment 7

[[:Aleksej]]

Happens with version 2.0.0.16 (20080707) Linux i686.

Comment 8

[Andrei Boros]

I can also confirm it. Happens with 2.0.0.16

Comment 9

[Andrei Boros]

Happens with 2.0.0.16 and 17, both Win32 (all) and Linux.

Comment 10

[John David Galt]

In TB 3.0.1 this bug not only still exists, it is more serious because auto-compacting happens more often.  And it doesn't just interfere with downloading new messages, it also makes TB *very* slow to display a message after you select it (while downloading and compaction are taking place), even on a fast computer.

Comment 11

[WADA:World Anti-bad-Duping Agency]

> Steps to Reproduce:
> (snip)
> 2.set "compact folders if it saves xx disk space
> 3.next time you start TB, you are asked if "proceed with compact folders". say
YES
> Actual Results:  
> sometimes, not always, i get an error, mailbox folder in use. 
> TB seems to be trying to compact a folder (usually Inbox)
> and at the same time to write a new message that it downloads from the server. 

If you revert "Do this automatically" for dialog at step 3, you can see dialog just before start auto-compact always again, and you can select "Cancel of auto-comact" and "Execution of auto-compact".
See meta bug 498274 comment #2, and change back to mail.purge.ask=true(restart is required), and change from "seems" to "your problem occurs always if auto-compact runs" and "never if auto-compact is canceled", please.

> Result : write lock problems. 

What phenomenon do you mean by "write lock problems"? What kind of message is displayed?

AFAIK, although delete from Inbox fails(copy to Trash, other folder is OK, but remove from MsgDB of Inbox fails), adding mail to Inbox, status change(such as tag add), etc. is possible even while compaction is running on Inbox. See bugs listed in Dependency tree for bug 498274, please.
If "outdated .msf condition" exists on Inbox.msf, rebuild-index is usually invoked upon compact folder, because compact folder explicitly opens mail folder. If internal rebuild-index happens, access to Inbox is locked out with message like "another process is running". Internal rebuild-index case?

Comment 12

[[:Aureliano Buendía]]

Could be a dupe of bug #101584 ?