Closed Bug 195499 Opened 22 years ago Closed 14 years ago

Set temp folder to provide space for compaction

Categories

(MailNews Core :: Database, enhancement)

enhancement
Not set
normal

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: jon.roland, Unassigned)

References

Details

User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.3b) Gecko/20030221 Build Identifier: Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.3b) Gecko/20030221 Using filter to archive messages to removable drive. Messages do get copied to target file, and removed from the message list of the Inbox folder, but the file size of Inbox does not get reduced, and am running out of space on the drive. Yes, I can set preferences to compact folders, but messages moved out of Inbox using a filter should be physically removed from the Inbox file to make more space available. I started with 32000 messages and an Inbox file size of about 700 MB. First used the filter to move about 16000 messages to folder in Local Files, which is on a removable drive. Next used it to move another 7000 messages to another Local Files folder, leaving 9000 messages in message list, but Inbox file now at 800 MB, only 26 MB free space left on drive, and getting messages that I am running out of space on that drive. How do I get that Inbox file size reduced so that it contains only the 9000 messages in the message list? This subject does not seem to be covered in the help files, and an entry on it should be made. Reproducible: Always Steps to Reproduce: 1. 2. 3.
Just got an error message when tried to compact Inbox: The folder 'Inbox" could not be compacted because writing to folder failed. Verify that you have enough disk space, and that you have write privileges to the file system, then try again. So can't compact, can't delete messages, and can't free up any more disk space on that drive. What are the options remaining?
After further investigation I have discovered that what is happening is that messages are not being physically removed from folder files. It doesn't matter how one tries to remove them. Moving messages to another folder, or deleting them from the Mozilla interface, removes them from the message list, but not from the physical file, which just keeps growing, and growing, and growing, until disk space is exhausted. I have found a workaround: Move all the messages in a folder to a folder on an external disk, then delete (or otherwise remove) the original folder file, together with its msf file, then move all the messages back again. Before I did this, the Inbox file on my most active account (300-400 messages a day) had grown to 820,438 KB, and I had only 20 MB remaining. After "moving" the messages to a folder in the Local Files account, which is physically located on an external removable drive (a 120 GB Western Digital 1394 HD), removing the original Inbox file, and then "moving" the messages back to the Inbox folder, its physical file size was only 222,342 KB. Fortunately, no messages appear to have been lost during these transfers (but I saved the files just in case). Needless to say, if one has to do this kind of thing on every folder of every account to free up disk space when it becomes scarce, then it is a critical bug, and I advocate upgrading it to that level. Compacting is all very well and good, but compacting doesn't work if disk space is insufficient, and what is really needed is to physically purge messages folders of removed messages.
Severity: normal → critical
One more thing that should be added. It doesn't work to try to move too many messages at once. On my system (128 MB RAM) I can't move more than about 3000 messages at a time. For those who may have similar limits, and if you want to preserve the order in which messages are received (and many of us do, especially if our senders' system clocks are not set accurately), you will need to first order the messages by order received. Then estimate how far along the scroll bar will give you about, say, 2000 messages, select them, and move them, starting with the oldest messages remaining. If disk space is limited, the transfer folder will need to be on another drive with plenty of space. Each move will take a long, long time, so be patient. Good time to take a break. After all the messages are moved, remove both the messages and msf files, before moving the messages back. They first batch you move will also trigger the creation of a new summary (msf) file. Set the view order for all folders to view by order received for convenience and to make sure the messages are in the same order when this is completed.
Summary: Filter does not reduce size of Inbox file → Move does not reduce size of folder file
I am changing the summary line to more accurately reflect what I am requesting. The basic problem is that compaction fails when there is not enough free space available on the drive where the message folders are located. The obvious solution is to allow the user to define a temp folder on another drive to provide enough space, perhaps by copying the unremoved messages to a folder there, then deleting the original folder file, and copying the messages back from the temp directory. I also propose the option of immediate removal of messages from the folder file, a kind of compaction of just the one removed message, rather than searching for and removing all of them in a single batch operation. The help function should also advise the user to set the trigger for initiating compaction to be something like 10,000 KB so the prompt to compact messages won't come too often, interrupting normal usage.
Summary: Move does not reduce size of folder file → Set temp folder to provide space for compaction
Any further action on this? It is still a priority for me, as it is a constant chore.
Product: MailNews → Core
(In reply to comment #5) > Any further action on this? It is still a priority for me, as it is a constant > chore. I can't believe this is still classed as Unconfirmed. I can confirm it is a problem and it has totally stuffed my mail. I foolishly tried to move my (large) messages around to tidy up my inbox, I then found that I was running very low on disk space. I tried to compact the messages but to no avail (as mentioned above). I then deleted a lot of the messages only to find that no disk space was freed up at all. It seems that the only file which genuinely deletes messages is the Trash file. I now have no disk space and none of my other drives are large enough to use the workaround listed above. I for one would be very grateful if this bug could be fixed soon, otherwise I will have to use another mail client, which would be a shame as I've been very impressed with everything else about Thunderbird.
This is an automated message, with ID "auto-resolve01". This bug has had no comments for a long time. Statistically, we have found that bug reports that have not been confirmed by a second user after three months are highly unlikely to be the source of a fix to the code. While your input is very important to us, our resources are limited and so we are asking for your help in focussing our efforts. If you can still reproduce this problem in the latest version of the product (see below for how to obtain a copy) or, for feature requests, if it's not present in the latest version and you still believe we should implement it, please visit the URL of this bug (given at the top of this mail) and add a comment to that effect, giving more reproduction information if you have it. If it is not a problem any longer, you need take no action. If this bug is not changed in any way in the next two weeks, it will be automatically resolved. Thank you for your help in this matter. The latest beta releases can be obtained from: Firefox: http://www.mozilla.org/projects/firefox/ Thunderbird: http://www.mozilla.org/products/thunderbird/releases/1.5beta1.html Seamonkey: http://www.mozilla.org/projects/seamonkey/
This bug should definitely be kept open and fixed. The fix should be easy, and the lack of comments doesn't mean it isn't important to people, or won't become important as the product is used in more commercial environments that get a lot of email. It is a good design practice to always offer the user a choice of setting temp folders for anything. No application should ever assume sufficient disk space on a given drive.
Is this still a problem? This bug is listed as pertaining to Windows 98, which is no longer supported by Microsoft. If this bug cannot be reproduced, feel free to close the bug as WORKSFORME.
The last post fails completely to understand the problem, which is that the user can't select a temp folder on a filesystem with enough space, having an unchangeable default to a filesystem which may not have enough space. It has nothing whatsover to do with the operating system. Same problem exists for all operating systems. What might additionally be needed would be a way to do the processing when there isn't enough space on any filesystem, by partitioning the mailboxes and doing them in phases, making as much use of available space as possible without overflowing.
<- OS to All based on comment 10.
OS: Windows 98 → All
Jon, please list a short set of steps to reproduce 1. blah 2. blah 3. ... actual results: blah expected results: blah bear in mind, mailnews is not likely to gain the smarts of a filesystem or get predictive, which may have some bearing on why this bug has not been confirmed.
(In reply to comment #4) > I am changing the summary line to more accurately reflect what I am requesting. > The basic problem is that compaction fails when there is not enough free space > available on the drive where the message folders are located. The obvious > solution is to allow the user to define a temp folder on another drive to > provide enough space, perhaps by copying the unremoved messages to a folder > there, then deleting the original folder file, and copying the messages back > from the temp directory. Changing severity to enhancement for this part of the bug. > I also propose the option of immediate removal of messages from the folder file, > a kind of compaction of just the one removed message, rather than searching for > and removing all of them in a single batch operation. > > The help function should also advise the user to set the trigger for initiating > compaction to be something like 10,000 KB so the prompt to compact messages > won't come too often, interrupting normal usage. you will want to address these other issues in separate bugs.
Assignee: naving → nobody
Severity: critical → enhancement
Component: MailNews: Filters → General
QA Contact: laurel → general
Hardware: PC → All
I have had similar problems with running out of space to compact. Having the 'auto-compact' option on has helped somewhat, but if I forget to delete a bunch of large attachments then I am completely hooped and have to go through all the shenanigans that Jon Roland has detailed. I think the temp folder idea is a good one and probably dead simple to implement. Another suggestion is this: I have noticed that the compaction routine fails as soon as it can't compact a particularly large folder as there is not sufficient space on the drive for the compaction routine to run. However, there are often other folders that are smaller that could be compacted - leaving more space to compact the large ones. I know this because I quite often manually have to do this to these smaller folders before going back to my bigger folders. My suggestion is that the compaction routine not instantly fail with the message: "The folder 'Blah' could not be compacted because writing to folder failed. Verify that you have enough disk space, and that you have write privileges to the file system, then try again." But instead keep on trying to compact other folders. It should only return this error message when it has tried to compact every single folder, but there is not sufficient space to compact any of them. And in that case, a temp folder would be required... However, for my usage, a smarter compaction algorithm would likely solve 99% of my space issues.
While probably helpful for some people, are there other advantages (performance, etc) to allowing for a temp space that is not within the user's profile or in the standard windows defined temp space? If not, this is just a workaround for a combination of a) primarily insufficient disk space resources caused by the user, and b) secondarily not great compaction smarts or improvements that could be made on the part of Thunderbird in several areas, and c) in some cases unwillingness on the part of the user to delete messages that are no longer needed, or lack of knowledge that not deleting messages has significant impact on disk space and performance compact bugs which mention space - https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc=Compact&longdesc=space&short_desc_type=allwordssubstr&type0-0-0=nowords&value0-0-0=count%20counts&resolution=---&product=MailNews%20Core&product=Thunderbird&longdesc_type=allwordssubstr (moving back to mailnews land)
Component: General → Database
Product: Core → MailNews Core
QA Contact: general → database
hmm, no response to comment 15. bienvenu, I can't see this ever being worth the code and testing cost. Plus we aren't in the business of making thunderbird run for users who can't maintain drive health. FWIW, the dead easy solutions for a user are: a) move the profile do a different drive b) free enough space space on the current drive
Whiteboard: [wontfix?]
yeah, I'd agree with wontfix. diskspace is cheap and very few users would ever be able to take advantage of this.
Status: UNCONFIRMED → RESOLVED
Closed: 14 years ago
Resolution: --- → WONTFIX
Whiteboard: [wontfix?]
I'd have to disagree with your assumption(s). I've been using Thunderbird for many versions from the begining after I switched from Netscape > Mozilla > Firefox + Thunderbird and I seldom had any problems that couldn't be rectified. However, I've been using Thunderbird for U3 for a few years now and I have been having the same problems as described above. As one might guess my thumbdrives are mainly for use outside the office and have files on it that are needed in those situations and I find it inconvenient to have to swap off a bunch of files to get my e-mail client to function properly. My SanDisk Gator 8GB [7.47GB] shows me how much free space I have on it. I have had between 650MB and 1GB+ of free space and run into the dreaded message that it cannot compact folder because there is either not enough free space or I don't have administrator privileges. I do compact my folders but about a 1/3 of the time I run into this issue. And it usually seems to fail when ~80-90% finished if the progress bar is to be believed to be accurate. Why not compact the folders individually? Or default to finishing compacting the folders it has found sufficient space for and then continuing on with the remaining when more space has been opened up? How about splitting that error box into two separate errors and telling the end user when they fail to complete compacting it was because a folder of X size was encountered, please free up at least X amount of room and retry. At least then the user has some idea of what to look to to resolve the issue. For most of the suggestions listed above I can't even access the folders to copy the files out to another location. I also can't export the profile which I could then conceivably import it on another machine such as the laptop or desktop with plenty of room to allow compaction to complete and then re-export and re-import it back to my U3 drive. I've run into this issue on my Windows XP Pro SP3, Vista Home Premium and Windows 7 Home Premium OSs on both laptops and desktops.
You need to log in before you can comment on or make changes to this bug.