Open Bug 286888 Opened 20 years ago Updated 3 months ago

Always make compacting folders automatic, with no UI

Categories

(MailNews Core :: Backend, enhancement)

Product:
MailNews Core
Component:
Backend
Type:
enhancement

Tracking

(Not tracked)

People

(Reporter: mpt, Unassigned)

References

(Depends on 1 open bug)

Details

(Keywords: privacy)

This is the Thunderbird equivalent of bug 229034, in the hope that Thunderbird's developers are "removing ... chrome we don't need" and aiming for "simplified preferences UI and menus" <http://www.mozilla.org/projects/thunderbird/>. Compacting folders is an arcane operation that I shouldn't have to worry about. It should happen automatically, in the background, whenever Thunderbird is not busy doing anything else, and it shouldn't need any menu items or checkboxes. Doing this would fix, or render obsolete, bug 205756, bug 223525, bug 236922, bug 245607, bug 255687, bug 260004, bug 260252, bug 262485 (probably), bug 265431, bug 265472, and bug 271988, and it would subsume bug 61960.
(In reply to comment #0) > This is the Thunderbird equivalent of bug 229034, ...which met much opposition and was eventually wontfixed. mpt, perhaps I'm missing something, but would you want the Trash (or Recycle Bin) to auto-empty without user intervention? Prog.
(In reply to comment #1) > ...which met much opposition and was eventually wontfixed. four vocal users isn't exactly much opposition.
(In reply to comment #2) > four vocal users isn't exactly much opposition. It is much opposition when only four people commented on the bug. Prog.
let's not go into vote counting. You need a decent sample of users to do that. bugzilla doesn't allow that. And never did. the question is: does fixing this bug help anything? And it does, given the list of bugs mpt gave that would be fixed by this. So i say to go for this.
Bottom line: Besides the name of the mail client, I don't see any change between the reasoning that lead to wontfixing bug 229034 and doing the same with this one. Prog.
1. That bug wasn't wontfixed by somebody with the power to do so, but by a random other person 2. That bug ignored the long list of bugs that would be fixed by making compacting automatic 3. That bug doesn't have real reasons for wontfix, just "i don't want that" and "i might want to compact manually"
I said "Thunderbird's developers", not "the sort of people who still use the Mozilla suite". If you don't like "simplified preferences UI and menus", please use the Mozilla suite (or Opera), and stop soiling Thunderbird bug reports. (And yes, I do have Apple Mail set to auto-delete messages in the Trash more than 7 days old. A Trash folder is to hold stuff I *probably* don't want until I almost *certainly* don't want it, well-indicated by its age. But that's a separate RFE.)
There's one very good reason why this bug should be wontfixed: Users who access their mail both through IMAP and through a web-interface need this feature. Otherwise, a mail that was marked for deletion in TB would stay crosslined in the web-interface until the 7 days period has passed. If you make this interval significantly shorter, users are bound to lose mail they might not have wanted to delete. I can see the benefit for those who only use TB, but to assume that all TB users don't also use web-interface is quite unfounded. Many users choose to use a mail client from their home and a web-interface in public places. For them, having no Purge/Expunge/Compact folders options would be counterproductive. Prog.
you are confusing cleaning up the trash and compacting folders. cleaning the trash should happen after a few days, and isn't this bug. compacting should happen much more often, like every hour or more. So webmail users won't see crossed out items. the mails are just deleted.
It's not clear to me this feature is IMAP-related at all; IMAP already has a feature for "Clean up (expunge) Inbox on Exit." The feature I wish to see is automated compaction of folders in POP accounts and Local Folders, to regain disk space, without having to do it manually. (There is also a desirable compaction function for news accounts, but it doesn't appear to be implemented.) The primary reason not to make this an automatic, background task is because compacting a folder can interfere with storing newly arriving mail for that same folder -- bug 274330, bug 139215 (same basic problem as bug 275132, bug 168648). The program's backend needs to be able to queue these actions internally, deferring a Get New Mail if a compact is in process and vice versa. This problem occurs even now, with user-initiated compaction, but would be more widespread if automatically occurring behind the scenes.
My IMAP account often overflows because of non-compaction, so no, it isn't just POP/local. My Junk and Trash folders need compacting, so no, it doesn't apply just to my Inbox. And "Clean up (expunge) Inbox on exit" is a pref, so no, it doesn't satisfy this bug. And yes, I know compaction interferes with other tasks; that's why I wrote "whenever Thunderbird is not busy doing anything else".
*** Bug 287037 has been marked as a duplicate of this bug. ***
We must do this for privacy reasons. People freak out when they find their old deleted mail still in their mboxes; no one has a clue about the unobtrusive Compact Folders menu item. Desktop OS's have taught (most? some?) people about the concept of emptying the trash/recycle bin, so bug 61960 would do in a pinch.
Flags: blocking-aviary1.1?
Keywords: privacy
I HAVE IMAP folders + *MARK AS DELETED* and i have auto-move-junk (in essence also *mark as deleted*) in order to give me a second chance to undelete erroneously deleted or marked-as-junk messages. So i'm hoping this "fix" doesn't compact my IMAP folders before i've had a chance to verify all my deletions. What makes this bug even worse for me is that marked-as-deleted messages often *disappear* from view until a "Get Messages" triggers their re-appearance. Simplifying UI is fine, but breaking useful behavior is not worth it. I think auto-compacting of Local and POP folders is an excellent idea. I would suggest that the usefulness of "mark-as-deleted" should prevent this bug from affecting IMAP, until a better solution taking this into account is found.
> a second chance to undelete erroneously deleted or marked-as-junk messages That's why the Trash was invented. (Having "Delete" mean "don't delete, but mark as 'deleted'" is utter crack, but that's a separate bug.) If you like, file a separate RFE for a command to restore Trashed messages to their previous folder.
(In reply to comment #15) > > a second chance to undelete erroneously deleted or marked-as-junk messages > That's why the Trash was invented. The trash is much less useful for this scenario because: 1. *Greater distance traveled to retrieve* - To save vertical space, it would make little sense to have a trash folder in each account, so having one trash folder in Local folders makes sense. If the user now accidentally deletes something (or the junk filter moves something) he/she has to move (far) away from where he/she is (from IMAP/inbox to LocalFolders/Trash), and then come all the way back, and re-orient him-/herself. 2. It's *Harder to find* the erroneously deleted message among all the other many messages in trash that are also there from all the other acounts. > (Having "Delete" mean "don't delete, but mark > as 'deleted'" is utter crack, but that's a separate bug.) That's just like saying: Having "Delete" mean "don't delete, but move to trash" is utter crack. Both have their useful purpose - one for POP & Local Folders, the other for IMAP + mark-as-deleted. > If you like, file a > separate RFE for a command to restore Trashed messages to their previous folder. No, this bug should be WONTFIXED until you find a solution that doesn't damage the described usefulness of mark-as-deleted.
(In reply to comment #16) > this bug should be WONTFIXED until you find a solution that doesn't damage > the described usefulness of mark-as-deleted. ... or [this bug] limited to POP and Local Folders (but not IMAP + m-a-d).
For a user, there is no difference between imap and pop. Why should there be when deleting messages? Why is trash useful for pop, but not for imap? Delete should delete the message from the list. imap delete does that, the message is just still stored. That is weird. And showing them makes delete not delete. That's weird to. Your story depends on one global trash to get a tiny bit of screen space. I think the space is worth the usability of just making delete mean delete.
just for reference, my last isp offered imap, i used it, mark as deleted was the only supported mode, the imap server did not allow other folders, and you could not use the imap server for arbitrary storage. (the backing store was mailspool, and once you removed a message from the Inbox 'folder', you could not put it back.)
In that case the Trash folder maybe should be a virtual folder showing the mark-as-deleted objects. Then cleaning the thrash would mean compacting folders. This could be automatic or manual, but at least is something the user knows about.
If the virtual folder where under the imap account, the user would be confused to not see those messages when at another PC (home<->work). Your suggestion also seems much less intuitive than the current "compact" for imap+m-a-d. BTW: Michiel, have u ever tried "imap + m-a-d" for any length of time or at all?
With a virtual folder, i mean a folder that displays the events that imap thinks are marked for deletion. It searches deleted messages in other folders. Those messages are still on the server. So it works at work and at home. But this isn't the topic of this bug. The question is: is compacting folders a concept users know about? Do they really need to suffer the pain because a few power users use some other mechanism? And why is imap different from pop?
Guessing: IMAP is more powerful than POP? POP dooesn't have m-a-d? Should we then dumb-down IMAP to be as limited as POP, just to make them "look" the same? (i.e., pretend that m-a-d doesn't exist)
Locally, e-mail programs can give POP messages whatever statuses they like, such as deleted-but-not-really, or junk-when-it's-raining, or flagged-in-a-full-moon. But they don't, because that would be silly. It's silly for IMAP too; possibly deleted-but-not-really is exposed in IMAP clients only because developers are thinking too much about the protocol and not enough about the mental model. Providing a consistent mental model, with the same power to discard messages, and the same ability to "Edit" > "Undo", is not dumber -- it's smarter.
If we must have a homogenous mental model, then add m-a-d to POP. I don't want to have to go hunting around in the trash and junk folders for accidentally junked or deleted messages - especially with bug 218467, bug 91218 and bug 149054 not really fixed yet.
yes, we want a homogenous model. Homogenous with the mental model the user has. And in that model, delete means to get rid of a message. Throw it away. Maybe into a thrash. It doesn't mean to say that maybe one day you might want to get rid of it. (and we are going around in circles here. One users want to mark as delete, all the others that don't even know about bugzille very likely want delete to mean delete. Lets stop commenting, and try to have the module owners make a decision)
Let's try to refrain from wild unproven claims, OK?(In reply to comment #26) > One users want to mark as delete, all the others [...] > very likely want delete to mean delete Let's try to refrain from wild unproven claims (that "happen" to support your preference), OK? Please note that IMAP is rarely used by novice users, and m-a-d is off by default. The defaut behavior would (i suppose) be that IMAP+delete would move a message to the trash folder, where this bug's request would apply. Maybe IMAP + m-a-d could auto-compact at session-end, thus also satisfying this bug's request? This (IIRC) unfortunately currently only works for the inbox, but not any of the other IMAP (sub)folders. That would be a more sensible solution. PS. I agree that the UI should be simple and consistent. However, the trick to excellent software is not to completely eliminate everything that you think might confuse some users, but to provide a clear and unobstructive path to ever more powerful functionality, without sacrificing the simplicity. "Lets stop commenting, and try to have the module owners make a decision" Agreed
It may be relevant that "compact" does three slightly separate things: 1. Local Folders & POP: Permanently removes invisible yet present deleted messages. 2. Trash: Emties the trash. (slightly related to "compact") 3. IMAP + Mark-As-Deleted: Permanently deletes messages the user has "marked" for deletion.
afaik, compact folder always deletes deleted-but-not-just-yet messages. I'm very dure it does that for imap. Thrash is not related to compact. I don't know if that folder auto-compacts at the moment, but the operation to empty it is called 'Empty Trash', not 'Compact' This bug is about compact, not about empty trash.
a few thoughts: Need an algorithm for timing the compaction. Offhand, I'd suggest >50% slack space as a good trigger, that scales with the file size. There's that auto-compact pref that uses a kilobyte number, but I'd guesstimate there isn't a good one-size-fits-all number for that. Compacting all folders on an account when its Trash is emptied (comment 13, bug 61960), would seem to imply very slow shutdown for folks with Empty Trash on Exit and largish folders. A fixed interval alone wouldn't be helpful when doing cleanup. If a folder is under the mark-as-deleted scheme, show Compact Folder context menu item on it as currently and don't consider the folder for automatic compaction. Unless .01% of the .01% (sombrero numbers, obviously) who use mark-as-deleted in the first place actually want to combine it with automatic cleanup. I suppose they'll cry pref if it comes to that.
How about just n seconds after the last operation on the mailbox, and when not very busy doing other things? (the delay makes sure you are not compacting 10 times for deleting 10 emails)
Consider: a 100M Inbox, auto-move to Junk folder, badly spammed under account, default biff interval, compact interval in the same ballpark as default biff interval; since the junk messages make a brief visit to the Inbox, we'd be writing an average of around ten megabytes a minute for about one kilobyte of incoming messages per minute, which is a bit much. That said, 10% slack space with 15min idle timer + on exit if necessary (similar to junk training data flush timing), sounds reasonable.
*** Bug 301195 has been marked as a duplicate of this bug. ***
Flags: blocking-aviary1.5? → blocking-aviary1.5-
Flags: blocking-aviary2.0?
I'm a user, not a developer. I've read through your lively discussion, understanding only half, maybe less. I only vaguely know about IMAP. I stopped using Local Folders when I couldn't get Send Later and Send Unsent Messages to work. I am POP and only POP. What I want is to choose 0 KB as the criterion for Automatic Compact Of All Folders Upon Close Of Application. If you implement it as an option, I'll be happy to be the only person in the world to choose it.
*** Bug 318190 has been marked as a duplicate of this bug. ***
Flags: blocking-firefox2? → blocking-thunderbird2?
*** Bug 319032 has been marked as a duplicate of this bug. ***
I agree with the concept that "Compact Folders" is a techo thing users shouldn't have to worry about. But, the mailstore design in Moz/TBird is not really designed for this and will have severe difficulties trying to implement it. On the whole I think that this is better considered in terms of using a new mailstore which CAN be compacted in the background - something more like a real database. Remember that some peopple have very large mailboxes. One of mine is 500MB. Issues to consider: When compaction happens, the mailbox is totally unusable until it's finished (right?). You can't start and stop, doing it in chunks, because it involves writing an entirely new copy of the mail file, not editing the existing file. I suppose it would be possible to come up with a scheme that accessed both the new and old mailbox as necessary, but that seems rather complex at the least. On laptops, compacting folders would keep the hard drives and CPU going potentially for hours, running the battery flat. It's not OK just to automatically tie up the hard drive when the user is away.
I think that auto-compacting is the correct way of dealing with mail folders. Pine (http://www.washington.edu/pine/) automatically compacts folders on the fly for years. The process is transparent to the user since the compacting process is done correctly: * The compacting process DOES NOT CREATE a new copy of the folder, but rather moves blocks of data within the file. * The compacting process DOES NOT BLOCK other operations on the folder, such as adding, reading mails or deleting (moving) emails. Users usually delete the most recent emails. Therefore the block to be moved contains the recent messages, and is relatively small even if you have 500Mb folders. Please consider automatic compacting with the assumption that it will be implemented correctly, not as it is done currently. If pine can do so, I am sure Thunderbird can do so too.
getting late in the cycle to block on this feature.
Flags: blocking-thunderbird2? → blocking-thunderbird2-
QA Contact: general
Hello I m only a user but I had often problems with "slows" thunderbirds. Now I know from where come this problem : the compacting of the folders isn't activated. If by using "compact", you mean "empty" trash, I don't really if it would be a good idea or not. If by using "compact", you mean "really delete the mails from the hard drive" (this mainly affect pop3 accounts in my case) I fully agree. The compacting of folders checkbox should be checked by default.This in an option for optimizing performance, for the confort of the user, but it s hidden in a dark option panel... I don't even count any more the number of times I "saved" user by just activate this option.
This goes against or alike bug 75927 which is target tb3? and btw, that one is "front end" not "general" ..
No, the main use cases are quite different. This is mainly about not having to know anything about compact at all, bug 75927 you want to know exactly that (and do it easily).
Assignee: mscott → nobody
THE WRONG WAY: THE CURRENT COMPACTING BUGS Let's face it, the current compacting mechanism is deeply flawed. The current implementation copies all the messages in the folder file into a new folder file, and replaces the original file. This solution is blocking (major evil), and is very inefficient (lesser evil). A proper implementation of the compacting mechanism -- which does not block other operations, which makes use of in-file movement of messages blocks rather then copying messages between files, which implement atomic operations -- can be automatic or semi-automatic and will fix most of the real compacting bugs (31, of which 5 are critical and 4 are major), while rendering many other bugs obsolete (at least 30). Here is an organized list of the relevant compacting bugs. Bugs which are related to the blocking misbehavior or the file-copy compacting "solution" side effects: bug 101584, bug 137210, bug 139215, bug 152675, bug 218075, bug 220179, bug 274330, bug 305930, bug 321277, bug 324576, bug 327674, bug 337554, bug 383599, bug 390351, bug 396470, bug 398684, bug 406851, bug 411971, bug 444343, bug 444697, bug 446316, bug 446437, bug 458335, bug 467395, bug 479285, bug 481650. Incorrect status after compacting: bug 368768, bug 369643, bug 370409, bug 380598, bug 414106. Bugs which are related to automatic/manual behavior: bug 61960, bug 160281, bug 177074, bug 220064, bug=286888 [this bug], bug 328945, bug 361806, bug 398523, bug 437657, bug 439089. Other bugs related to the compacting mechanism: bug 224825, bug 273334, bug 280804, bug 302205, bug 315366, bug 319142, bug 331108, bug 370229, bug 398471, bug 406204, bug 428947, bug 463359, bug 467305, bug 468722, bug 470834, bug 477486. Bugs which are related to the user interface of compacting (some already fixed, but still open): bug 161752, bug 190285, bug 255687, bug 324085, bug 341206, bug 345795, bug 384092, bug 390355, bug 423763, bug 460648, bug 460819, bug 471937, bug 473642, bug 477419, bug 478380, bug 479064. Enhancement requests related to the compacting mechanism: bug 235153, bug 235741, bug 285681, bug 291812, bug 306228, bug 306561, bug 379103, bug 401295, bug 413955, bug 416532, bug 469050. THE RIGHT WAY: HOW COMPACTING SHOULD BE DONE. The correct compacting mechanism is based on moving messages within the folder file, rather than moving massages between files. You are using two pointers in the messages file -- the first one points to the beginning of the unused space in the file, and the other points to the beginning of the first message after the unused space. For simplicity, let's assume the size of the message is less then the unused space, so copying the message can be interrupted by other actions, for example the user is viewing another message, moving a message to another folder or retrieving new messages from the mail server. All those operations can pause the in-file copying process, which can be resumed as the interrupting operation is completed. Once the message is fully copied, a short atomic operation is needed to update the message offset in the index/db file, and update the pointers of the unused space and the next message. When there is no next message, the file is truncated at the unused space pointer, and the operation is finished. Another minor note is the usage of a "dirty flag", which flags if the currently copied message was modified (for example, by detaching or deleting its attachments), and therefore the copying should be redone. If the message is larger then the unused space, there is a simple solution of blocking other actions for a second, or more complex solution of handling non-continuous messages in the file. This solution (which is used by Pine and other mail clients, as well as by almost all descent text/hex editors which handle large files), guarantees that the compacting operation does not block other operations, as the only blocking operation is the atomic update of the messages offsets. This mechanism is much faster, as the user usually erases new messages, which are stored near the end of the file. The new mechanism moves around relatively few messages, while the previous one copies all the older messages in the file. The new mechanism also avoids the long operation and the additional needed space for copying large message files. Since this mechanism does not block other operations, it can be done automatically as part of the regular operation of the mail application, without the the user awareness. All the UI problems and most of the enhancement request will become obsolete.
Blocks: 454074
Blocks: 306228
Blocks: 361806
I think this bug is duplicate with this one: https://bugzilla.mozilla.org/show_bug.cgi?id=341206 (at least, the roots cause is the same, but the solution just ask to change default behaviour and keep the UI).
I support
Zvi, thanks for presenting an organized view of the bugs. Bienvenu or someone else might be able to speak to the rest of comment 47, but I would suspect if a more efficient method of compaction was safe in the thunderbird environment that it would have been done already.
Component: General → Backend
Product: Thunderbird → MailNews Core
QA Contact: general → backend
perhaps this won't be known until we have feedback from thunderbird 5 in the field... given that - Bug 439089 exists for the efficiency of the backend process - Better Faster IMAP: Auto compact on idle time with better threshold (which isn't just about imap) - bug 437657 is fixed - increase autocompact threshold "mail.purge_threshold" [and pref on by default] - which addresses the automatic part of this bug is there anything further for this bug to tweak or do?
I can't follow this long thread. It seems you divert into stray subjects and other BUGs. The initial suggestion here was to make Compaction automatic without asking the user. Why? There's an option in preferences. If a user wants to make it automatic, one already can. What is missing in preferences window is an option, a "tick box" to ask the user. It is always ON. You may still turn it OFF in the list of preferences by searching for "purge". Compaction contains lots of BUGs, both known, and still unknown. A user must be able to stop it, or to prevent it from starting. Do not take control away from user. I suggest to close this suggestion for enhancement.
I really don't know if this was addressed b/c I didn't read this entire thread, but every time a compact is done & I have a reply e-mail open & I'm in the process of me writing it, I can no longer use that e-mail. TB won't save it I get an error. I have to copy & paste the e-mail into a brand new fresh reply. Lastly, I'm really not sure TB is compacting ALL the folders & I have HUNDREDS of them & sub & sub sub folders. It never tells me which folder it's compacting, how much longer it will take, & sometimes it can want to compact only minutes after I compacted earlier. I don't think it works b/c I now have to deal with my "sent" folder b/c it's corrupted. I've never seen it compact the "sent" folder, but then again I barely go into the "sent" folder, I just use the "search" function on the "sent" folder. Thanks Michelle
Mozilla/5.0 (X11; Linux x86_64; rv:9.0a1) Gecko/20110822 Firefox/9.0a1 SeaMonkey/2.6a1 ID:20110822003134 I have compacting set to 1 MB (the current minimum; before I had 512 K IIRC) and "ask me" (which already requires going to about:config AFAICT); and the few seconds it takes me to react when asked are enough, I think, to avoid collision with mail-fetch in most cases (collisions happened before, several versions ago, but I know some bugs have been fixed since). Also the small threshold makes compaction events more frequent but shorter which seems to help keep my mail folders clean, and avoid corruption (and I don't think I have thousands of folders, but I certainly have hundreds, though not more than two or rarely three levels of depth). In reply to comment #58: I see "Sent" being compacted after I have moved an email out of there (maybe in order to get both sides of a conversation into some other folder). This is normal since the email that has been moved away leaves a "gap" which is not reclaimed until the next compaction. It could also happen legitimately after an automatic "forgetting" of the oldest messages in Sent (if one of the "Delete if..." radio buttons is selected in the "Disk space" preferences for Local Folders).
Would there be a way to get it to do compacting when the overall CPU usage is at or near idle? Then have it go either in order of folders, or by large to small or small to large, or even better, allow folks to choose the compact order. I know this would need some more development of a user interface, but "one stop shopping" for compacting would not be a bad thing, and people can then argue up and down the merits of different options.

Wow, I was JUST thinking about this yesterday Wayne.

TB isn't telling me that it's compacting or that I need to compact.

That's not good, right?

See Also: → 716412
See Also: → 1678856

There is of course also maildir, which don't need compacting.

URL: https://bugzilla.mozilla.org/show_bug...
Depends on: 495666
See Also: → 1462666
Depends on: 1718443

I read the documentation on compacting folders:
https://support.mozilla.org/en-US/kb/compacting-folders

However, it is not clear to me why this is not an automatic-by-default operation on Thunderbird.
It reminds me of defragmentation of C drive. In Windows 98 it was a manual operation that took forever. Nowadays it's an automatic operation that runs every week or every 3 days, without using input.
Wouldn't it be better to run a small compacting task every week when Thunderbird has not had activity in say, the last hour, rather than one big compacting task when the limit is finally reached?

I know users who defer compacting because they know the impact of clicking yes will be huge. It might even freeze their program completely, so they keep pushing it for later...

Numerous users do NOT compact folders. They are most likely afraid of losing data. Over and over and over again, I see this on different computers. If it is beneficial, then just do this and be done with it. If there is a good reason to NOT compact folders, then stop pushing it, and end the discussion.

(In reply to Worcester12345 from comment #68)

Numerous users do NOT compact folders. They are most likely afraid of losing data. Over and over and over again, I see this on different computers. If it is beneficial, then just do this and be done with it. If there is a good reason to NOT compact folders, then stop pushing it, and end the discussion.

I agree with this. I'll add that in the several organizations that I've worked in, compacting is never clear to the user why it's beneficial. Two different options come to mind:
1-Does it save hard drive space at the expense of longer operations? (due to compression/decompression) and if so, is that desirable?
2-Does it make operations faster, since it optimises the database?

Also, I'm sorry to say this, but it's been 17 years. It'd be good to classify this as wontfix or to decide on a course of action.

Flags: needinfo?(mpt)

(In reply to L from comment #67)

I read the documentation on compacting folders:
https://support.mozilla.org/en-US/kb/compacting-folders

However, it is not clear to me why this is not an automatic-by-default operation on Thunderbird.
It reminds me of defragmentation of C drive. In Windows 98 it was a manual operation that took forever. Nowadays it's an automatic operation that runs every week or every 3 days, without using input.

It's a reasonable comparison.

Wouldn't it be better to run a small compacting task every week when Thunderbird has not had activity in say, the last hour, rather than one big compacting task when the limit is finally reached?

That won't work for users who don't keep Thunderbird running 24x7.

I know users who defer compacting because they know the impact of clicking yes will be huge. It might even freeze their program completely, so they keep pushing it for later...

Yes. That's one reason why this bug hasn't been implemented.

(In reply to Worcester12345 from comment #68)

If it is beneficial, then just do this and be done with it. If there is a good reason to NOT compact folders, then stop pushing it, and end the discussion.

If you understand how compact works and what it does, then you also know that there is good reason TO do compact.

(In reply to L from comment #69)

Two different options come to mind:
1-Does it save hard drive space at the expense of longer operations? (due to compression/decompression)

Yes

and if so, is that desirable?

Is what desirable?

2-Does it make operations faster, since it optimises the database?

In the short term, most likely not noticeable. In the long term, mostly like noticeable for most users. But the greatest benefit (at least for people who actually delete messages) is reducing disk space, because deleting messages doesn't reduce disk space until compact happens.

Also, I'm sorry to say this, but it's been 17 years. It'd be good to classify this as wontfix or to decide on a course of action.

This is not in the hands of the reporter.

Flags: needinfo?(mpt)

(In reply to Wayne Mery (:wsmwk) from comment #70)

(In reply to L from comment #67)
Is what desirable?
Is it desirable to have longer operations in order to save hard drive space? Today with ever increasing HDs for lower prices I'd say performance is what most users would go for. So if compact makes operations slower, it might not be a good thing in the end.

This is not in the hands of the reporter.
Apologies. Who can make a decision on this bug? Not trying to rush things, but it's clear that the report could be open for another 10 years if no one pushes..

Severity: normal → S3

(In reply to L from comment #67)

I read the documentation on compacting folders:
https://support.mozilla.org/en-US/kb/compacting-folders

However, it is not clear to me why this is not an automatic-by-default operation on Thunderbird.
It reminds me of defragmentation of C drive. In Windows 98 it was a manual operation that took forever. Nowadays it's an automatic operation that runs every week or every 3 days, without using input.
Wouldn't it be better to run a small compacting task every week when Thunderbird has not had activity in say, the last hour, rather than one big compacting task when the limit is finally reached?

That would make sense. However:
a. thunderbird stores messages on the disk in a poorly designed manner. Which necessitates compacting as a means to fix some of the problems it causes.
b. thunderbird compacting is poorly designed, which makes it extremely inefficient and buggy.

As such, the main argument people levied against compacting automation is that fixing compacting to not be so awful is a prerequisite blocking improvement.
Currently users need to be educated on what compacting is and then balance the need to do it as often as possible to deal with problem A, while avoiding it as much as possible due to problem B. With every person having a different opinion on how often they should or should not compact.

This has even led to some users instead writing up redesigns for the architecture that would mitigate those design issues.

You need to log in before you can comment on or make changes to this bug.