11055 - Message aging (empty trash/auto-delete messages after N days)

Status: VERIFIED FIXED

(MailNews Core :: Backend, enhancement, P3)

Description

[(not reading, please use [e-mail] instead)]

marking m15

Comment 1

[Phil Peterson]

Bulk-resolving requests for enhancement as "later" to get them off the Seamonkey
bug tracking radar. Even though these bugs are not "open" in bugzilla, we
welcome fixes and improvements in these areas at any time. Mail/news RFEs
continue to be tracked on http://www.mozilla.org/mailnews/jobs.html

Comment 2

[Phil Peterson]

Reopen mail/news HELP WANTED bugs and reassign to [e-mail]

Comment 3

[Jarrod Gray]

*** Bug 68430 has been marked as a duplicate of this bug. ***

Comment 4

[R.K.Aa.]

*** Bug 100059 has been marked as a duplicate of this bug. ***

Comment 5

[R.K.Aa.]

*** Bug 105824 has been marked as a duplicate of this bug. ***

Comment 6

[David :Bienvenu]

*** Bug 106001 has been marked as a duplicate of this bug. ***

Comment 7

[Matt Behrens]

Ahh, whoops, didn't see this in my search for forms of "expire".  Sorry.

Let me transfer a comment I made in bug 106001 over here though: rather than
just delete after x days, it would be more useful to perform other actions, such
as move to an archive folder.  It may be useful to have a different kind of
message filter, one that processes an entire folder on first folder open every
24 hours or client open.

Comment 8

[Sebastian Späth]

The mail filter UI already includes a "age is N days" possibility, I don't know
if that is functional though. So deleting or archiving shouldn't be a problem
once the bug about filter after message has been downloaded and implement filter
for news have been fixed (don't know the numbers offhand).

Comment 9

[R.K.Aa.]

*** Bug 121846 has been marked as a duplicate of this bug. ***

Comment 10

[laurel]

*** Bug 128307 has been marked as a duplicate of this bug. ***

Comment 11

[Jason Bennett]

If this bug is going to be a catch-all, it needs to have a better description.
As it is, it's impossible to find this bug with most searches on this topic. At
least mention the trash in the summary.

Comment 12

[Jonas Jørgensen]

Like this? :-)

Comment 13

[Christopher Aillon (sabbatical, not receiving bugmail)]

*** Bug 129244 has been marked as a duplicate of this bug. ***

Comment 14

[Jilles van Gurp]

The comments for this bug talk about deleting mail on the client side. 
However, I would like to see serverside removal after N days. Outlook has this 
setting when you select leave mail on server. I think most of the bugs marked 
as duplicates of this one (including one of my own a couple of months back) 
are about this feature.

I depend on this to look at my pop account from home and work. At home I have 
leave mail on server selected whereas at work it is deleted from the server 
after a couple of days. I never throw away any fetched email (except spam) so 
I don't want to delete it locally, just on the server.

Comment 15

[laurel]

*** Bug 140374 has been marked as a duplicate of this bug. ***

Comment 16

[Jim Booth]

I agree with Jilles!  The ability to automatically delete FROM THE POP SERVER
after x days is a VERY USEFUL FEATURE.  Eudora has had it as long as I can
remember (1997 or so).  

Anyone who accesses their mail account from more than one computer can really
benefit from this feature.  I hope someone will give it some priority!

Comment 17

[laurel]

*** Bug 171670 has been marked as a duplicate of this bug. ***

Comment 18

[chemist109]

Jilles and Jim Booth hit the nail right on the head.  I would prefer to use
Mozilla as my mail client everywhere, but am forced to use Eudora on one of my
machines (at work).  Anyone who wants to maintain mail on more than one machine
NEEDS this feature!

Comment 19

[Jim Booth]

Please see Bug 107883 for the issue of Deleting from server after X days, and
transfer your votes to that bug if appropriate.

Comment 20

[Mike Palczewski]

I'm not sure if this applies to this bug, but I would like to see messages older
than 1 month deleted from the Trash on my IMAP server.  All the previous talk
has been about POP so I thought I'd give my 2 cents. 

Comment 21

[Igor Velkov]

Yes, I was confused, too, whet not found "age" in filter for imap folder. Why
this option not available for IMAP folders? 

Comment 22

[Jo Hermans]

*** Bug 192156 has been marked as a duplicate of this bug. ***

Comment 23

[Joe McCabe]

Code to track the age of a message an take action based upon it is already
functional (at least for IMAP and Local Mail). It's used by the Junk Mail
feature to delete messages in the Junk Folder older than a certain age.

Comment 24

[Robert Wall]

*** Bug 194591 has been marked as a duplicate of this bug. ***

Comment 25

[Daniel Wang]

*** Bug 190295 has been marked as a duplicate of this bug. ***

Comment 26

[Vaclav Dvorak]

Just out of curiosity: how much would one have to pay someone to get a feature
like this implemented?

Comment 27

[eric gorr]

I believe the addition of this single feature, which as someone pointed out, has
existed in Eudora for a very long time, would allow me to make the switch from
Eudora to Thunderbird. I've even dedicated a vote to this bug.

Comment 28

[Jim Booth]

Bug 100059, bug 105824 and bug 129244 are duped against this bug when they
should instead be duped against Bug 107883 - Feature request: Remove from server
after x days (POP).  

Please note the difference.  THIS bug refers to removing from CLIENT.  Bug
107833 refers to REMOVING FROM A (POP) SERVER after X DAYS, and has 40 votes and
10 dupes on it already.  

Please move your VOTES accordingly if that's the feature you were asking for.

Comment 29

[Igor Velkov]

I want to add my vote to auto-delete from imap folder. where I can do it?

Comment 30

[David :Bienvenu]

vote for this bug, I think. The pop3 delete from server after XX days feature
has been implemented and is in 1.6 daily builds now. Auto-aging from the client
is this bug, at this point.

We already have auto-aging for the junk mail folder (IMAP or LOCAL/POP3). It
would be fairly easy to extend that to folders in general. We'd need a UI for
specifying it, however. If it's just aging the trash, we could have a UI in the
account for the server...

Comment 31

[Igor Velkov]

At work I have admin mailbox, with some imap folders, contains messages from
some robots. I think it will be suteble to auto-delete messages from robost in
this folders after 40 days.

Comment 32

[Jason Barnabe (np)]

*** Bug 222515 has been marked as a duplicate of this bug. ***

Comment 33

[Mark Stier]

Automatic export to a product-independent format may be great, too.

Comment 34

[wzeikat]

vote for this bug, and for http://bugzilla.mozilla.org/show_bug.cgi?id=223200
which has hardly gotten any votes so far

Comment 35

[Peter Kroll]

for consistency. if we age on the server (fixed in 1.6a) then we should have the
age in trash bin/specified folder in this release as well. 

Comment 36

[Howard Chu]

I was just looking for something like this. I actually stumbled over the
RetentionSettings first, but those only seem to apply to News folders. I get the
feeling that the RetentionSettings and the MsgPurgeService could be merged into
a single feature. Maybe all of the flexibility of the RetentionSettings is
overkill for general folders...

Comment 37

[Jaap A. Haitsma]

*** Bug 252867 has been marked as a duplicate of this bug. ***

Comment 38

[Wayne Mery (:wsmwk)]

severity should be 'enh'?

Comment 39

[Travis Chase]

*** Bug 248248 has been marked as a duplicate of this bug. ***

Comment 40

[Stefan Borggraefe]

*** Bug 258566 has been marked as a duplicate of this bug. ***

Comment 41

[Stefan Borggraefe]

*** Bug 274911 has been marked as a duplicate of this bug. ***

Comment 42

[Dale]

Well, here is someone else that wants a setting to remove/delete old messages. 
I used this a lot in Kmail before I switched to Mozilla-mail.  I like
Mozilla-mail but this may get me to start using Kmail again.  

The reason I switched was I could right click a link and it open in Mozilla as
new tab.  I guess I could ask Kmail to see if they can do that like Mozilla-mail
does.  0_0

This would be a good option to add.  Kmail has it so I know it can be done.

Dale

Comment 43

[Jonas Bengtsson]

Attachment: Dialogbox example

Comment 44

[ilf]

i also want this feature -> voted.

also i'd like to automatically let tb delete messages in folders when the # of
mails in those folders surpasses a certain amount.
example: i want to have only the last 1000 mails of a mailinglist and everyone
before that should be automaticaly deleted.

Comment 45

[ilf]

*** Bug 291468 has been marked as a duplicate of this bug. ***

Comment 46

[David :Bienvenu]

Attachment: work in progress

this hooks up the UI for the existing backend aging code, and makes that
backend code work for local and imap (it previously just worked for news).

This patch allows you to set aging preferences on a per-server basis, and
override them on a per-folder basis. The aging preferences are the same as the
existing news preferences, without the settings that affect offline msg bodies.


There are a few issues with this patch:

1. It currently applies the retention settings only when you open a folder. So
if you never open a folder, we never apply the retention settings to that
folder. My plan here is to extend the nsMsgPurgeService to also do message
aging. In fact, the junk mail purging stuff could really be replaced with this
UI, except junk mail is more discoverable with its current UI.

2. This should be added to the RSS server account settings UI, which doesn't
play in the normal server settings game. 

3. We need some sort of big scary warning that messages that are aged away will
be permanently deleted. I've added some text to that effect to the account
settings UI, but it might not be scary enough.	We don't do archiving yet, and
I don't plan on doing that in 1.1

Comment 47

[David :Bienvenu]

adding Neil and Scott to cc list for their input.

Comment 48

[[e-mail]]

1. Maybe the junk mail UI should just open the folder properties dialog?

3. Perhaps reversing the sense of the text would help?
"To recover disk space, old messages can be permanently deleted.
 (o) Don't delete any messages
 (o) Delete messages more than [  ] days old
 (o) Delete all but the last [  ] messages
 [x] Always delete read messages
(is that really what it does?)

Comment 49

[David :Bienvenu]

Not sure how 1) would work - I guess we could add a button to the junk controls
2) needs to be fleshed out more :-)
3) I like that a lot. I'll make those changes. Yes, that option really deletes
all read mail. It's intended for newsgroups where you never want to keep read
messages - it makes a little more sense for news, I guess, because you're only
deleting the headers. I'm open to removing it if people think no one would ever
use it, or hiding it for non-news.

Comment 50

[Jens Bannmann]

(In reply to comment #49)
> Yes, that option really deletes
> all read mail. It's intended for newsgroups where you never want to keep read
> messages - it makes a little more sense for news, I guess, because you're only
> deleting the headers. I'm open to removing it if people think no one would ever
> use it, or hiding it for non-news.

Yes, it should definitely be hidden; for mail it almost never makes sense and no
one should accidently enable it.

> (o) Delete all but the last [  ] messages

Making that "latest" would be more clear.

Comment 51

[ilf]

(In reply to comment #50)
> Yes, it should definitely be hidden; for mail it almost never makes sense and no
> one should accidently enable it.

no it shouldn't.
that's exactly what i want and requested in
https://bugzilla.mozilla.org/show_bug.cgi?id=291468

Comment 52

[David :Bienvenu]

I meant hide just the option that deletes all read mail, not all the auto-aging
options. I see no reference to wanting to delete all read mail in bug 291468

Comment 53

[Robert Dahlem]

(In reply to comment #50)
> > (o) Delete all but the last [  ] messages

> Making that "latest" would be more clear.

"Latest referring to Date written" or "latest referring to order received"?

Making that "last|latest according to [whatever] sort order" would be even
clearer.  

Comment 54

[David :Bienvenu]

Attachment: proposed fix

this builds on the previous patch to make the purge service apply retention
settings to folders, if they haven't been applied in the last 8 hours. It also
fixes it so we do  junk purging every 8 hours instead of every 5 minutes, which
we were doing because of a 32 bit int overflow.

With the current patch, we apply retention settings whenever you open a folder
- we might want to avoid doing that every time, but people with retention
settings would tend to have smaller folders anyway.

Comment 55

[David :Bienvenu]

Attachment: address Neil's comments

Comment 56

[David :Bienvenu]

Attachment: get rid of radio button id's, fix command updating, fix overriding server defaults in db.

the one outstanding issue is that in folderPane.xul, the keep unread only
checkbox doesn't line up with the radio buttons.

Comment 57

[[e-mail]]

Comment on attachment 183834 [details] [diff] [review]
get rid of radio button id's, fix command updating, fix overriding server defaults in db.

>+<!ENTITY retentionCleanup.label "To recover disk space, old messages can be permanently deleted">
You didn't make these wording changes in the mailnews/ verions... 

>+    var titleStringID = (gIncomingServer.offlineSupportLevel >= 10) 
>+     ? "prefPanel-offline-and-diskspace" : "prefPanel-diskspace";
Nit: prefer the ? at the end of the previous line.

>+        <radio wsm_persist="true" id="retention.keepAllMsg" value="1" accesskey="&retentionKeepAll.accesskey;"
While you're changing these lines I wonder if you could get away with removing
the wsm_persist="true" on all the <radio>s.

>+            label="&retentionKeepUnread.label;" hidefor="" checked="true"/>
Is this hidefor="" to unhide the box for a non-deferred server? Seems a bit
obscure ;-)

>+function onUseDefaultRetentionSettings()
>+{
>+  var useDefault = document.getElementById("retention.useDefault").checked;
>+  document.getElementById('retention.keepMsg').disabled = useDefault;
>+}
Nit: it would be nice if you could arrange to disable the number of
days/messages textboxes as appropriate too.

>+	<tab id="Retention" label="&retention.label;"/>
Nit: We allow <tab>s but not TABs ;-)

>+ * -*- Mode: Java; tab-width: 2; indent-tabs-mode: nil; c-basic-offset: 2 -*-
So why does all the code have 4-space indents?

>+    document.getElementById("retention.keepMsg").setAttribute("value", retentionSettings.retainByPreference);
>+    if(retentionSettings.daysToKeepHdrs > 0)
>+        document.getElementById("retention.keepOldMsgMin").setAttribute("value", retentionSettings.daysToKeepHdrs);
>+    else
>+        document.getElementById("retention.keepOldMsgMin").setAttribute("value", "30");
>+    document.getElementById("retention.keepNewMsgMin").setAttribute("value", 
>+        (retentionSettings.numHeadersToKeep > 0) ? retentionSettings.numHeadersToKeep : 30);
>+
>+    document.getElementById("retention.keepMsg").value = retentionSettings.retainByPreference;
This supercedes the setAttribute("value", ...) above. In fact, the intervening
setAttribute calls do not work, they should be .value too. The reason is that
if you change the value, and switch to another disk space value, the value will
not get overwritten if you only set the attribute.

>+    var retentionSettings = new Array;
Could be just new Object; or {}; or you could even use
return {
  retainByPreference: document.getElementById(...).value,
  daysToKeepHdrs: ... ,
  numHeadersToKeep: ... ,
  keepUnreadMessagesOnly: ...(...).checked
};

>+    switch (document.getElementById("retention.keepMsg").value)
>+    {
>+      case "1":
>+        document.getElementById("retention.keepNewMsgMin").setAttribute("disabled", "true");
>+        document.getElementById("retention.keepOldMsgMin").setAttribute("disabled", "true");
>+        break;
>+      case "2":
>+        document.getElementById("retention.keepOldMsgMin").removeAttribute("disabled");
>+        document.getElementById("retention.keepNewMsgMin").setAttribute("disabled", "true");
>+        break;
>+      case "3":
>+        document.getElementById("retention.keepNewMsgMin").removeAttribute("disabled");
>+        document.getElementById("retention.keepOldMsgMin").setAttribute("disabled", "true");
>+        break;
>+    }
It might be worth turning the logic on its head i.e.
var keepMsg = document.getElementById("retention.keepMsg").value;
document.getElementById("retention.keepOldMsgMin").disabled = keepMsg != 2;
document.getElementById("retention.keepNewMsgMin").disabled = keepMsg != 3;

r=me if you fix at least * the dtds * the TAB * the indents * the .values

Comment 58

[David :Bienvenu]

I'll fix the dtds and the indents and the .values, but there really isn't a TAB
there - it's just the diff "+ " that makes the indent look wrong :-)

Comment 59

[[e-mail]]

(In reply to comment #58)
>I'll fix the dtds and the indents and the .values, but there really isn't a TAB
>there - it's just the diff "+ " that makes the indent look wrong :-)
I double-checked and found three other TABs too:
+<!ENTITY retentionUseDefault.label	"Use server defaults">
                                   ^^^^^ (appears in both locale files)
+	var retentionSettings = saveCommonRetentionSettings();
 ^^^^^^^

Comment 60

[David :Bienvenu]

Attachment: address most of Neil's comments.

Comment 61

[David :Bienvenu]

Comment on attachment 184938 [details] [diff] [review]
address most of Neil's comments.

carrying forward Neil's r=, and requesting a=1.1a2

Comment 62

[chris hofmann]

Comment on attachment 184938 [details] [diff] [review]
address most of Neil's comments.

a=chofmann

Comment 63

[Jason Bennett]

Thanks for all your work on this, David. Looking forward to checking this out!

Comment 64

[ilf]

thanks a lot!! in which version will this be implemented?
sorry, if this is the wrong place to ask, i'm kinda new.

Comment 65

[David :Bienvenu]

tbird 1.1, and seamonkey 1.8, and nightly trunk builds starting tomorrow.

Comment 66

[Igor Velkov]

I got mozilla 20050602, set "keep messages which have been arrived within the
last 40 days" on my trash folder, located on imap server - and see tne last 40
messages instead around 4000 messages, received during the last 40 days. Maybe, bug?

Comment 67

[David :Bienvenu]

yes, that was fixed Friday, so it's in Saturday's build.

Comment 68

[David :Bienvenu]

Comment on attachment 183554 [details] [diff] [review]
proposed fix

clearing request

Comment 69

[Jens Bannmann]

*** Bug 223200 has been marked as a duplicate of this bug. ***

Comment 70

[David :Bienvenu]

*** Bug 232839 has been marked as a duplicate of this bug. ***

Comment 71

[tingmingtian]

On thunderbird 3.0beta 3 and All platform,the bug still reproduce.

Comment 72

[Adrien Roger]

On TB3 RC1 Build1, Linux version, fr locale (Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.9.1.5) Gecko/20091112 Thunderbird/3.0), the bug is still here.