525622 - Delete messages x Days old not working

Status: RESOLVED WORKSFORME

(Thunderbird :: General, defect)

Description

[Nick Howitt]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.4) Gecko/20091016 Firefox/3.5.4 (.NET CLR 3.5.30729)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.4pre) Gecko/20090915 Thunderbird/3.0b4

I have a number of IMAP accounts. The global account settings say to never delete messages. I have overridden the global settings on the Sent and Trash folders to 30 days or 365 days, but my old messages never get deleted.

Reproducible: Always

Steps to Reproduce:
1.Set the default account setting to never delete messages
2.Set the Trash and Sent folders to delete after 30 days
3.Confirm that this has not changed the default settings (another bug!)
4.Restart TB - no old messages are deleted
5.Compact folders - no old messages are deleted
Actual Results:  
If you manage to set the accounts as I described (I cannot always as on one of my accounts, changing the Sent or Trash folder changes the default settings as well), old messages should be deleted, but they are not.

Expected Results:  
Messages older than the folder setting should be deleted.

In trying to report the bug I have turned up a secondary bug where folder settings sometimes get put back into the default account settings.

Comment 2

[Nick Howitt]

I do not believe the main bug (as in the bug summary) is a duplicate of bug 515316. What I am seeing as the main part of is that when I can get the retention policy of the folders changed (as I can on some accounts) to delete messages of a certain age, nothing happens to the old messages. My bug is not to do with the issue of setting the prefs on folders, it is to do with the prefs having any effect at all.

Comment 3

[Mark Banner (:standard8)]

Given the errors that 3b4 was showing in that area, I would suggest retest on a nightly build which has that fix before assuming they aren't the same bug.

Nightly builds are available here:

http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/latest-comm-1.9.1/

Comment 4

[Nick Howitt]

I've installed Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.5pre) Gecko/20091031 Shredder/3.0pre. I'm not sure if it is a nightly or hourly.

I can now maintain different retention policies on subfolders on all my accounts so that side of it looks OK.

What triggers the delete based the retention policy? Closing and re-opening TB/Shredder does not, neither does compacting.

Comment 5

[Nick Howitt]

I am now up to build Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.6pre) Gecko/20091105 Shredder/3.0pre.

No old documents are being purged using the folder retention policy overrides.

Comment 6

[Nick Howitt]

Looking a bit more closely, it seems that my first mailbox sub-folders may be purging correctly, but not my second or third mailboxes' sub-folders.

Comment 7

[rsx11m]

Nick, I'm reposting your diagram from the MZ feedback thread here to clarify the usage of "mailbox" and "subfolder" here:

>     IMAP Account #1:
>       Inbox (account default set to retain all)
>         Drafts (uses account default, don't know)
>         Sent (works)
>         Deleted (works)
>         Others (use account defaults, don't know)
>     IMAP Account #2:
>       Inbox (account default set to retain all)
>         Drafts (uses account default, don't know)
>         Sent (doesn't work)
>         Deleted (doesn't work)
>         Others (use account defaults, don't know)
>     IMAP Account #3:
>       Inbox (account default set to retain all)
>         Drafts (uses account default, don't know)
>         Sent (doesn't work)
>         Deleted (doesn't work)
>         Others (use account defaults, don't know)

Thus, you are talking about three distinct IMAP accounts with various folders. Are Drafts/Sent/Deleted on the same level as Inbox (you've drawn them with an indention, thus they suggest being on a second level, e.g., as for Gmail).

Also, it appears that IMAP Account #1 is your default account, and the only one for which any folder-specific retention settings work. What happens if you change the default to IMAP Account #2, thus making Account #1 a secondary mailbox? Does it work for #2 then but no longer for #1?

This would give some insight whether it depends on the default account (thus ignoring the others somewhere in the process) or on the internally maintained order of accounts.

Regarding comment #3, the reporter states that setting are retained in the Folder Properties, thus this can't be explained with bug 515316.

Comment 8

[Nick Howitt]

Drafts/Sent/Deleted are not on the same level as the Inbox. They are sub-folders of it. (I don't know is this is the normal IMAP setup, but it is the one used by my server)

The only way I know of changing default accounts is to use FolderPane Tools. Changing the default account like this did not then enable Account #2 or Account #3 to auto-delete its old messages.

Comment 9

[rsx11m]

There is now also bug 531463 on similarly retention policy not working in subfolders, that may be related (though it works here at least for one of
the servers).

Comment 10

[WADA:World Anti-bad-Duping Agency]

Quick/ad-hoc log check with Tb 3 2009/11/12 build, using DebugView.
> SET NSPR_LOG_MODULES=imap:5,ImapAutoSync:5,IMAPOFFLINE:5,MsgPurge:5
> SET NSPR_LOG_FILE=WinDebug
> See https://wiki.mozilla.org/MailNews:Logging and Bug 402793 Comment #6

(1) An IMAP account, not default account, Disable IDLE
 a. Junk purge : Disabled,
 b. Account's sync setting : Don't delete any message
 c. A sub folder under Inbox : Delete messages more than 360 days old
 => Message purge of IMAP was never invoked
(2) The IMAP account, not default account
 a. Junk purge : Disabled,
 b. Account's sync setting : Delete messages more than 360 days old <= alter
    Because "Use Server Setting" is checked by many IMAP folder,
    this enables many IMAP folder's "delete messages older" option.  
 c. A sub folder under Inbox : Delete messages more than 360 days old
 => Message purge of IMAP didn't look to be inoked
(3) The IMAP account, not default account
 a. Junk purge : Enabled <= alter
 b. Account's sync setting : Delete messages more than 360 days old
 c. A sub folder under Inbox : Delete messages more than 360 days old
 => Message purge was invoked for many IMAP folders.

I'm not sure (2)/b was required action or (3)/a was required action.

What do you set in next account level msgpurge options for the IMAP account?
 - Account Settings/Junk Settings
 - Account Settings/Syncronization and Storage

Comment 11

[Nick Howitt]

I've set up logging as you suggest. At the end of the TB startup, I can see it purging the primary account and local folders (not used).

I am not sure what you mean by msgpurge options, but on each account I have:

Server Settings:
Expunge Inbox on exit: false
Empty deleted folder on exit: false
Advanced: All options checked and set to default.

Synchronisation & Storage:
Do not delete any messages
Message synchronising (keep messages for this account on this computer): not checked
Synchronise all messages locally regardless of age
Always keep starred messages: checked

Junk Settings:
Enable adaptive junk mail control: Checked
Do not mark mail as junk: If in personal and collected addresses
Trust junk mail headers set by spam assassin
Move new junk  messages: to junk folder under each inbox

On primary account: Automatically delete junk mail older than 7 days enabled
On other accounts: Automatically delete junk mail older than x days disabled

In the log with these settings, I could see the primary account and local folders being purged, so I enabled "Automatically delete junk mail older than 7 days" and I'm now in such a state that on my secondary accounts I have lost the ability to set the various account options and have them kept.

I would normally expect cases (1), (2) and (3) above all to work. I cannot get any of them to work.

Comment 12

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #11)
> I am not sure what you mean by msgpurge options, (snip)

Sorry for unclear wording. I meant next by "msgpurge options";
> (a) Junk Settings:
>     "Automatically delete junk mail older than NN days" is enabled or not
> (b) Synchronisation & Storage:
>     "Do not delete any messages" is checked or not,
>     delete by age or status is enabled or not  
> (c) per folder Folder Properties/Retention Policy setting

Problem like next?
 If "Junk Purge" is disabled, last purge timestamp is cleared.
 If last purge timestamp is cleared, msgpurge of IMAP is not invoked.

Comment 13

[Nick Howitt]

My profile died yesterday - on the secondary accounts I could not get any of the Junk Settings to stick. I could only enable and disable the adaptive junk controls.

I have now created a new profile and from what I've read here, I've enabled "Automatically delete junk mail older than NN days" on all accounts. With this enabled first time, old messages were purged correctly. On one secondary account, I then unset the flag and restarted TB. Messages on this account were no longer purged.

For the account which is no longer purging, if I look at the log file as in comment #10, it says "[1] purgeSpam=false (if false, don't purge)" but in the account settings, it is very clearly enabled and set to 14 days! In the config editor all mail.server.serverx.purgeSpam settings show true. mail.server.default.purgeSpam is false.

Comment 14

[WADA:World Anti-bad-Duping Agency]

I could observe msgpurge of IMAP folder when I kept Tb3 running all night. So I tried small mail.purge.timer_interval and mail.purge.min_delay.

I could observe msgpurge by (c) only at last.
> Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091129 Shredder/3.0.1pre
> mail.purge.timer_interval = 1 (default=5 min)
> mail.purge.min_delay = 3      (default=480 min)
> Gmail IMAP (To see "deleted" status, IMAP Delete model=Mark as deleted is set)
> (a) Junk Settings:
>     "Automatically delete junk mail older than NN days" : DISABLED
> (b) Synchronisation & Storage:
>     "Do not delete any messages" is CHECKED.
>     Any delete by age or status is Disabled
> (c) per folder Folder Properties/Retention Policy setting
>     An IMAP folder (Inbox/F1) :
         [ X ] Delete messages more than 180 days old
>     All other      : [ X ] Use my account settings (never purge by retention)
> Inbox/F1 : some old mails : Date = 1970/01/01
>            some new mails : Date = 2009/10/nn           
After restart of Tb, I kept Tb3 running, then I could see "old mails(1970/01/01) are deleted" after a while.

I don't know spec of mail.purge.min_delay, but I guess;
> Because default of mail.purge.min_delay is 480(480 min, 8 hours),
> it takes long to execute msgpurge after check for retention by mail.purge.timer_interval.

Comment 15

[Nick Howitt]

The last bit of my comment #10 is incorrect. The "[1] purgeSpam=false (if false, don't purge)" came from Local Folders which I am not using.

I've updated to RC2 today and purging appears to work irrespective of the setting of "Automatically delete junk mail older than x days disabled" which is what I would expect.

Unless anyone has other issues I'd be happy for the bug to be closed.

Comment 16

[rsx11m]

I'm closing this bug per comment #15, feel free to reopen if the issue isn't resolved and you have further information/steps to reproduce you can add.

-> WFM