Open Bug 647493 Opened 14 years ago Updated 1 year ago

Retention policy disregarded for imap folder

Categories

(Thunderbird :: Folder and Message Lists, defect)

Product:
Thunderbird
Component:
Folder and Message Lists
Platform:
x86
All
Type:
defect

Tracking

(Not tracked)

Status:
UNCONFIRMED

People

(Reporter: firstpeterfourten, Unassigned)

References

Details

(Keywords: testcase, Whiteboard: [dupme])

User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.16) Gecko/20110319 Firefox/3.6.16 (.NET CLR 3.5.30729) Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.15) Gecko/20110303 Thunderbird/3.1.9 Newsgroup retention policy is disregarded on load. Reproducible: Always Steps to Reproduce: I am subscribed to an IMAP server and there's a particular high-traffic bulletin board (newsgroup) that I've been subscribed to for a few months. 1. Right click on the newsgroup folder and choose Retention Policy tab. 2. Uncheck Use my Account Settings, if checked. 3. Select "Delete all but the most recent 100 messages." Check Always Keep Starred Messages. 4. Click OK. 5. Right click on folder again. 6. Click Compact. 7. Select all to observe that the folder now has only 100 messages in it, as desired. 8. Click to view another folder. 9. Click back to view the original folder. Actual Results: 10. All of the messages, from the time I started subscribing, are re-downloaded from the server. There are now 2400 messages in this folder. Expected Results: 10. If there are a few new messages, they are downloaded, and may replace some older unstarred messages (or bring the total over 100, until another Compact command). The previously deleted messages should not be restored. Many of the messages have attachments. This folder has filled my hard drive up completely on one machine and is getting there on a second. That is a major issue because it crashes the computer completely!
Possibly related: Bug 479207, "Message aging / retention (delete messages more than N days old) doesn't work"
This has been replicated by another subscriber (who will hopefully come comment here) who uses a temporal (7-day) retention policy. The newsgroup host thinks it's probably related to the fact that I don't have the "write" permissions that would let me delete messages from that [shared] folder, because that would delete them for all users. Pun intended. I've also looked at Tools->Options->Advanced->Network & Disk Space->Disk Space, where it says "Use up to 50 MB of space for the cache." 40+ GB is definitely not respecting that limit. Setting it to 0 doesn't help.
(In reply to comment #0) > I am subscribed to an IMAP server and there's a particular high-traffic > bulletin board (newsgroup) that I've been subscribed to for a few months. Is this a newsgroup exposed as a folder on an IMAP server or a newsgroup on an NNTP server? That makes a big difference. > Many of the messages have attachments. This folder has filled my hard drive up > completely on one machine and is getting there on a second. That is a major > issue because it crashes the computer completely! NNTP does not store messages offline by default. IMAP does. Either way, you can specify whether or not you want the files to be stored offline (Synchronization & Storage tab in account settings). You can also specify to not store very large offline messages (cut off by total size). (In reply to comment #2) > I've also looked at Tools->Options->Advanced->Network & Disk Space->Disk Space, > where it says "Use up to 50 MB of space for the cache." 40+ GB is definitely > not respecting that limit. Setting it to 0 doesn't help. That is not for the offline storage cache, that is for the generic Necko cache, i.e., backing disk store for the memory cache.
(In reply to comment #3) > Is this a newsgroup exposed as a folder on an IMAP server or a newsgroup on an > NNTP server? That makes a big difference. It is a newsgroup exposed as a folder on an IMAP server. Thunderbird calls it a newsgroup, so I used that term. > NNTP does not store messages offline by default. IMAP does. > > Either way, you can specify whether or not you want the files to be stored > offline (Synchronization & Storage tab in account settings). You can also > specify to not store very large offline messages (cut off by total size). These policies should be overridden by the per-folder retention and synchronization policies. I want to keep local copies of even large or old messages for the important e-mail folders, such as documents circulated with my work group. There are a few places where one can select the folder for "synchronization" for offline use. These are selected by default, and selecting/deselecting a folder doesn't affect its subfolders. Also, when a folder is added, it is selected for synchronization by default. As I have two machines (A and B), if I add the folder on B - even as unsynchronized - it appears on A as a folder to be synchronized. Certain operations may have the effect of unsubscribing and resubscribing to a folder, which has invisible effects to the user of that machine, but may cause the other one to start synchronizing that folder. I now suspect that this may have happened to me, and that the backup I relied on (retention policy) just didn't hold up either. If I find other clues or details, or see a reset of synchronization policy, I'll post that information.
(In reply to WBT from comment #4) > (In reply to comment #3) > > Is this a newsgroup exposed as a folder on an IMAP server or a newsgroup on an > > NNTP server? That makes a big difference. > > It is a newsgroup exposed as a folder on an IMAP server. Thunderbird calls > it a newsgroup, so I used that term. WBT, do you still see your issue? I changed mozilla.test newsgroup to 20 message retention, posted 7 new articles, the 7 oldest articles aged off leaving just 20 articles
Flags: needinfo?(firstpeterfourten)
I just reproduced everything in Comment 0 (noting that I previously missed a substep 1b: click "Properties" in the right-click menu). On this occasion, "use my account settings" had been selected, there were >2100 unread messages in the newsgroup folder. After following the steps, there were 100 messages marked unread (those that survived the Compact step) and all the 2K+ messages re-downloaded in Step 10 were marked read. Is there reason to believe this would have been fixed?
Flags: needinfo?(firstpeterfourten)
comment 0 mentions compact, so this can't be about a newsgroup. It must be an imap folder. There are occassional reports of retention issues. http://mzl.la/1QTZg7o I won't be of much help here, but at least the summary now reflects reality
Summary: Retention policy disregarded for newsgroups → Retention policy disregarded for imap folder
David, can you distill comment 0, comment 4, comment 6 to something actionable?
Flags: needinfo?(david)
See Also: → 806973
Action item in comment #9 was misdirected to the wrong David. I do not use IMAP and and no experience with it.
Flags: needinfo?(david)
(In reply to David E. Ross from comment #10) > Action item in comment #9 was misdirected to the wrong David. I do not use > IMAP and and no experience with it. Sorry, summary is wrong - this is actually about news - ref. comment 6
Flags: needinfo?(david)
Flags: needinfo?(antoine.mechelynck)
Summary: Retention policy disregarded for imap folder → Retention policy disregarded for newsgroup folder
(In reply to Wayne Mery (:wsmwk) from comment #11) > (In reply to David E. Ross from comment #10) > > Action item in comment #9 was misdirected to the wrong David. I do not use > > IMAP and and no experience with it. > > Sorry, summary is wrong - this is actually about news - ref. comment 6 Nevertheless, I am not at all familiar with this bug report. The circumstances presented in the Description is definitely not how I setup my use of newsgroups.
Flags: needinfo?(david)
1. Sorry for late reply. 2. Sorry, not using Mail & Newsgroup anymore, I read my mail by webmail nowadays.
Flags: needinfo?(antoine.mechelynck)
Summary: Retention policy disregarded for newsgroup folder → Retention policy disregarded for imap folder
Whiteboard: [dupme]

(In reply to WBT from comment #4)

(In reply to comment #3)

Is this a newsgroup exposed as a folder on an IMAP server or a newsgroup on an
NNTP server? That makes a big difference.

It is a newsgroup exposed as a folder on an IMAP server. Thunderbird calls
it a newsgroup, so I used that term.

I still cannot fathom this. Please post a screen shot of the folder properties general information tab, so we can clear this up

Flags: needinfo?(firstpeterfourten)

Apologies, I no longer use TB after too many years of waiting on unfixed bugs like this which seriously impaired my ability to use the computers for other tasks. I did get very detailed in the original post and would prefer if someone else tried duplication, instead of requiring that only OP return 8 years later or else have the bug closed for everybody.

Flags: needinfo?(firstpeterfourten)

Hopefully Ben can have a look. Gene or someone else was also looking at retention issues

Keywords: testcase
OS: Windows Vista → All

I know everyone is busy, but Gene do you mind having a look? Wayne which Ben are you referring to?

Flags: needinfo?(gds)

I tried on a gmail imap folder with about 10 test (useless) messages in it. I told to to keep just the most recent 3 and it worked, but compact was needed. However, what I see is that it is not based on the date of the message but on the order received (actually UID of the message). So it actually kept the 3 messages with the highest uid which means they were most recently placed in the folder (but not necessarily the 3 with most recent date).

I then copied another message to the folder with the limit of 3 messages. This added a 4th message that stayed. Only compact again deleted the "oldest" message. Looking at imap log, I see that the older messages are only marked as deleted when a compact is done. I haven't seen a background thread (like autosync) going through and trimming folders automatically . So this might be considered a bug. However, I don't see old messages that were previously deleted being re-downloaded as mentioned in comment 0.

Note also, that the reporter mentioned shared folders. That might be why he reported a problem and I don't see the same problem with a normal folder.

Flags: needinfo?(gds)
Severity: normal → S3

WFM per comment 18

Hi! Just to confirm that this still happens. With a number of mailboxes of various accounts. One is an inbox of 17000 messages with perhaps 300 messages per day that makes Thunderbird hang for more than one minute every time I open it. The retention policy never clears out old messages; neither with a number limit nor with a days limit. It is on an IMAP server that is shared among a handful of users.

Thunderbird 115.7.0 (64-bit) on Archlinux.

I couldn't find details on when Thunderbird triggers the retention policy cleaning.

Note, Compact forces a purge action, which is what triggers the retention check.

(In reply to gene smith from comment #18)

...
I haven't seen a background thread (like autosync) going through and trimming folders automatically . So this might be considered a bug. However, I don't see old messages that were previously deleted being re-downloaded as mentioned in comment 0.

(In reply to Ben from comment #20)

I couldn't find details on when Thunderbird triggers the retention policy cleaning.

According to https://bugzilla.mozilla.org/show_bug.cgi?id=637352#c3, the purge (service) interval is 5 minutes, but at most purge for any given folder happens every 8 hours.

Hi! Just to confirm that this still happens. With a number of mailboxes of various accounts. One is an inbox of 17000 messages with perhaps 300 messages per day that makes Thunderbird hang for more than one minute every time I open it. The retention policy never clears out old messages; neither with a number limit nor with a days limit. It is on an IMAP server that is shared among a handful of users.

Thunderbird 115.7.0 (64-bit) on Archlinux.

Ben, does this reproduce with 128, with the purge amount

Note also, that the reporter mentioned shared folders. That might be why he reported a problem and I don't see the same problem with a normal folder.

Unsure whether that would be a factor. But that doesn't seem to be the case for Ben.

(In reply to Wayne Mery (:wsmwk) from comment #21)

According to https://bugzilla.mozilla.org/show_bug.cgi?id=637352#c3, the purge (service) interval is 5 minutes, but at most purge for any given folder happens every 8 hours.

Thank you!

Hi! Just to confirm that this still happens. With a number of mailboxes of various accounts. One is an inbox of 17000 messages with perhaps 300 messages per day that makes Thunderbird hang for more than one minute every time I open it. The retention policy never clears out old messages; neither with a number limit nor with a days limit. It is on an IMAP server that is shared among a handful of users.

Thunderbird 115.7.0 (64-bit) on Archlinux.

Ben, does this reproduce with 128, with the purge amount

No. It seems to purge correctly now on Thunderbird 128.4.4.

You need to log in before you can comment on or make changes to this bug.