Closed Bug 990551 Opened 11 years ago Closed 6 years ago

IMAP Trash emails deleted despite Retention Policy of Never Delete

Categories

(Thunderbird :: Untriaged, defect)

Product:
Thunderbird
Component:
Untriaged
Version:
26 Branch
Platform:
x86_64
Linux
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED INVALID

People

(Reporter: tms2, Unassigned)

References

Details

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:26.0) Gecko/20100101 Firefox/26.0 (Beta/Release) Build ID: 20131206145537 Steps to reproduce: 1) Set Trash folder Retention Policy to Never Delete. 2) Delete emails in the course of my regular work. Actual results: Nothing older than a week or two is retained in Trash. Expected results: I should have months/years of deleted emails in Trash.
scrognoid, that's an odd way of handling trash. I should think you experienced slow delete performance as a result. Anyway, does this reproduce in version 25 beta? You can get it from https://ftp.mozilla.org/pub/mozilla.org/thunderbird/releases/25.0b1/linux-x86_64/en-US/
Flags: needinfo?(tms2)
Summary: IMAP Trash emails permanently deleted with Retention Policy Never Delete → IMAP Trash emails deleted despite Retention Policy of Never Delete
Whiteboard: [regression:TB26?]
I am currently useing v31.0 in Linux Mint. I have had this problem for years, across numerous versions in both Windows and Linux. I have recreated my profile several times over the years to no effect. It is also the case that I cannot send trash to an alternate folder. I tried that a couple of times over the years and each time deleted emails simply vanished. I don't know where they went but it was not to Trash or the alternate folder. I could not change back to having them go to Trash. They continued to just vanish. The only fix was to clear core recreating my profile. I never had a performance problem when it worked. Would keeping several months of Trash be any different from keeping several months of Sent?
Flags: needinfo?(tms2)
(In reply to scrognoid from comment #2) > I am currently useing v31.0 in Linux Mint. I have had this problem for > years, across numerous versions in both Windows and Linux. I have recreated > my profile several times over the years to no effect. It is also the case > that I cannot send trash to an alternate folder. Are you using a non-English locale of Thunderbird?
Flags: needinfo?(tms2)
No. I have added no language extensions.
scrognoid, Do you still see this when using version 52 or newer?
Whiteboard: [regression:TB26?] → [closeme 2018-03-15]
Yes, but... I have three accounts. My most active account has Trash emails no more than a week old. My two less active accounts have stuff going back a few months. I am currently at v52.6.0. I did not expect to hear about this again, so thanks for asking. I have long since worked around it by copying everything to a Trash2 folder using a filter. I never look in Trash anymore so I had not noticed that some deleted emails were lasting longer.
Flags: needinfo?(tms2)
Whiteboard: [closeme 2018-03-15]

(In reply to scrognoid from comment #6)

Yes, but...
I have three accounts. My most active account has Trash emails no more than a week old.

Is this account with a different mail provider than the other two accounts?

My two less active accounts have stuff going back a few months. I am currently at v52.6.0.

How is version 60?

Flags: needinfo?(tms2)

Same. My oldest email in Trash is just a couple of weeks old. I still have Retention Policy set to Do Not Delete Any Messages.

Flags: needinfo?(tms2)

Thanks for hanging in there.

As far as I can tell, the purge service pretty much hasn't changed since the feature was introduced in Bug 11055 - Message aging (empty trash/auto-delete messages after N days)

The purge service runs every 8 hours. If the purge service is deleting these messages there is a logging facility which will find the activity. Please create a log file with logmodule settting MsgPurge:5 as documented in https://wiki.mozilla.org/MailNews:Logging If messages have been deleted please check the logfile to see if you find the service has deleted messages that should not have been deleted. THen Zip and email me the logfile.

Some relevant code locations
https://hg.mozilla.org/comm-central/log/tip/mailnews/base/util/nsMsgDBFolder.cpp
https://hg.mozilla.org/comm-central/log/tip/mailnews/base/src/nsMsgPurgeService.cpp
https://hg.mozilla.org/comm-central/log/tip/mailnews/base/src/nsMsgPurgeService.h

Flags: needinfo?(tms2)

I'll need some more help to do that. I'm not a developer; just a user. Can you give me the commands and tell me how the are invoked relative to launching TB. The link says the log files are overwritten each time I launch TB. Since we are talking about a phenomenon that manifests over a period of days or weeks I'm worried about doing it write. Would I need to rename the log files each time before I launch TB?

Flags: needinfo?(tms2)

Scrognoid,

If you're on Linux you can add these lines to the bottom of your .profile file found in your home directory.

export MOZ_LOG=MsgPurge:5,timestamp
export MOZ_LOG_FILE=/tmp/purge.log

If your are worried about the file being deleted on reboot, you might consider changing the log file location 'MOZ_LOG_FILE' to something like /home/<yourname>/purge.log.

Did this for about a day's worth of normal activity. Let me know if you need more. purge.log is here:

https://www.dropbox.com/s/ri6h3kkducvtm1v/purge.log?dl=0

Does you mail provider delete messages from the trash?

Gene, does this sound familiar?

Flags: needinfo?(gds)

Sounds like Bug 1547043 that I am currently working. The reporter is away for a week and will check when he gets back using webmail if his ISP is removing the Trash emails. If not, tb must be doing it. He also has the auto purge settings turned off. He thinks it might be related to use of maildir offline storage.

Yes, my charter ISP server (Openwave) removes Trash after a week or two based on the Date header I think. Older emails put in Trash are deleted the next night I have noticed.

I think the reporter for this bug is saying that the right-click properties | retention setting for Trash is "use my account settings" and account setting is never delete?

Flags: needinfo?(gds)

I asked my ISP in 2014 and they claim they do not delete or archive Trash.
Yes, I set right-click properties on Trash to Never Delete but stuff gets deleted anyway.

Reporter scrognoid: Just to be certain are you saying that the Account setting / Sync and Storage / "Don't delete any messages" AND "Keep starred messages" are set for each problem account? And for the Trash folder properties it is set to "User my account settings"? These are the default settings Do you have any folder set to non-default retention settings? (I am also asking this of reporter Zurd of the other Bug 1547043 who I have CC'd.)

Also, scrognoid, you mentioned in comment 2 above that using an alternate trash folder still caused deleted emails to vanish. But later you say you use a filter to move emails to Trash2 folder and that works OK? How did you set up the "alternate" trash folder?

Also, scrognoid, can I assume you are using the default imap mail storage and are not using maildir format?

are you saying that the Account setting / Sync and Storage / "Don't delete any messages" AND "Keep starred messages" are set for each problem account?
Yes

And for the Trash folder properties it is set to "User my account settings"?
Yes.

Do you have any folder set to non-default retention settings?
Yes. Other folders have other settings, like Delete Older Than 180 Days, which are respected.

Also, scrognoid, you mentioned in comment 2 above that using an alternate trash folder still caused deleted emails to vanish. How did you set up the "alternate" trash folder?

  1. Create New Subfolder: TrashX
  2. Account Settings>Server Settings>When I Delete Move To: TrashX
    As commented, deleted emails did not go to TrashX or anywhere else that I could find.

But later you say you use a filter to move emails to Trash2 folder and that works OK?
Yes, message filters work.

Also, scrognoid, can I assume you are using the default imap mail storage and are not using maildir format?
I don't know. How do I tell?

(In reply to scrognoid from comment #17)

Also, scrognoid, you mentioned in comment 2 above that using an alternate trash folder still caused deleted emails to vanish. How did you set up the "alternate" trash folder?

  1. Create New Subfolder: TrashX
  2. Account Settings>Server Settings>When I Delete Move To: TrashX
    As commented, deleted emails did not go to TrashX or anywhere else that I could find.

Just tried doing this and it worked for me. Deleted email goes into TrashX instead of Trash. (TrashX now has the trash icon.)

Also, scrognoid, can I assume you are using the default imap mail storage and are not using maildir format?
I don't know. How do I tell?

Look in server setting for the account. Near the bottom you will see "Message store type" with selection "File per folder (mbox)" gray. You can only enable maildir for new accounts using the advanced preferences. Anyhow, if you don't know what it is I'm 99% sure you aren't using maildir.

Yes, confirmed:
"File per folder (mbox)" gray

Crazy thought, but the trash folder you reference, and the new trash folder you are trying to create. Are those both folders that would be synchronized with your email service provider? If so, could you point the deleted messages to store in your local folders outside of the email account?

I have four IMAP accounts setup in Thunderbird two of which I maintain myself. None of which exhibit the issue. Given that you have tested and seen the issue on both Linux and Windows, on various versions of TB, and have had the issue for years now. It seems that the only common denominator is your ISP.

Other than that you don't have your email account connected to multiple computers do you? If so maybe one of those machines has a 3rd party application that would clean out your trash bin?

Flags: needinfo?(tms2)

(In reply to gene smith from comment #14)

Sounds like Bug 1547043 that I am currently working. The reporter is away for a week and will check when he gets back using webmail if his ISP is removing the Trash emails.

The ISP of bug 1547043 fessed up and admitted that they indeed delete emails from Trash after denying it to the reporter several times, so that wasn't a TB problem. Maybe in the reporter's case the fact that the trash folder TrashX contains the sub-string "trash" so the ISP assumed it was trash and also deleted it. I have found that the support for large ISPs (e.g., charter/spectum) really have no idea how their 3rd party imap server works.

(In reply to Benjamin Flanagin from comment #20)

Other than that you don't have your email account connected to multiple computers do you? If so maybe one of those machines has a 3rd party application that would clean out your trash bin?

That's a possibility too.

Reporter, if not resolved, running an overnight IMAP:5 log may also show if emails are deleted better than the purge log of comment 11. Please run tb during a period when you notice emails in Trash or TrashX deleted. Edit out any personal info and attach using the "Attach file" link above or provide a dropbox link.

export MOZ_LOG=IMAP:5,timestamp
export MOZ_LOG_FILE=~/imap.log
thunderbird

It's hard to read thunderbird's log. For a very good and easy test, create a new email account, use it with webmail only, send a few emails to it, put them in the trash and wait. If they get deleted then it's the ISP deleting emails and not thunderbird.

(In reply to Zurd from comment #23)

It's hard to read thunderbird's log. For a very good and easy test, create a new email account, use it with webmail only, send a few emails to it, put them in the trash and wait. If they get deleted then it's the ISP deleting emails and not thunderbird.

Zurd, I was pretty sure looking at your log that tb wasn't marking anything in Trash as \deleted or expunging the folder. However, you're right, what you described would be a simpler and definitive test to try first. Thanks.

I have created a test email account and sent several emails to it. I accessed it only with webmail and deleted the messages, setting Preferences to delete by moving to trash and never purge. We'll wait and see what happens. Feel free to post a reminder in a month if you haven't heard from me. I'll probably forget about it.

Flags: needinfo?(tms2)

Nothing deleted from Trash yet.

I have 4 emails accounts with same ISP in TB. The are all set to Use My Account Settings>Never Delete. The oldest emails in Trash vary in age a bit:
1 - 12/22/18 (second most active account)
2 - 7/6/19 (most active account)
3 - 5/21/19 (seldom used)
4 - 7/17/19 (seldom used)

No change.

No change 8/10/19

(In reply to scrognoid from comment #28)

No change 8/10/19

scrog,
Does this mean tb is behaving OK. (Sorry, don't remember exactly what you are testing.)
-gene

Zurd: "For a very good and easy test, create a new email account, use it with webmail only, send a few emails to it, put them in the trash and wait. If they get deleted then it's the ISP deleting emails and not thunderbird."

gene smith: " Does this mean tb is behaving OK. (Sorry, don't remember exactly what you are testing.)"

No. If anything, it points the finger at tb. I have done as Zurd suggested. We are in the "wait" part. Nothing has been deleted from Trash. When we are done waiting, I could try accessing that test account with tb and see if emails get deleted from Trash. So far I am only accessing the test account with web mail.

Trash is empty. The problem must be with my ISP, not TB.
To recap:
Zurd suggested I run a test to verify that TB is deleting emails from Trash is violation of my Retention Policy.
I created an IMAP email account and only ever accessed it with web mail. I sent several emails to it and deleted them. Upon delete they were transferred to the Trash folder. I checked them once a week and the deleted emails were in Trash, until today. Emails were deleted from Trash even though TB never touched that account.
So this bug can be closed as far as I am concerned.

Thank you for the test scorgnoid,

Closing ticket as "Invalid" sense the problem wasn't in TB.

Status: UNCONFIRMED → RESOLVED
Closed: 6 years ago
Resolution: --- → INVALID
See Also: → 1392709

So....
I just noticed that there is an Advanced setting on my iphone that is set to remove Trash messages after two weeks. I never noticed it and I never set it so that must be the default. I use my iphone for email only occasionally and delete messages even less often.
I don't know if this caused my decade long issue, but it seems plausible. I don't know what happens when different devices are imposing conflicting settings on an IMAP server. But I do know that my messages in Trash got deleted after ABOUT two weeks, but there were odd variations. I could believe this was related to how often I actually checked email on my phone.

Does anyone know: What happens when I access an IMAP account with two devices and they have different settings for something like Deleted Messages After N Days?

Does anyone know: What happens when I access an IMAP account with two devices and they have different settings for something like Deleted Messages After N Days?

I would say if, for example, device A says delete trash after 14 days and device B says delete trash after 7 days, then depending on how the devices determine there own "time zero", you will see your trash gone after at most 7 days.
The devices don't tell the imap server to schedule deleting of trash every 7 (or 14 days) but the device itself tells the server to delete trash with a command it sends to the server which is sent by the device every, per my example, 7 (or 14) days.

You need to log in before you can comment on or make changes to this bug.