Closed Bug 1777953 Opened 2 years ago Closed 2 years ago

"Empty Trash on Exit" option does not always trigger with quarantine enabled (pop3 and Local Folders)

Categories

(Thunderbird :: Account Manager, defect, P4)

Thunderbird 102

Tracking

(thunderbird_esr102 fixed)

RESOLVED FIXED
104 Branch
Tracking Status
thunderbird_esr102 --- fixed

People

(Reporter: drfreund, Unassigned)

References

(Blocks 1 open bug)

Details

(Keywords: regression, Whiteboard: [fixed by bug 1777454][102.0.3])

Attachments

(1 file)

Steps to reproduce:

Configured junkmails to be stored directly in the trash bin.
Configured trash bin to be emptied on exit of Thunderbird.

Actual results:

Junkmails are properly stored in the trash bin, but are still there after a restart of Thunderbird. Also regular deleted mails sometimes remain in the trash bin. Usually, mails are finally deleted not before several restarts of Thunderbird. This is relevant for security, because junkmails with malware attachments remain in the trash bin and are not deleted immediately and permanently, as expected.
Furthermore, the number of mails behind the trash bin folder sometimes misses and only appears after clicking on the folder.

Expected results:

If configured this way, the trash bin should bei cleared after every restart of Thunderbird, as it was in Versions before 102. The number behind the trash bin folder should alway reflect the actual count of mails in the folder.

I'm not able to reproduce this, but in any case it's not a security bug. I agree having malware in the trash could be bad, but it's not going to be any worse there than it was in your Inbox or Spam, and in any case it's not something that an attacker can exploit to achieve any kind of unauthorized outcome. So there's no need for the security flag.

If you have the time to check, can you add one of your mail accounts to a new profile and see if the bug still occurs there? Thanks!

Group: mail-core-security
Component: Untriaged → Account Manager
Flags: needinfo?(drfreund)
Priority: -- → P4
Summary: Trash bin is not cleared on exit → "Empty Trash on Exit" option does not always trigger
See Also: → 1777878

I'm not able to reproduce this

That's the answer, I expected, because as a developer I usually tell the same to my customers ;-)
You should take a closer look at the following fix, which came with version 102:
"Empty Trash on Exit did not work with IMAP accounts using OAuth2 authentication".
The current behaviour could be an unwanted sideeffect. I have a POP3 account.

So there's no need for the security flag.

May be that an attacker can not use it. But if I want something to be deleted, it should really be deleted, not only a little bit. If some information remains where it is not expected, this may be regarded as a security issue.

If you have the time to check, can you add one of your mail accounts to a new profile and see if the bug still occurs there? Thanks!

No need to do that. It worked until last version of Thunderbird and it doesn't work anymore since I updated to 102. It doesn't help me, if it works again with a brand new profile and account, where I lose my data and configuration.

Flags: needinfo?(drfreund)
Blocks: tb102found
Attached image Error.png

With Version 102.0.2 still the problem exists, when SPAM should be automatically moved to the Trash Folder the attached error message does occur (sometimes, not always).

Keywords: regression

Looks like this is caused by a missing MSF which leads to:
The messages could not be filtered to folder 'XXX' because adding a message to it failed. Verify that the folder is displaying properly or try to repair it from the folder properties.

Yes, this is the english version of the error message:
"The messages could not be filtered to folder 'XXX' because adding a message to it failed. Verify that the folder is displaying properly or try to repair it from the folder properties."
But the Trash folder (where the SPAM messages should be moved) exists. In 90% of the SPAM messages the automatically move to the Trash folder works fine. In 10% of the SPAM messages this error message does occur. Maybe the problem does occur, when many SPAM messages should be moved at the same time.

As an experiment, I manually emptied the Trash folder, then deleted a single, non-spam email. I exited TBird. When I restarted TBird, the deleted item was still in Trash. The problem I reported has nothing to do with spam. There are messages in the Trash folder. In Settings there is a checked box "Empty trash on exit," but Trash isn't emptying on exit. It's not a security issue, it's a failed-execution issue.

Here's an alternate solution: If it can't be made to work properly, get rid of the "Empty trash on exit" check box.

Reinhard--no offense meant. In addition to your report, I reported this as an "Empty trash on exit" bug. It was marked, IMO mismarked, as a dupe of your bug. My bug has nothing to do with spam, but because you mentioned spam, the response to your report has been focussed, IMO misfocused) on spam, with the implication that this was no big deal. That irritated me. I apologize.

From bug 1745205 comment 29, and as noted in comment 6, please try the following - "As for emptying the a local trash for a POP3 account. Please repair, compact and repair the Trash folder, or alternatively, delete it manually from the profile. There doesn't appear to be a reason for emptying a local trash not working other than a broken folder."

Deleted the trash folder, restarted TBird, Trash folder recreated, deleted a message, exited TBird, restarted TBird. Message still in Trash. I have five similar installs of TBird on five PCs. Each exhibits the same behavior since upgrading to v102. Seems unlikely that all five Trash folders would be similarly broken.

My bug has nothing to do with spam

You are right. My initial description may have been confusing, because the bug occurs with every deleted mail, no matter how it reached the trash bin. But I observed it first with spam mails.

Deleted the trash folder, restarted TBird

Same result for me. Deleting/Compacting/Repairing trash folder doesn't help. In addition I remember, that the trash folder disappeared soon after installation of version 102 and had to be recreated.

I've tested "Empty Trash on Exit" for an IMAP account and it still works. It should even work better now for certain servers, see bug 1745205.

It's possible that you suffer from bug 1773511 which was about special folders (Drafts, Sent, and likely also Trash) were messed up, in fact, they all ended up pointing to the corresponding folders in "Local Folders". So please check your account settings to see which (trash) folder is actually configured as the account's trash folder. That is the one that will be emptied.

Also, if the issue persists after that, please report your IMAP provider or state that this is about a POP3 account with a local trash folder. As far as I can see, no one stated so far whether this is about an IMAP or POP3 account (and sadly the bug also got side-tracked by some unrelated junk filtering issue).

Tested on a POP3 with local Trash folder now. I needed to repair the trash folder for "Empty Trash on Exit" to work. Looks like there is no configuration of the trash folder for POP3 accounts, so bug 1773511 doesn't apply there. One thing you should check is whether the Trash folder still has the trash folder flag. Does it show with a trash can icon? The add-on FolderFlags (https://addons.thunderbird.net/en-GB/thunderbird/addon/folderflags/) lets you check.

no one stated so far whether this is about an IMAP or POP3 account

It's about a POP3 account. I mentioned that in a subclause, because it was not clear, if it is relevant.

I needed to repair the trash folder for "Empty Trash on Exit" to work.

That does definitely not work for me. I repaired/compressed/deleted the trash folder several times, but no change in behaviour.

Does it show with a trash can icon?

Yes.

IMAP and POP3 are completely different.

I've tested this a bit more for POP3 and see this:

  1. If I move a message to the trash and exit, the message is not deleted.
  2. If I move a message to the trash, then repair the trash folder, then exit, the message is deleted.

So the wrong behaviour is quite reproducible, unless the trash folder is repaired immediately before exiting.

Flags: needinfo?(gds)
Flags: needinfo?(benc)

Alice, can you reproduce this on a POP3 account? Set the trash to empty on exit and keep copying messaged (Hold shift + drag) into the trash to see whether the trash is emptied upon exit.

Flags: needinfo?(alice0775)

POP3. Per Newsfan's suggestion, I added FolderFlags. Trash flag is set on Trash folder. I got excited because immediately after adding FolderFlag, I closed and restarted TBird, and the Trash folder emptied. However the bug returned on subsequent closings/restartings.

newsfan, It works for me with self-built trunk (hg pull -u yesterday evening). This is on a POP3 account with normal password (not oauth2). I haven't had to repair the Trash folder which is under the POP3 account. I tried it several times and on tb restart the Trash folder always became empty.

Thanks for testing. Try a 102 build, there's a new one on https://treeherder.mozilla.org/jobs?repo=comm-esr102. I'm using a stock-standard POP3 account, no OAuth2. As others report here, it's very unreliable, mostly not working. Maybe Alice can find a regression, or also when it got fixed.

I have my POP3 account configured as mbox, the default. Any chance users seeing the problem are configured to maildir (one file per message)?
Anyhow, tried a few more times and still no problem emptying Trash on exit. Haven't tried converting the account to maildir.
...
Ok, collided with your response. So I assume you're not using maildir since stock-std POP3. Anyhow, I'll try 102.

Flags: needinfo?(gds)

It's mbox (not a maildir friend, don't know what the default is these days, old profile).

Looks good for me on 102.0.1 and 102.0.2. Does it matter how many messages are in Trash? I've only tried it with 3 or 4 so far.
Just tried with 200 messages in Trash and it still emptied on exit. (The Trash mbox file goes to zero size and on restart Trash folder is has zero messages.) No repairs have been needed.

I'm using POP3 on 102.0.2. Problem's still there.

(In reply to gkarasik from comment #13)

Deleted the trash folder, restarted TBird, Trash folder recreated, deleted a message, exited TBird, restarted TBird. Message still in Trash. I have five similar installs of TBird on five PCs. Each exhibits the same behavior since upgrading to v102. Seems unlikely that all five Trash folders would be similarly broken.

How did you "delete the trash folder"? It doesn't appear that there is a "delete folder" selection on right-click for the Trash folder. For my setup the Trash folder has a "trash can" icon which makes it un-deleteable. Is there an add-on involved to set the folder flags so Trash folder behaves like a user-created folder?

Maybe users seeing this issue have a conflicting addon and need to run in "safe mode"? (Help | Troubleshooting Mode ...)

(In reply to gene smith from comment #26)

Looks good for me on 102.0.1 and 102.0.2. Does it matter how many messages are in Trash? I've only tried it with 3 or 4 so far.

I've tried with 1 or 2. Actually, I've just retested that again, so here's the new finding:

  1. Trash folder is emptied on exit if TB is run online.
  2. Trash folder is NOT emptied on exit if TB is started offline (in the profile manager).

Apologies for not documenting the steps correctly. I've run TB offline to avoid messages being downloaded via POP3 and that triggers the issue. Not sure whether it was like this in 91. Do the reporters here use offline mode? Maybe related to the new POP3 module? Reporters, what happens if you switch the preference mailnews.pop3.jsmodule to false. Does it then reliably empty the trash? There may be a POP3 connection since it might also delete messages from the server when they are deleted from the trash.

(Let's cancel the regression search for now.)

Flags: needinfo?(benc)
Flags: needinfo?(alice0775)

Safe mode makes no difference. Using TBird online. One completely off-the-wall observation: Trash folder is successfully emptied on second close/restart providing no new message is added. If a new message is added, all messages, new and old, remain in Trash folder. Please someone else try this and tell me I'm not crazy. Also, someone asked about how I deleted the Trash folder: I deleted the two Trash files from the profile Mail directory. Both were recreated on TBird restart.

In online mode, with pref mailnews.pop3.jsmodule at true (default), adding a message to the trash and then exiting results in an empty trash when restarting. No second restart required. As Gene said, you can observe the file size of the Trash file on disk. It's even compacted, so it drops down to 0 bytes. However, I'm running TB 102.0.3 which has a fix for bug 1777454. Maybe that is related. Can you try a candidate build from here: https://archive.mozilla.org/pub/thunderbird/candidates/102.0.3-candidates/build1/.

If that still doesn't work, please set pref mailnews.pop3.jsmodule to false. If that also doesn't work, you can do some POP3 logging.

Working as intended in TB 102.0.3. Thank you all for your help.

Whiteboard: [seems working in TB 102.0.3]

From newsfan comment 29:

Trash folder is NOT emptied on exit if TB is started offline (in the profile manager).

Yes, I see this too with tb set offline. Don't know if that's a feature or a bug.

Anyhow, with 102.0.2 I still don't see a problem with online operation (POP3 Trash is emptied on exit).
Thought maybe if I enable "anti-virus quarantining" under Settings->Privacy & Security->Antivirus I might see something since that's what bug 1777454 mainly addressed but for another issue (new messages showed blank in Inbox).

From gkarasik comment 32:

Working as intended in TB 102.0.3. Thank you all for your help.

Don't know why but I'll ask if those seeing this bug with <=102.0.2 have "anti-virus quarantining" enabled under Settings->Privacy & Security->Antivirus ??

I tested unchecking "Anti-virus quarantining" in TB 102.0.2. As you surmise, it results in TB working as intended, i.e., the Trash folder empties on closing TB 102.0.2.

(In reply to gkarasik from comment #34)

I tested unchecking "Anti-virus quarantining" in TB 102.0.2. As you surmise, it results in TB working as intended, i.e., the Trash folder empties on closing TB 102.0.2.

Very interesting! But I can't seem to get 102.0.2 to fail with "anti-virus quarantining" checked or not checked.

Be great if we could figure out what's different. I'm running 32bit Win7 Ent SP1. AV is Trend Micro.

Ok, I finally duplicated the problem. But I had to boot into Windows and saw it with the previously installed 102rc1. Only see Trash fail to empty on a tb shutdown with Anti-virus quarantine switched on. Also, as reported by gkarasik, it does empty on the 2nd shutdown.
Don't know why it seems to work as expected with linux with <= 102.0.2.
Don't know what it would do with OSX

.(In reply to gkarasik from comment #36)

Be great if we could figure out what's different. I'm running 32bit Win7 Ent SP1. AV is Trend Micro.

TB's AV Quarantine really doesn't hook into the AV program (Trend, Norton or whatever) so the type of AV, at least for this issue, probably doesn't matter. But I'll leave that to the experts...

Glad you were able to duplicate it. (Misery loves company.) Anyway, as it's solved with 102.0.3: Onward!

(In reply to gkarasik from comment #38)

Glad you were able to duplicate it. (Misery loves company.) Anyway, as it's solved with 102.0.3: Onward!

Ok, thanks for your work in troubleshooting this. Before closing this, would be good to get a confirmation from the original reporter Dr.RF
Also, maybe a confirmation of 102.0.3 on OSX would be good (I think Wayne has a Mac now and maybe a random POP3 account).

Flags: needinfo?(drfreund)

(In reply to Dr. Reinhard Freund from comment #0)

Furthermore, the number of mails behind the trash bin folder sometimes misses and only appears after clicking on the folder.

The number behind the trash bin folder should alway reflect the actual count of mails in the folder.

Dr. Freund, I'm not sure which number(s) you are referring to here that is/are wrong. The numbers I see seem to be correct. If you still see this problem after testing with the new 102.0.3 version to fix the trash emptying problem, probably it should be submitted in a new bug report.
Thanks!

Wow! So much traffic ...
I will try to answer all questions:

First all data about my system:

  • Windows 11 x64 with all updates
  • Anti-Virus: only Windows Defender, "anti-virus quarantaining" enabled in Thunderbird.

Thunderbird x64:

  • Current version: 102.0.2
  • Bug occured first in initial 102.
  • Account: POP3, STARTTLS, normal password
  • mailnews.pop3.jsmodule = true
  • Local folders: mbox (one file per folder)
  • online-mode

If a new message is added, all messages, new and old, remain in Trash folder.

That's also my observation. Mails remain in trash folder as long as new mails are added. If you restart Thunderbird without having send a mail to trash folder, it is empty again.

I'm not sure which number(s) you are referring

If you configure the folder-tree to display all columns, you will see the number of stored mails behind each folder and the byte-size of each folder. If a folder contains unread mails, there is also a fat number in parentheses. All numbers are initially missing for the trash folder and appear not until clicking on it.

Working as intended in TB 102.0.3.

I will test 102.0.3 as soon, as it is officially released. I don't install betas in my productive system.

I hope, I didn't forget anything important ...

Flags: needinfo?(drfreund)

(In reply to Dr. Reinhard Freund from comment #41)

I will test 102.0.3 as soon, as it is officially released. I don't install betas in my productive system.

This is the official release candidate which will likely be released today or tomorrow, not a beta version.

Flags: needinfo?(drfreund)
Summary: "Empty Trash on Exit" option does not always trigger → "Empty Trash on Exit" option does not always trigger with quarantine enabled (pop)

From comment 41 by reporter Dr.RF:

If you configure the folder-tree to display all columns, you will see the number of stored mails behind each folder and the byte-size of each folder. If a folder contains unread mails, there is also a fat number in parentheses. All numbers are initially missing for the trash folder and appear not until clicking on it.

I'm not seeing this. For me on tb startup on linux all the numbers beside the folder names appear correctly. I don't have to click on Trash (or other folders) to see the numbers and they are correct. Anyhow, maybe I'm missing something or maybe this is also Windows specific. Please enter a new bug report if you still see this when you test 102.0.3 when it's released.

Also, the "fat [bold] number in parentheses" only appears if you don't display the unread number column. Just an observation.

Since POP3 and Local Folders are handled by the same or similar TB code, I tested Local Folders on windows and its Trash also fails to empty on shutdown. So I added "Local Folders" to the bug summary line. I didn't try it but I expect this to also be fixed in 102.0.3.

See Also: 17778781777454
Summary: "Empty Trash on Exit" option does not always trigger with quarantine enabled (pop) → "Empty Trash on Exit" option does not always trigger with quarantine enabled (pop3 and Local Folders)

Good News! I just updated to Thunderbird 102.0.3 and the bug seems to be fixed. The trash folder is emptied immediately after restart, as it should be.

Flags: needinfo?(drfreund)

Thanks.

Status: UNCONFIRMED → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Whiteboard: [seems working in TB 102.0.3] → [fixed by bug 1777454][102.0.3]
Target Milestone: --- → 104 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: