156168 - IMAP: messages not really deleted with delete immediately delete model

Status: RESOLVED FIXED

(MailNews Core :: Networking: IMAP, defect)

Description

[Gerard Gachelin]

I am using IMAP (WU-IMAP).
Everything seems working fine BUT when I delete emails and empty trash, through
the Mozilla mailer, emails emails are not visible but they are still in my mbox
in /var/mail.

Comment 1

[Eleanor Blair]

We've also seen this problem with imap folders.  Using the "compact folder"
command from the file menu makes the messages actually be deleted from the server.

This has caused us problems because at least one of our users was well over
quota despite having apparently deleted all his messages.  In particular if
users are using the Trash rather than just deleting messages immediately they
can move the message to the trash then empty the trash and now both Trash and
the original folder still contain copies of the message in question so it is
taking up even more space!

I presume it is intended to be a feature to save time on deletes, but it is
definitely very unclear as a user that the messages are still there, and unclear
what "compact folder" actually means.

Comment 2

[Gregg Meehan]

Confirming, marking as NEW

Comment 3

[Wolfgang Rosenauer [:wolfiR]]

this is also true for Cyrus-IMAPD, so the summary should be updated?
I've tried with 1.4a.
In the Server Settings I have set:
"When I delete a message: Remove it immediately"

Mozilla doesn't show this mail any longer, but it's still available through other
clients and is marked as deleted.
A "compact folder" will delete them but this is the behaviour I expect when I set
"mark it as deleted".

Comment 4

[Wolfgang Rosenauer [:wolfiR]]

after some thoughts, I think it's not easy to interpret the IMAP protocol in
that case. I think if "remove it immediately" is set (delete_model == 2),
after every delete command (mark) an "expunge" on every sub-folder should be
performed. At least this should be done when mailnews is exited.
It seems that the code is in mailnews/imap/src/nsImapMailFolder.cpp ?
Then nsImapMailFolder::CompactAll should be called automatically. But I couldn't
find the place where it has to be :-(

Comment 5

[Thomas Dodd]

Anyone know how other client handle this?

I've only played with Outlook Express, and Evolution a little, and don't
remember the details. Seams that OE, marks as deleted and draws a line through
the subject, but the message remains until you use the "Purge" button. Not sure
what moving to another folder does (like the Mozilla "Delete = move to Trash"
would do).

If several other mail clients have the same default behavior, Mozilla should
behave the same way by default. I think Mozilla should have an option to auto
compact on move/delete though, perhaps with a warning that it causes slow
performance. Also, the old "compact if it will save X space option" could
return. (I think mozilla had it. I saw it some where before)

Comment 6

[Wolfgang Rosenauer [:wolfiR]]

currently the situation is, that three "delete models" are available:

1. Move it to the trash folder
I've not tested it well yet. But I think this is handled correctly.

2. Mark it as deleted
This is the default IMAP behaviour. You can only mark messages as deleted
and later you can expunge/compact/purge your folder to remove them.
The only issue here is, that the "Compact" option is not well reachable and there
is no option to compact ALL folders (this is not really needed IMHO)

3. Remove it immediately
The current behaviour is, that Mozilla mark it as deleted and remove it from its
own database IMHO. But this is the very same behaviour as point 2. 
This is the bug here I think.
The messages should be physically deleted at least on exit of messenger.

Comment 7

[Thomas Dodd]

> 1. Move it to the trash folder

Copies to "Trash", marks original as deleted. Original not purged.
> 2. Mark it as deleted
> The only issue here is, that the "Compact" option is not well reachable
> and there is no option to compact ALL folders (this is not really needed IMHO)

There is a "Compact Folders" in the File Menu, which I would expect to be
compact all, but it's not.

It is needed though. If I delete messages form several folders, I must remember
to compact each. "Compact All" would let me do it once when I'm done.


> 3. Remove it immediately

It's just #1 without the copy operation.

Other problems though. My Inbox is filtered, and copies placed in the correct
folder. The Inbox copy is marked deleted, but remains. So the 1000 messages I
get form mailing lists over night are stored twice untill I come in and compact
my Inbox. Over the weekend this leads to a large Inbox. The sysadmin is often
not pleased. Same for Junk mail being moved.

Once the copy is made, the original should be really deleted.

Comment 8

[David :Bienvenu]

we compact the folder when there are more than 20 deleted messages if you are
using the standard delete to trash delete model, or the delete immediately
model. When you empty the trash, we delete all the messages and expunge the
folder. I'm not sure I see an issue here. There is also an option to expunge the
inbox on shutdown, as well as empty the trash.

If you are using the imap delete model, we don't automatically expunge folders,
because choosing the imap delete model implicitly means you want to handle when
deleted messages are really deleted.

Comment 9

[Wolfgang Rosenauer [:wolfiR]]

I didn't know that deleted messages are expunged when there are 20 messages 
deleted.
This sound better than I've thought.
But the "Expunge INBOX on exit" thing is not optimal I think. Why only the INBOX?
I think it would be better to delete all subfolders, too?

Comment 10

[David :Bienvenu]

I should say that expunging only happens when you click on a folder with 20
deleted messages, e.g., click on an other folder and then click back on the
first folder. 

We expunge the inbox because we know we have it open(we cache a connection to
the inbox) and we know it's likely to have deleted messages. Other folders we
leave up to the user. Expunging every folder on shutdown would be expensive.

Comment 11

[David :Bienvenu]

can I close this since we do compact folders on open when a certain number of 
messages have been deleted? this works for most users, and if you really want 
the messages to be expunged, you can compact the folder.

Comment 12

[Wolfgang Rosenauer [:wolfiR]]

I think it's not perfect but you can close it. (at least IMHO)

Comment 13

[David :Bienvenu]

thx, closing wfm

Comment 14

[Gerard Gachelin]

You can close this bug

GG

Comment 15

[Wolfgang Rosenauer [:wolfiR]]

I discovered this some time. I'm still using delete model "Remove it immediately".
But I do never see that the messages are deleted. Maybe I understood the
description how it should work incorrectly?
I understood that the messages are expunged automatically if there are more than 20.
But my messages are still physically there until I remove them manually via
"Compact folder".

Comment 16

[David :Bienvenu]

the messages are removed if you select a folder that has more than 20 deleted
messages - they're not removed after you delete the 20th message, but just when
you open the folder again, iirc.

Comment 17

[Wolfgang Rosenauer [:wolfiR]]

then this feature seems to be buggy. As I wrote the marked messages are never 
deleted. Not immediately and not after reopening the folder.
I never get rid of those messages without manual "compact".

Comment 18

[Wolfgang Rosenauer [:wolfiR]]

please reopen this bug. One of my folders has over 450 "marked as deleted"
messages and I use "remove immediately", but the messages are never expunged.

Comment 19

[Wolfgang Rosenauer [:wolfiR]]

I assume the deletion should be done in nsImapProtocol.cpp:
if ((m_flagState->GetNumberOfDeletedMessages() >= 20/* gExpungeThreshold */)
                 && GetDeleteIsMoveToTrash() && m_imapAction !=
nsIImapUrl::nsImapLiteSelectFolder)
          Expunge();  // might be expensive, test for user cancel

Perhaps GetDeleteIsMoveToTrash() is not true in my case?
I will try to track it down.

Comment 20

[David :Bienvenu]

you're right - that check is wrong - it should be checking that we're not using
the imap delete model, not that delete is move to trash.

Comment 21

[David :Bienvenu]

patch upcoming.

Comment 22

[David :Bienvenu]

Attachment: proposed fix

do the expunge if the user isn't showing deleted messages.

Comment 23

[David :Bienvenu]

fix checked into 1.6 trunk, r/sr=mscott

Comment 24

[Wolfgang Rosenauer [:wolfiR]]

would it be possible to fix this in 1.5, too?

Comment 25

[David :Bienvenu]

Not 1.5 - that ships today, I believe. But perhaps in a 1.5.x build. When a flag
is added for that, I'll try to remember to ask for approval.