618809 - Unread deleted mail can not be restored from the trash folder with mail.server.default.dup.action = 2 (move duplicated mails to trash)

Status: RESOLVED WONTFIX

(MailNews Core :: Networking: IMAP, defect)

Description

[patrick.abiven]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.0.19) Gecko/2010031422 Firefox/3.0.19 ( .NET CLR 3.5.30729; .NET4.0E)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7

When an unread mail is moved to the trash folder (by antispam, by manual action, ...), an attempt to restore this mail into the inbox folder fails. The mail is automatically moved back to the trash folder.

Reproducible: Always

Steps to Reproduce:
1. Manual action to move an unread just received mail from inbox to trash folder.
2. Manual action to restore (move back) the mail from trash to inbox folder.
3. Manual action to view the content of the inbox folder

Actual Results:  
The restored mail in automatically moved back into the trash folder.

Expected Results:  
The restored mail should stay in the inbox folder.

Activating imap log: we see an expected call to OnlineMessageCopy () which move back the restored mail to the trash folder.

0[182a140]: queuing url:imap://<myuser>@<mydomain>:993/onlinemove>UID>/INBOX>722>/Corbeille  
(Corbeille is the name of the trash folder for french)

Comment 1

[patrick.abiven]

Another way to reproduce this behavior:
1. Manual action to move an unread just received mail from inbox to another
folder (not necessarly the trash folder).
2. Manual action to restore (move back) the mail from the other folder to inbox folder.
3. Manual action to view the content of the inbox folder

Our Imap server is :
version    : v2.2.12 2005/02/14 16:43:51
vendor     : Project Cyrus
support-url: http://asg.web.cmu.edu/cyrus
os         : Linux
os-version : 2.6.16.21-0.8-smp
environment: Built w/Cyrus SASL 2.1.21
             Running w/Cyrus SASL 2.1.21
             Built w/Sleepycat Software: Berkeley DB 4.3.29: (November 10, 2006)
             Running w/Sleepycat Software: Berkeley DB 4.3.29: (June 16, 2006)
             Built w/OpenSSL 0.9.8a 11 Oct 2005
             Running w/OpenSSL 0.9.8a 11 Oct 2005
             CMU Sieve 2.2
             DRAC
             TCP Wrappers
             mmap = shared
             lock = fcntl
             nonblock = fcntl
             auth = unix
             idle = idled

Comment 2

[patrick.abiven]

This behavior seems to be linked to the option mail.server.default.dup.action = 2 (move duplicated mails to trash). Reseting this option to its default value = 0 (keep duplicated mails) resolves this issue.

Comment 3

[Ludovic Hirlimann [:Usul]]

Do you see the issue if TB is started in -safe-mode (http://support.mozillamessaging.com/en-US/kb/Safe+Mode) ?

Comment 4

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #2)
> option mail.server.default.dup.action = 2 (move duplicated mails to trash).
> Reseting this option to its default value = 0 (keep duplicated mails) resolves this issue.

If so, your comment #0 is report that Tb works very well as designed, isn't it?

Comment 5

[patrick.abiven]

In fact no. There is an issue when moving a mail out of a folder, it still stays in the hashTable of the folder, and when the mail is moved back to the original folder, the check against the hashTable incorrectly detects a duplicate. 
When moving a mail out of a folder, it should be removed from the hashTable of this folder.

Comment 6

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #5)
> There is an issue when moving a mail out of a folder,
> it still stays in the hashTable of the folder, 
> and when the mail is moved back to the original folder,
> the check against the hashTable incorrectly detects a duplicate.
> When moving a mail out of a folder, it should be removed from the hashTable of this folder.

If I understand Tb's behaviour correctly, Tb removes entry from .msf(MsgDB) when mail is deleted from a folder(==store +flag \Deleted is issued), when IMAP delete model of "Move to trash" or "Remove immediately".
What IMAP delete model do you use? "Just mark it as deleted"?
Is hashTable you reffer is a table for duplication check by mail.server.default.dup.action?

Comment 7

[David :Bienvenu]

that hashtable would be nsIMsgIncomingServer::m_downloadedHdrs, and you're right that moving a message out of a folder doesn't get reflected in that hash table. But, it might be simpler just to add a check that there's still a duplicate header (e.g., with the same message id) in the folder.

Comment 8

[WADA:World Anti-bad-Duping Agency]

It sounds similar situation to "Just mark it as deleted" for duplication check. What will happen if "Compact of Inbox" is executed?

Comment 9

[Wayne Mery (:wsmwk)]

WADA, bienvenu, is this a _confirmed_ issue?

Comment 10

[David :Bienvenu]

(In reply to comment #9)
> WADA, bienvenu, is this a _confirmed_ issue?

I didn't reproduce the problem, but it would be expected behavior from looking at the code (and is a bug)

Comment 11

[Ludovic Hirlimann [:Usul]]

Removing myslef on all the bugs I'm cced on. Please NI me if you need something on MailNews Core bugs from me.

Comment 12

[gene smith]

Description of duplicate detection and bug where it was added: bug 9413 comment 37

Comment 13

[gene smith]

I can duplicate this bug but with a somewhat different STR than is described in comment 0 and comment 1:

1. Delete to trash a never before deleted message in Inbox
2. Go to trash and move it back to Inbox.
3. Go to Inbox and see that the message is back.
4. Delete the same message again.
5. Go to trash and again move it back to Inbox
6. The message disappears from trash but shortly comes right back.
7. Go to Inbox and notice that the message is not there.

So this only seems to be a problem when the same message is deleted to trash two or more times. Only in that case will the hash table detect a duplicate and move the message back to trash when dup_action is 2. Note also that if dup_action is set to 1, only after the 2nd delete to trash and move back to inbox will the message be marked \deleted as was observer in bug 1580060, so that bug is actually a duplicate of this bug.

Comment 15

[gene smith]

(In reply to David :Bienvenu from comment #7)

that hashtable would be nsIMsgIncomingServer::m_downloadedHdrs, and you're
right that moving a message out of a folder doesn't get reflected in that
hash table. But, it might be simpler just to add a check that there's still
a duplicate header (e.g., with the same message id) in the folder.

Not sure what this is suggesting. Get rid of hash table and scan all the messages in folder looking for a matching message-id (and subject?) and if found, do dup_action on the highest "duplicate" UID?

Comment 16

[gene smith]

Ok, I've looked at the code and can't find an easy fix for this, so I will just suggest a workaround and mark this as WONTFIX.

If dup_action is set to 1 (mark dup as deleted) or 2 (move dup to Trash) and when you try to move a message that was previously deleted or moved from Inbox in the same session back to Inbox, if set to 1, it disappears in Inbox or, if set to 2, it moves right back to Trash.

If dup_action is set to 1, you will need to temporarily set the delete method to "just mark it as deleted" and re-select Inbox to see the deleted messages with red X and crossed out, then un-delete the messages. (Don't compact Inbox or the messages marked deleted will be permanently deleted!)

If dup_action is set to 2, you need to restart Thunderbird and then you can move the messages from Trash back to Inbox. Restarting TB will reset the hash table so the messages are not immediately detected as duplicates and can be moved back to Inbox and stay there.

If dup_action is set to 3 (mark dup as read) there is probably no issue here since the message should still be visible when moved back to Inbox.

Since dup_action is a hidden pref that "voids your warranty" when set to something other than 0 (allow duplicates), I think WONTFIX is OK unless at some future time the dup_action setting is incorporated into the formal UI other than config editor.

Note: The mozillazine documentation for mail.server.default.dup_action describes it as a boolean and contains no useful information:
http://kb.mozillazine.org/Mail_and_news_settings