1399008 - can not move message from trash to Inbox of an IMAP account, after message dragged from Inbox to trash

Status: RESOLVED INCOMPLETE

(MailNews Core :: Networking: IMAP, defect)

Description

[rjakabosky+github]

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36

Steps to reproduce:

Configuartion of IMAP account : Server Settings / When I delete a message / Move it to this folder (is set to "Trash" folder, within this IMAP account)

1. Delete a message from Inbox.  (I hit the delete key too many times, so I needed to "undelete" one of the messages)
2. Go to Trash folder and drag message back to the Inbox.


Actual results:

The message I was trying to "undelete" only showed in the Inbox for about a second.  Then it went back to the Trash folder.

I even tried to undelete the message on the IMAP server using the roundcube webmail UI.  This only works until Thunderbird syncs with the IMAP server again.  As soon as Thunderbird syncs, the message is moved back to the Trash folder.

I think there might be a "delete" flag on the message that might not be cleared.


Expected results:

I expect it to stay in the Inbox.

Work around: Forward a copy of the message to myself.

Comment 1

[Jorg K (CEST = GMT+2)]

Unless you have a filter/rule that moves certain messages to the trash, I can't explain this behaviour.
Any ideas, Gene?

Also, next time you accidentally delete a message, use Undo.

Comment 2

[gene smith]

On 2 accounts I tried, after deleting an email, crtl-z (or Edit/Undo) brought the email back only after clicking "Get Messages". So I personally prefer to move the message back from Deleted to the original folder like the reporter described.

Otherwise, I don't see anything like what is reported. The original problem I had with charter imap (bug 1231592) was that I couldn't restore deleted or junk emails back into INBOX, which is similar to what is reported but not exactly. In this case the email seemed to completely vanish.

I would suggest that the reporter provide an imap log for the time from when the delete occurs until the emails goes back unexpectely to Deleted folder. Or better, if a temporary test account can be obtained from the reporter's imap provider that might help me to duplicate the problem.

Another thing the reporter can try as a temporary experiment is to reduce the number of cached connections down to 1 (from the default 5), restart tb and see if that helps (this helped some with the charter imap problem above). This is in Server Settings/Advanced/Maximum number of server connection to cache.

Reporter, the imap logging instruction are here: https://wiki.mozilla.org/MailNews:Logging

Comment 4

[rjakabosky+github]

Attachment: screenshot of IMAP packets between thunderbird and IMAP server

Comment 5

[rjakabosky+github]

Attachment: An annotated dump of the IMAP communication between Thunderbird and the IMAP server

I have attached a dump of the IMAP communication between Thunderbird and the IMAP server.

Comment 6

[rjakabosky+github]

There are no filter/rules for moving messages to the Trash.  Only 3 rules for moving messages into sub-folders (none for the trash folder).

I will try to reduce the number of cached connections to 1 and see if that helps.  I noticed in the packet dump that two TCP connections were used and each action(move to Inbox, move to Trash) was on a different connection.

Comment 7

[gene smith]

Thanks. I will check the logs when back on my device running tb.

Comment 8

[gene smith]

You're right, tb is somehow moving your message back to trash. It appears that some of the annotated data is repeated compared to what I see in the wireshark screen shot, but, if so, not a big deal if just a copy/paste issue.

Also, since your data doesn't show port numbers, I assume that the first half, where you copy back to inbox, is SELECTed with one tcp connection to trash mbox and the 2nd half, where tb copies the email back to trash, it is SELECTed with another connection to inbox. It might help to also see the trace for when you did the original delete from inbox. It should show a move to trash. (It appears your imap server supports MOVE otherwise we would see a COPY followed by the message in inbox being marked with the /deleted flag.)

Anyhow, it really looks like a tb rule/filter is kicking in to do this. I can't think of why tb would just spontaneously move something from inbox back to trash. This doesn't look anything like what might be affected by reducing the number of cached connection down to 1. But I guess it couldn't hurt to try.

Does this only occur on emails in trash that you just deleted? What if you move another older trash email back to inbox? Does the move-back-to-trash occur then? Or if you simply move an email from inbox to trash (without using a delete function) and then move it back to inbox? Or maybe move from another non-trash folder back to inbox? Is this something that previously worked for you but has just started behaving in this strange way?

Have you tried disabling or removing the 3 rules to see if *somehow* that are affecting this?

Comment 9

[gene smith]

If not caused by a rule, could it be that the moved-to-inbox email is somehow being detected as spam/junk and your setting are such that spam/junk in inbox gets moved to trash?

Comment 10

[gene smith]

Another thing to try: Mark a random (and possibly expendable) email in inbox as spam. It should then move to where ever you store spam. Select your spam folder and un-mark it as spam. It should move back to inbox and should stay there. Does this work? (This assumes you have enabled junk detection and movement in the junk settings.)

Comment 11

[gene smith]

(In reply to rjakabosky+github from comment #6)
> I noticed in the packet dump that two TCP connections were used and
> each action(move to Inbox, move to Trash) was on a different connection.

Sorry, I missed that you said this. Yes, if you have up to 5 connections, 1 will be to the folder you have selected in tb. Probably always be one for INBOX especially if you imap server supports IDLE. The other seems to be randomly select among the remaining folders (usually excluding Trash and Spam). If you reduce your connection to 1 then, of course, everything will occur on that one connection.

You mentioned "roundcube". I thought that was a server but apparently it's a client. Can you tell us which IMAP server you are using?

Comment 12

[rjakabosky+github]

I am using Thunderbird on a Windows 7 PC at work.  The IMAP server is dovecot and is on the same wired LAN as TB.

I have just found something interesting.  The problem only happens to messages that are "deleted" (using the delete key) but not to messages that are dragged/moved to the Trash.  Also the problem goes away for deleted messages if TB is closed and opened again.  So the problem only happens to messages that have been "deleted" in the current session.  I have even deleted messages using another IMAP client, TB can be used to move those messages to the Inbox.

Roundcube is just a web email client.  I have also used Claws-Mail on Linux to move a message from the trash folder to the inbox.  As soon as tb syncs it moves the message back to the trash.

I haven't tried limiting the connections to 1 yet, but I don't think that will help.  Since the problem even happens when tb is set to offline mode and another IMAP client moves a message from the trash to the inbox.  As soon as tb going online and syncs, it moves the message back to the trash.

I am not going to be able to do much more testing, going on holiday for a few weeks and only use tb at work.

Comment 13

[gene smith]

Does it do it if you use rt click menu to delete or just del key? Also not caused by being marked as spam?

Comment 14

[rjakabosky+github]

I haven't been marking any messages as spam.  All of the messages that I have been testing with are from people in the same company.  There is no reason for TB to see them as spam.

Just tried with the rt click menu "delete", the problem happens with that too.  messages deleted that way are stuck in the trash until TB is restarted, then they can be moved back to the Inbox.

Also did a bit more testing with dragging a message to the Trash folder.  The problem seems to only sometimes happen, when the message was dragged to the Trash.  But at least I have verified that some messages that are dragged to the trash can be moved back into the Inbox without problems.

Maybe there is some race condition somewhere.  It almost seem like there is a "to be deleted" flag (in memory) that is not being cleared, since a restart of tb fixes the problem.

From the IMAP packet dump, I know that there is no flag or header from the IMAP server that could be telling tb that the message needs to be deleted/moved to the trash.

I hope that helps.

Comment 15

[gene smith]

Ok. One more thing if you hv time. If tb configured to mv spam to trash, disable it temporarily.

Comment 16

[rjakabosky+github]

I didn't have the "move spam to trash" option enabled.  I don't get much spam, so I haven't needed to set any actions for spam in TB.  So it doesn't even move spam messages to the "Junk" folder (which is empty) or delete spam messages.

Comment 17

[gene smith]

Ok, just wanted to definitely eliminate that which you now have. 
Just went through my 5 active accounts and deleted a message using del key and all went to trash/deleted. Then was able to drag each right back to inbox and they all remained there.
At this point I don't know what to suggest. 

For another bug that is similar to this, bug 1398627, Jorg suggested the reporter repair all his folders. Haven't yet heard back to know if it helped. You could try that using the properties menu for each folder. Alternatively, under .../<yourProfile>/ImapMail/<accountName>/ you can, after shutting down tb, remove all the folder files, e.g., INBOX, INBOX.msf etc, and restart tb and let the folders all re-download from the imap server and be rebuilt by tb. You can make a backup before removing the files if you would feel safer. This is equivalent to the repair but probably better than doing the repair since you do it all at one time and you can make a backup if you want. I have done this many times without a backup. If the download should hang or report an error, just shut down tb, delete the partially downloaded and rebuilt files again, start tb and try again.

Anyhow, if you will be gone, have a good vacation and let me know if the repair/file rebuild helps when you get back.

Comment 18

[rjakabosky+github]

I rebuilt the IMAP account by deleted those files & folders like you said.  But the problem still happens.

One more thing to note is that the TB profile is not stored in the default location.  I used the instructions at:
http://kb.mozillazine.org/Moving_your_profile_folder

To put the profile on an encrypted Bitlocker drive.

Comment 19

[gene smith]

Can you provide the full tb imap log requested in comment 2. I see something in the server side log you attached that looks strange. Thanks.

Comment 20

[gene smith]

Reporter, If you have the time, I think maybe the full imap log would be helpful. If you enable logging using the environment  vars as described in the link in Comment 2 and then start tb and then delete and restore the message back like you describe and then see it go away again. Just once is enough and then shutdown tb and attach the generated log file using the "Attach File" link above. 

Assuming you are on windows the procedure is like this:

set MOZ_LOG=IMAP:5,timestamp
set MOZ_LOG_FILE=%USERPROFILE%\Desktop\imap.log
"%ProgramFiles(x86)%\Mozilla Thunderbird\thunderbird.exe"

Just do these manually at cmd line or make a batch/script file to do them if you prefer. You can use other paths rather than symbols if you need to. 

Thanks.

Comment 21

[gene smith]

Reporter, I haven't heard anything from you in a while. FYI, a bug has been reported that seems very similar to what you have observed, Bug 1414733. Not sure if it's the same and have asked them for a full imap log too.

Comment 22

[Wayne Mery (:wsmwk)]

Can't make progress without an imap log.

Comment 23

[Wayne Mery (:wsmwk)]

So, the operative aspect here is ithappens for message that have been DRAGGED from the inbox.  I suspect this has roots in another previously reported bug