417167 - When using IMAP for Gmail, "undo delete message" doesn't do anything

Status: RESOLVED DUPLICATE of bug 323875

(MailNews Core :: Networking: IMAP, defect)

Description

[Shaun Fischer]

User-Agent:       Opera/9.25 (Windows NT 5.1; U; en)
Build Identifier: 2.0.0.9

When using Thunderbird 2.0.0.9 under Windows XP SP2 to check a Gmail account via SSL/IMAP, a custom trash_folder_name of [Gmail]/Trash, and Thunderbird configured to move deleted messages to the Trash folder, deleted messages do move to the appropriate folder, but "undo delete message" doesn't do anything.

Reproducible: Always

Steps to Reproduce:
0.(If needed) Logon to your Gmail account via web browser & enable IMAP
1.Configure Thunderbird 2.0.0.9 on Windows XP SP2 to check a Gmail account via SSL/IMAP
2.Tools -> Options, Advanced tab, Config Editor button
3.Create a new mail.server.serverX.trash_folder_name variable with a value of [Gmail]/Trash, where X is the corresponding number of your Gmail account
4.OK out of everything
5.Tools -> Account Settings, Server Settings underneath your Gmail account on the left side
6.Set "When I delete a message" to "Move it to the Trash folder"
7.OK and close Thunderbird
8.Logon to your Gmail account via web browser
9.Delete the [Imap]/Trash label
10.Logout of your Gmail account
11.Launch Thunderbird
12.Confirm that [Gmail] -> Trash has the expected Trash icon
13.Delete 1 or more messages from your Gmail Inbox
14.Edit -> Undo Delete Message
15.Notice that nothing happens
16.Check the [Gmail]/Trash folder and notice that the deleted message is there
Actual Results:  
Deleted messages remain in the [Gmail]/Trash folder

Expected Results:  
Move the deleted messages back to the Inbox folder

I tried the same process in the latest nightly build of version 3 pre-release, but the way to tell Thunderbird to move deleted messages to a custom folder is done differently in this version.  Also, in this pre-release, deleted messages just vanish & are nowhere to be found after I set it to use this [Gmail]/Trash folder.

Comment 1

[WADA:World Anti-bad-Duping Agency]

> Expected Results:  
> Move the deleted messages back to the Inbox folder

According to Gmail IMAP document, when delete(Flag \deleted) via IMAP, Gmail removes "Label of Gmail"(==mail folder name in IMAP) if other than Inbox, and Gmail removes the mail from Inbox if Inbox.
  See http://mail.google.com/support/bin/answer.py?answer=77657&topic=12762
  (Read documents listed in meta Bug 402793 Comment #0.               )
  (See also dependency tree for meta Bug 402793, with "Show Resolved".)    
So, when Gmail IMAP, original mail(deleted mail) data won't be held in original mailbox any more once "Flag \deleted" is requested.

When "Move to Trash" model is used for delete, "Delete of a mail" is "Copy to Trash" then "Flag \deleted". And "Undelete" is possibly "Remove \deleted flag" and "Flag copied mail in Trash as \deleted".
If this is true, "Undelete" is impossible when Gmail IMAP.
"Move back from Trash to original folder" model will be required for "Undelete". 

Get IMAP protocol log and check real protocol level flow.
See Bug 402793 Comment #1 for getting log.

Comment 2

[WADA:World Anti-bad-Duping Agency]

No response from bug opener for more than two weeks...
To bug opener: Does it mean this bug is INVALID? 

Comment 3

[WADA:World Anti-bad-Duping Agency]

FYI. "Undo move" version is Bug 427007.

Comment 4

[Shaun Fischer]

Attachment: Edited IMAP log demonstrating this problem

Comment 5

[Shaun Fischer]

It moves the message to the [Gmail]/Trash folder.  Please see attachment for details.

Comment 6

[WADA:World Anti-bad-Duping Agency]

Tb's action on delete(move to [Gmail]/Trash) is simply copy then flag \Deleted.
> imap.gmail.com:S-INBOX:SendData: 13 uid copy 4739 "[Gmail]/Trash"
> imap.gmail.com:S-INBOX:SendData: 14 uid store 4739 +FLAGS (\Deleted \Seen)
On "undelete", Tb queries 4739 in Inbox.
> queuing url:imap://[e-mail]@imap.gmail.com:993/subtractmsgflags>UID>/INBOX>4739>8
And the query for 4739 fails.
> failed creating protocol instance to play queued url:imap://[e-mail]@imap.gmail.com:993/subtractmsgflags>UID>/INBOX>4739>8
Gmail IMAP apparently deleted 4739 from Inbox permanently, even though Tb didn't request Expunge. What can mail client do in this case?

Note: When "Undo of Move" at Gmail IMAP, delete step of moved mail(not original of move, move target) is executed, and is usually successful. And following undelete step of original mail(flagged as \Deleted by move mail) fails, as "Undo Delete" fails. It'll cause mail loss situation. This issue is reported by Bug 427007.

By the way, SSL & "custom trash_folder_name" doesn't have relation to this bug's issue. Remove them from bug summary, in order to avoid misleading or misunderstanding, please.

Comment 7

[WADA:World Anti-bad-Duping Agency]

Mail folder name in Gmail IMAP is simply a label in Web Interface of Gmail, and copy/move/delete of a mail merely adds/removes the label of Gmail(==folder name in Gmail IMAP).
But it seems to be different when [Gmail]/Trash & [Gmail]All Mail.
  - If a mail is newly arrived/created, it is written in [Gmail]/All Mail,
    and corrsponding label looks to be assigend to the mail.
    - Arrived mail : Inbox(Inbox can be considered to be a usual label in Gmail)
    - Sent mail via Gmail SMTP : [Gmail]/Sent
    - Draft mail   : [Gmail]/Drafts
  - If a mail is moved to [Gmail]/Trash, mail data looks to be removed from
    [Gmail]/All Mail too. And, once removed from [Gmail]/All Mail, copy back to
    [Gmail]/All Mail is impossible(copy a mail to [Gmail]/All Mail does nothing)
When delete model of Gmail IMAP account is other than "move to [Gmail]/Trash", delete form [Gmail]/All Mail doesn't seem to occur on delete. It means that copy from [Gmail]/All Mail is perhaps an effective recovery procedure from "Undo Delete" failure.

Comment 8

[Shaun Fischer]

Thanks for the explanation.  So, this is the fault of Google's implementation/translation of IMAP commands.  Do you suggest I take this up with Google and/or request they add "Undo delete" & "Undo move" to their <a href=https://mail.google.com/support/bin/answer.py?answer=78761>Does Gmail support all IMAP features?</a> document?

Comment 9

[Brian Kennelly]

You can disable the Undo & Redo operations in Gimap accounts by setting the "mail.server.server#.canUndoDeleteOnServer" preference to false.  It doesn't solve the problem, but it avoids the data loss.

Comment 10

[WADA:World Anti-bad-Duping Agency]

Another workaround of this bug.
 - Choose delete model of "Remove them immediately" intentionally by himself
This choice means "flag \Deleted then Expunge immediately", so user won't expect "Undo Delete" works for the Gmail IMAP account :-)
(See Bug 400931 Comment #8 and Bug 400931 Comment #9)   

Comment 11

[WADA:World Anti-bad-Duping Agency]

There may be Tb's problem too, in addition to apparent Gmail IMAP problem.

Following is IMAP log after log(copy&flag \Deleted step) in Comment #6.
> imap.gmail.com:S-INBOX:CreateNewLineFromSocket: 18 OK Success
> considering playing queued url:imap://[e-mail]@imap.gmail.com:993/subtractmsgflags>UID>/INBOX>4739>8
> creating protocol instance to play queued url:imap://[e-mail]@imap.gmail.com:993/subtractmsgflags>UID>/INBOX>4739>8
> playing queued url:imap://[e-mail]@imap.gmail.com:993/subtractmsgflags>UID>/INBOX>4739>8
> imap.gmail.com:S-INBOX:ProcessCurrentURL: entering
> imap.gmail.com:S-INBOX:ProcessCurrentURL:imap://[e-mail]:993/subtractmsgflags%3EUID%3E/INBOX%3E4739%3E8:  = currentUrl
> imap.gmail.com:S-INBOX:SendData: 19 uid store 4739 -Flags (\Deleted)
> imap.gmail.com:S-INBOX:CreateNewLineFromSocket: 19 OK Success
> considering playing queued url:imap://[e-mail]@imap.gmail.com:993/header>UID>/INBOX>4739
> creating protocol instance to play queued url:imap://[e-mail]@imap.gmail.com:993/header>UID>/INBOX>4739
> playing queued url:imap://[e-mail]@imap.gmail.com:993/header>UID>/INBOX>4739
> imap.gmail.com:S-INBOX:ProcessCurrentURL: entering
> 2359290:imap.gmail.com:S-INBOX:ProcessCurrentURL:imap://[e-mail]:993/header%3EUID%3E/INBOX%3E4739:  = currentUrl
> 2359290:imap.gmail.com:S-INBOX:SendData: 20 UID fetch 4739 (UID RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type)])
> imap.gmail.com:S-INBOX:CreateNewLineFromSocket: 20 OK Success

(A) Log of "failed creating protocol instance" for UID 4739 is not seen
    before "19 uid store 4739" request.
    And Tb requests "19 uid store 4739 -Flags (\Deleted)".
(B) Gmail IMAP returns OK to "19 uid store 4739 -Flags (\Deleted)".

Tb's bug is involved in (A)?. I think situation just before "19 uid store 4739" is similar to following case, although I think normal IMAP server returns error to "19 uid store 4739 -Flags (\Deleted)" when followig case.
  (1) Tb deletes a mail in Inbox(move to Trash folder)
  (2) Other mailer request Expunge to Inbox
  (3) "Undo delete" by Tb

Comment 12

[Brian Kennelly]

That log segment only shows Gimap errors.  The UID STORE and UID FETCH commands both got OK responses, even though the UID referenced no longer existed in the folder.  (Gimap should have responded NO to both commands.)

"failed creating protocol instance" was not logged because Thunderbird was able to create the protocol instance (a connection to the server).  This log entry, or its absence, does not indicate an error. 

Comment 13

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #12)
> (Gimap should have responded NO to both commands.)

Brian Kennelly, thanks for explanation.
Can you check Bug 427007 case too? ("Undo Move" case).
I can't guess why moved mail was not deleted from move target folder on "Undo Move", if move was from Gmail IMAP folder-A to Gmail IMAP folder-B.

Comment 14

[Shaun Fischer]

Just so I have a viable option to present to my clients, is it possible to configure Thunderbird to pop up a "Are you sure?" message box when you delete messages?  Or any other work-around that doesn't just hide the "undo delete" option?  I found a few "confirm delete"-related extensions, but none of them work for just a regular delete out of a regular folder.

Comment 15

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #14)

There is no other way than "never execute undelete when Gmail IMAP" to avoid this bug, until Gmail IMAP will resolve the problem.
Shaun Fischer, do you mean that there is extension(add-on) who executes undelete (Undo Delete) even when "mail.server.server#.canUndoDeleteOnServer=false is set?

FYI.
See Bug 402793 Comment #4 for "delete in Gmail".
See Bug 402793 Comment #5 for recovery of missing mail.

Comment 16

[WADA:World Anti-bad-Duping Agency]

Following is copy of Bug 467841 Comment #2 From Brian Kennelly.
> You can use the undo operations with Gimap,
> if you enable the Advanced IMAP Controls in the Gmail labs, and disable "automatically expunge".

Comment 18

[ctubbsii]

This is not a Windows specific bug.

Comment 20

[Chascon]

I can confirm this you still can't undelete on OS X Intel, version 2.0.0.21 (20090302)

Comment 21

[David Ascher (:davida)]

Bienvenu, my quick read of this bug is that it's caused by a GMail IMAP non-compliance.  Is that your read?  And if so, we should let them know at least.

Comment 22

[David :Bienvenu]

It's definitely GMail IMAP non-compliance - we could probably work around it using the All Mail folder, but I'll ping the gmail developers.

Comment 23

[WADA:World Anti-bad-Duping Agency]

David & David, Google already knows. Read comment #16, please.

Comment 24

[David :Bienvenu]

WADA, thx. That is well worth relnoting.

Comment 25

[Brian Kennelly]

(In reply to comment #23)
> David & David, Google already knows. Read comment #16, please.

I hope you are not taking my comment as proof that Google know.  I do not work for them.  (I am sure they know, because there has been much discussion on their help boards, but I am not an authority.)

(I do have some comments.)
UNDO delete never works after EXPUNGE, with any IMAP server.  In its default configuration, Gimap performs an automatic expunge, so the problem is immediate.
  * As noted, this can be turned off with a "lab", and UNDO works.

Thunderbird does not seem to process the EXPUNGE response after the STORE FLAGS.  If it was processed, it could be used to prevent the UNDO operation.

Comment 26

[Dan Mosedale (:dmosedale, :dmose)]

Since this is a fairly bad user-experience, the ideal way to deal with it would be for Thunderbird to use a supported Google API to turn off the automatic expunging.  Whether or not this blocks depends on whether such an API actually exists.  bienvenu to investigate...

Comment 27

[Brian Kennelly]

(In reply to comment #26)
> Since this is a fairly bad user-experience, the ideal way to deal with it would
> be for Thunderbird to use a supported Google API to turn off the automatic
> expunging.  Whether or not this blocks depends on whether such an API actually
> exists.  bienvenu to investigate...

Wouldn't it make more sense to disable the "canUndoDeleteOnServer" preference for Gmail accounts by default?  That will prevent the "bad user-experience", and it can be reset if the user enables the lab and disables auto-expunge.

Comment 28

[Magnus Melin [:mkmelin]]

I don't know if it's something the client can affect, but there's a gmail labs feature you can enable which will let you turn auto-expunge off in the imap options.

Comment 29

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #27)
> Wouldn't it make more sense to disable the "canUndoDeleteOnServer" preference for Gmail accounts by default?

Tb trunk now has pref of mail.server.serverN.is_gmail=true/false. is_gmail seems to be set to true as default for Gmail account. So I think it's very easier than before.

Comment 30

[WADA:World Anti-bad-Duping Agency]

I've opened bug 493893 for "defaulted canUndoDeleteOnServer=false of Gmail IMAP account".

Comment 31

[Brian Kennelly]

Does Thunderbird remove the UNDO DELETE action after receiving an EXPUNGE response for a message?  If not, ignore this comment.

Thunderbird seems to ignore the untagged EXPUNGE responses returned from a UID STORE command.  If they were processed, and the UNDO action removed, then this bug would be resolved, without needing to know if the server is Gmail.

(If the UNDO action is not removed when a message is expunged, then this problem can occur with any IMAP server, if the messages are expunged from another client, or a server action.)

Comment 32

[David :Bienvenu]

this will be fixed by the patch in bug 323875

Comment 33

[David :Bienvenu]

I'm just going to mark this a dup to get it off the blocker list.