Open Bug 427007 Opened 12 years ago Updated 4 months ago

"Undo Move Message" loses message - after move from Gmail IMAP inbox to local folders

Categories

(MailNews Core :: Networking: IMAP, defect)

x86
Linux
defect
Not set
critical

Tracking

(Not tracked)

People

(Reporter: adam.richard2020, Unassigned)

References

(Depends on 1 open bug, Blocks 1 open bug)

Details

(Keywords: dataloss, Whiteboard: [not tbs fault?][has protocol log])

Attachments

(1 file)

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.11) Gecko/20061201 Firefox/2.0.0.11 (Ubuntu-feisty)
Build Identifier: 1.5.0.14pre (20071023)

I'm using GMail with IMAP.

I had message in my Inbox on my GMail account, and moved it to a folder in my "Local Folders".

Then I selected "Edit -> Undo Move Message".

The message was gone from both my Inbox and the folder I had moved it to - as far as I could tell, lost completely.

I discovered that I could get it back in the Local Folders folder (but with the Subject line missing) by selecting Edit -> Redo Move Message.


Reproducible: Always

Steps to Reproduce:
1.
2.
3.
Please can you try the latest version of Thunderbird (2.0) available from http://www.getthunderbird.com/

The 1.5 branch is now unsupported, and there have been various bugs fixed that could have already fixed your problem.
Version: unspecified → 1.5
Hmm, well the problem is I'm using Ubuntu 7.04, which doesn't seem to want to upgrade me to Thunderbird 2.0 of its own accord.  But if you want to assume it's fixed in the newer version, that's fine.
Does seem to happen on trunk too. I would guess it's something with the gmail imap implementation though. (Works fine for a normal imap server.)
Status: UNCONFIRMED → NEW
Ever confirmed: true
Summary: "Undo Move Message" loses message → "Undo Move Message" loses message - after move from Gmail IMAP inbox to local folders
Whiteboard: [not tbs fault?]
Version: 1.5 → Trunk
I saw similar report on "Undo delete of Gmail IMAP", but I can't recall bug number.

Move of a mail is combination of "copy of a mail" and "flag \Deleted" when IMAP.
When usual IMAP server, "flag \Deleted" only adds flag of "\Deleted" to the mail, and the mail data is kept until Expunge.
But when Gmail IMAP, "flag \Deleted" means "remove of label", and the "label" of Gmail is mail folder name when Gmail IMAP. i.e. Result of "flag \Deleted" when Gmail IMAP is same as "flag \Deteted then Expunge".
Read Gmail documents listed in Bug 402793 Comment #0, please.

When Gmail IMAP, all mails are kept in [Gmail]/All Mail(all mails have label of [Gmail]/All Mail), so recovery from this "[Gmail]/All Mail" folder looks to be possible, if move of mail. But it seems that delete==mail move to [Gmail]/Trash(standard Trash of Gmail, so cleaned up by retention) is not recoverable in some situations.
Blocks: tb-gmailWIP
"Undo delete" version was Bug 417167.
You can disable the Undo & Redo operations in Gimap accounts by setting the "mail.server.server#.canUndoDeleteOnServer" preference to false.  It doesn't solve the problem, but it avoids the data loss.
Attached is IMAP protocol log for next test, with Tb trunk/Gmail IMAP.
 (1) Move a mail from Inbox to My_Trash (both are IMAP folders of same account)
 (2) Undo Move
Although "Undo delete" step is not properly done due to Gmail IMAP's problem,
  Gmail IMAP returns OK to "18 uid store 53 -Flags (\Deleted)",
  even though Gmail IMAP doesn't recover the mail once Gmail label is removed.
  This is problem of Bug 417167.
delete request of moved mail(mail in My_Trash in my test case) was not observed, then loss of the moved mail(mail in My_Trash) was not re-produced.

When move to local mail folder, mail in the local mail folder(move target folder) was deleted by "Undo Move" operation?
If yes, is it re-creatable with Trunk? (Branch only issue or not).
I could re-produce "delete of mail at move target folder" on "Undo Move" when move target folder is local mail folder.
Following flag was set in local mail folder after "Undo Move".
> X-Mozilla-Status: 0009
"Redo Move" after that is probably remove of deleted_flag in X-Mozilla-Status:.
This bug is a variation of phenomena of Bug 417167. Root cause is OK response to "18 uid store 53 -Flags (\Deleted)" by Gmail IMAP even though Gmail IMAP doesn't recover the mail. From mail client view, "Undo Move" has been successfully done by IMAP server, so delete from move target folder(local mail folder in your case) is normal behavior.

One question.
Why not deleted from move target folder when "Undo Move of move from IMAP folder to IMAP folder"?
Depends on: 417167
The undo processor is determined by the target folder type.

If the target folder is IMAP, Thunderbird does detect, probably because no untagged FETCH response reported success, that the message was not undeleted from the source folder, so it does not attempt to delete the message from the target folder.  (nsImapUndoTxn.cpp)

If the target folder is local, rather than IMAP, Thunderbird does not detect when the undelete fails in an IMAP source folder, and the target delete is allowed to proceed.  (nsLocalUndoTxn.cpp)
To Brian Kennelly: Thanks again for you explanation.
This bug should be closed as DUP of Bug 417167? Or better to be kept separately?  
I believe we have two issues.  

1. Gimap does not support 'undo delete'.  There is nothing that Thunderbird can do about it, except to set the 'canUndoDeleteOnServer' preference to false.  (Bug 417167)

2. Thunderbird does not detect a failed 'undo delete' when undoing a move from an IMAP folder to a local folder.  This is the real problem uncovered by this bug report. 


Addendum: this bug is not Gmail specific.  The same problem occurs with any IMAP server that does not support 'undo delete' (I tested it with AOL).  I did not test it, but I expect to see the same problem if the source folder is expunged before the undo.

> The same problem occurs with any IMAP server that does not support 'undo delete'

Questions to confirm situation.

If AOL IMAP & Tb with AOL IMAP support & AOL IMAP account defined via AOL support by Tb, imap.aol.canUndoDeleteOnServer looks to be set to false as default.
> http://users.comlab.ox.ac.uk/ian.collier/Misc/aboutconfig.html
Problem when AOL IMAP account is defined by hand/by usual procedure?

Is there any other IMAP server who doesn't support(or improperly/invalidly supports it like Gmail IMAP) "uid store XX -Flags (\Deleted)" after "store XX Flags (\Deleted)"?
It is an AOL IMAP account set up normally (not using the AOL.rdf file).  
I was initially testing the IMAP-to-IMAP moves within the same account.

More careful testing from the AOL account reveals that undo works correctly when moving a message to another account (either IMAP or local), but not within the same account.  (The XAOL capability supports an atomic move operation, which cannot be undone by the normal undelete/delete.)

Gmail supports undelete from the "All Mail", Trash and Spam folders, and moves from those folders to another account can be undone.  However, moving to another Gmail folder cannot be undone correctly.  The original message is retrieved, but the copy is not deleted.  This means that this bug is probably a duplicate of bug 162008 (Gimap does not support UIDPLUS).

Test with a Courier IMAP server (with UIDPLUS) show that undo works in general, but, if the deleted message is expunged before the undo, the original message is not recovered (as expected), but the copied message is still deleted.  (Courier also reports OK to the failed UID STORE.)

As you can see, my original analysis was wrong.  It is still true that Gimap does not, in general, support undo delete, so the other report is unchanged.

This report is modified as follows: 
1. Undo move does not work if the target server is IMAP and does not support UIDPLUS.  This part is a duplicate of bug 162008.

2. Thunderbird does not detect a failure to undelete the source message, and, if the target folder is local or on IMAP with UIDPLUS, the target message will be deleted, resulting in no copies.

** Assigning to myself **
Assignee: nobody → bugmil.ebirol
I am playing with current trunk version of seamonkey. Deleting a message from a local Inbox (updated through POP3) works but not Undo. At the very first attempt to access Edit/Undo from the menu I got:

WARNING: Key 'key_preferences' of menu item 'Preferences…' could not be found: file /home/mmokrejs/proj/comm-central/src/mozilla/layout/xul/base/src/nsMenuFrame.cpp, line 1055


Several times then I have received

WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80550005: file /home/mmokrejs/proj/comm-central/src/mailnews/local/src/nsLocalUndoTxn.cpp, line 220


I can also confirm the message is not even in Trash folder.
I’m having this problem too, and boy, loosing messages is really the worst kind of bug there is. I nearly lost a business deal because of it. I found tons of similar bugs in Bugzilla. Looks like there are too many bugs for too few developers…
I wish I could write C++, so  I could help. :/
Assignee: bugmil.ebirol → nobody
Component: General → Networking: IMAP
Keywords: dataloss
Product: Thunderbird → MailNews Core
QA Contact: general → networking.imap
does this still exist, and is it specific to gmail?
Whiteboard: [not tbs fault?] → [not tbs fault?][has protocol log]
(In reply to Wayne Mery (:wsmwk) from comment #20)
> does this still exist, and is it specific to gmail?

I just tested it a bit more, and it’s worse than expected. This may become confusing, and it certainly was for me. So please try hard to get past simple confusion, before asking specific questions, as to not further confuse things. :)

————————————————————————————————————————————————————————————

Manually moving and moving back:

For GMail, manually moving a message to the local archive folder results in a duplication, and hence *two* times that same mail in the local folder. Not so for archiving, which works normally.
Manually moving the first duplicate back works as expected. Manually moving the second one back looks like it “overwrites” the first one. (Upon opening the folder after the move, the message shortly goes away and then returns to the list again.)
Manually moving it (whichever that is) to local a second time, doesn’t result in duplication anymore. Moving it back again, works as expected too.

Doing the same for my local server (Courier IMAP) everything works as expected, and the above problem doesn’t occur.
Manually moving it from my local server to GMail, then to a local store, back to GMail, and then back to my local server, works too.

————————————————————————————————————————————————————————————

Manually moving, but using undo to move it back:

With undo, it’s a different story: After manually moving it from GMail to the local archive folder, undoing results in the second of the duplicated message going away from the local archive folder, but never ending up anywhere else (as far as I can tell). It vanishes.
I can not undo the first duplicate, since there obviously is only one undo, which I just used.

Doing the same, but moving it to the local *server* instead works as expected. But undoing that does nothing. I didn’t test the permutations after that.

With my local server, things work as expected.

————————————————————————————————————————————————————————————

Archiving, and manually moving back:

For GMail, archiving works as expected. Moving it back too.
BUT: Archiving it a second time, results in the above message duplication. Manually moving *both* duplicates back, again, results in *one* message in GMail.

With my local server, everything works as expected.

————————————————————————————————————————————————————————————

Archiving, but using undo to move it back:

For GMail, as said, archiving works, BUT the undo here does nothing.

With my local server, everything works as expected.

————————————————————————————————————————————————————————————

What I did not really test, is REDO. (Too many permutations. The above was already too messy for me.) But the two cases where I did it (GMail too), resulted in the mail vanishing.

When the above works, one can re-test redoing.

————————————————————————————————————————————————————————————

I hope this helps debug the problem.

Loss of data is of course really bad.
[Tracking Requested - why for this release]:
I have several times recently lost files after a move/undo move. I am currently using TB 45.4.0 with Gmail. I have not been able to find the missing messages, but I have not tried the "redo move message" approach. I will next time it happens, but I am reluctant to lose messages just to verify the problem!

I am also experiencing this problem. Why does the Edit > Undo Move Message command even exist for IMAP accounts if it results in messages disappearing? Losing messages is not a good "feature" for an email program.

I am also experiencing this problem, not with a Gmail-Imap, but with a university Imap-server.
Moving a message to a local folder works
ctrl-z or Edit -> Undo Move Message deletes the local copy, but does not place the original email back into the IMAP folder.

The emails do show up in the INBOX file in my .thunderbird directory (on Ubuntu), but they do not seem to be pushed back to the IMAP server. They hence aren't accessible from the IMAP anymore, and somehow are not shown in Thunderbird anymore, either.

Compacting INBOX then finally removes them from that file (so I guess they are never "undeleted", but have remained in the INBOX file only as artifacts).

In any case, my workaround for the moment is to close Thunderbird, copy the lost emails from the mbox file of the inbox (/ImapMail/.../INBOX) to the mbox file of the folder where I wanted them in the first place (Local\ Folders/.../<target>), delete the <target>.msf and restart Thunderbird to rebuild the msf file.

mbox files are a bit of a mess, suffice to say that emails start with a "From " (space included, at the start of a line - vim: "/^From ") string. Makes parsing a bit easier ;-)

I guess we could simply delete the INBOX.msf and rebuild it (I didn't try if this pushed the undeleted emails back to the IMAP server), but since this folder is much more active I would have to re-delete all the other emails (and my target folder is an archive where I don't delete or work, so the msf-file does not contain any useful information in any case)...

HTH, good luck!

-- t

You need to log in before you can comment on or make changes to this bug.