If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Message lost during IMAP 'move to' folder

UNCONFIRMED
Unassigned

Status

Thunderbird
Untriaged
UNCONFIRMED
5 years ago
a year ago

People

(Reporter: Colin, Unassigned, NeedInfo)

Tracking

17 Branch
x86
Mac OS X

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

5 years ago
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:20.0) Gecko/20100101 Firefox/20.0
Build ID: 20130326150557

Steps to reproduce:

Moved message from IMAP 'Inbox' to other IMAP folder


Actual results:

Received error that server connection was lost (yes, the server did go down momentarily) but the message disappeared from Inbox and never appeared in the other folder. was lost forever because apparently TB deleted the message before attempting to write it to the 'move to' folder. 


Expected results:

Should have given the error message about server connection lost WITHOUT DELETING THE MESSAGE from the inbox. Now the message is gone. Confirmed the same thing has happened 5 or 6 times in the past. Incidentally, message is still in the IMAP Inbox folder on the server, but TB won't show it.
Which case?
(a) Move from IMAP-1, Offline-Use=On  to IMAP-2, Offline-Use=On
(b) Move from IMAP-1, Offline-Use=On  to IMAP-2, Offline-Use=Off
(c) Move from IMAP-1, Offline-Use=Off to IMAP-2, Offline-Use=On
(d) Move from IMAP-1, Offline-Use=Off to IMAP-2, Offline-Use=Off

When IMAP to IMAP, move is approximately;
  (i)   at IMAP-1, select MoveSource, uid xx fetch body[] to temp file
  (ii)  at IMAP-2, append MoveTarget, upload mail data from temp file
  (iii) at IMAP-1, uid xx store +Flags \Deleted
  (vi)  delete temp file
When "Move from IMAP folder to local folder", it's (i)+(iii) with "temp file=>local mail folder", and similar problem of "mail is deleted even when copy fails" is reported to some bugs.

IIRC, "move mail in IMAP" was changed to similar one to "move while Work Offline mode with fakedKey, and re-sync with server when going back to Work Online mode" by a Tb release.

If "copy/move from Offline-Use=Off", "copy/move mail to different account" is prohibited when actual Work Offline mode.
But if "copy/move from Offline-Use=On", both "copy/move within same account" and "copy/move between different accounts" is accepted even when actual Work Offline mode.
So, if "move from Offline-Use=On to other account while actual Work Offline mode", msgDBHdr is copied to MoveTarget folder and mail is shown at thread pane, even though message data is not downloaded yet from MoveSource Mbox of MoveSource IMAP account.  

I guess problem may occur if "move from IMAP to different account" and if failure occurs while copying mail data, because actual copy of mail data is done by "fetch body[]" command instead of by simple "uid xx copy MoveTarget" command only, and because it is done in "re-sync with server" phase.

By the way, because message folder is IMAP Mbox, "mail flagged as \Deleted" can always be viewed by IMAP delete model of "Just mark it as deleted" in Tb(Server Settings, shown with strike-thru line at thread pane), unless "mail flagged as \Deleted" is removed from IMAP Mbox by expunge operation at server or by expunge request from client(Compact/Compact Folders of Tb issues expunge command).
if not expunge yet, "Undelete" of any "mail flagged as \Deleted" is possible if IMAP(Simply, uid xx store -Flags \Deleted is issued by Tb).
If you frequently see your problem, use "Just mark it as deleted" when you do bulk mail move from IMAP folder.
FYI.
As I already wrote, "delete mail in IMAP" is designed as "storing \Deleted flag" simply. So, traditional "Move to trash" in local mail folder is not needed by design. And, in addition to "Remove immediately" model, Tb already has View/"Not Deleted" in addition to All/Unread. So, "Just mark it as deleted" is usable in daily use and is far convenient than before, because "Undelete" is easy and is possible any time without annoyng 24hour/365days display of all deleted mails with strike-thru line.
FYI.
If "move from IMAP Offline-Use=Off foler to MaildirStore/local folder accout(type=none/pop3)", phenomenon of "mail is deleted even though copy of mail data is not successful" can be observed always(Bug 856519).
In that bug, symptom of "mail copy from IMAP to MaildirStore/local is not correctly executed" itself is apparently bug in MildirStore. However, I believe symptom of "uid xx store +Flag \Deleted is issued reardless of mail copy status" is caused by both of followings.
(a) Maildir doesn't return error status to requestor correctly in many cases.
(b) functions for CopyMail/MoveMail doesn't check return code or error status in many places.
Part of (b) is commonly used in both BerkleyStore folder and MaildirStore folder. So, I think (b) is applicable to problem like yours too.
Following may be relevant to problem.

nsImapProtocol::BeginMessageDownLoad is defined at;
> http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapProtocol.cpp#2967
> 2967 nsresult nsImapProtocol::BeginMessageDownLoad(
In BeginMessageDownLoad, following comment is seen.
> http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapProtocol.cpp#2990
> 2978   if (content_type)
> 2979   {
> 2980     m_fromHeaderSeen = false;
> 2981     if (GetServerStateParser().GetDownloadingHeaders())
> 2982     {
> 2983       // if we get multiple calls to BeginMessageDownload w/o intervening
> 2984       // calls to NormalEndMessageDownload or Abort, then we're just
> 2985       // going to fake a NormalMessageEndDownload. This will most likely
> 2986       // cause an empty header to get written to the db, and the user
> 2987       // will have to delete the empty header themselves, which
> 2988       // should remove the message from the server as well.
> 2989       if (m_curHdrInfo)
> 2990         NormalMessageEndDownload();

Because of (A) "multiple mail move from IMAP folder to other account", (B) "fetch moved mails to other account's folder" phase exists. This phase may consist of multiple (C) "fetch multiple messages to other account's folder" steps.
If error occurs during a step of (C), and if remaining (C)'s for not-fetched-yet mails is tried, following (D) what is written in source comment may happen.
> (D) multiple calls to BeginMessageDownload w/o intervening calls to NormalEndMessageDownload or Abort
If this happens, "download of all mails is normally end" for (A) "multiple mail move from IMAP folder to other account". If so, because (E) "delete mails from move source folder" is involved in "move mails", the (E) "delete mails from move source folder" is executed ordinary.
(In reply to Colin from comment #0)
> Actual results:
> Received error that server connection was lost (yes, the server did go down
> momentarily) but the message disappeared from Inbox and never appeared in
> the other folder. was lost forever because apparently TB deleted the message
> before attempting to write it to the 'move to' folder. 

As seen in bug 856532(test of MaildirStore, move from IMAP Offline-Use=On to local folder), in some cases, mails are immediately removed from thread pane of move-source folder but "uid xx store +Flags \Deleted" is not issued.

Was "uid xx store +Flags \Deleted" actually requested when the connection error occurs? Why was sending "uid xx store +Flags \Deleted" possible even though server down?
Server down at server of move target folder only?
(In reply to Colin from comment #0)
> Expected results:
>(snip)
> Incidentally, message is still in the IMAP Inbox folder on the server, but TB won't show it.

Expected result? Or Actual result?
I assume "Actual result".

When the "message is still in the IMAP Inbox folder on the server, but TB won't show it" occured?
While the server(move mail source server) is still down?
Or even after the server is recovered?

Questin about states at "move target server".
Did "server down" occur at "move target server" concurrently?
If no "server down at move target", how many mails were normally copied to move target Mbox on the move target server?
Blocks: 861634
Blocks: 861501
No longer blocks: 861634
Colin,
Do you still see this issue when using a newer version?
Flags: needinfo?(colin.stone)
You need to log in before you can comment on or make changes to this bug.