615514 - Connection lost during move messages results in unrecovable deleted message

Status: RESOLVED WORKSFORME

(Thunderbird :: General, defect)

Description

[Pander]

User-Agent:       Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.12) Gecko/20101027 Ubuntu/10.10 (maverick) Firefox/3.6.12
Build Identifier: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.12) Gecko/20101027 Lightning/1.0b2 Thunderbird/3.1.6

Connection lost during move messages results in unrecovable deleted message for IMAP.

Reproducible: Always

Steps to Reproduce:
1. Open Thunderbird and select files to move from one folder to another folder in same IMAP account.
2. Break internet connection by switching off the modem/ADSL/whatever
3. Do the move with mouse
Actual Results:  
Thunderbird starts complaining about missing connection and eventually the messages are gone from both source and destination folder.

Expected Results:  
Messagares should be stored somewhere safe and put back in source or in destination after connection (even after reboots) is restored.

Comment 1

[WADA:World Anti-bad-Duping Agency]

"Move mails between two folders of same IMAP account" consists of;
(1) select <move_soruce_folder>, uid a:b,...x:y copy <move_target_folder>
(2) uid a:b,...x:y store +flag \Deleted
(3) Remove entries in .msf for uid=a:b,...x:y, if IMAP delete model=Move to trash or Remove it immediately(==not Just mark it as deleted).
(4) select <move_target_folder>, and fetch headers of new mails(moved mails).

At which step did you kill connection?
If while (1) or (2), problem is that Tb goes to step (3) even though connection loss.
Which mails are really copied by step (1) at IMAP server depends IMAP server. Which mails are really marked as \Deleted by step (2) depends on IMAP server.
If between step (3) and step (4), and if connection is still lost, it's normal, because information about new mails at <move_target_folder> can not be obtained from server.

It may be different if <move_target_folder> is offline-use=on folder.
Do you set offline-use=on? (Folder Properties, Synchronization)

In connection loss at any step, copied mail to <move_target_folder> by step (1) at IMAP server will be shown after recovery of connection, and deleted mails by step (2) will disappear(or shown as deleted mail if Just mark it as deleted) after recovery of connection.
Because step (1) takes far longer than step (2), problem around connection loss while "move mails between two folders of same IMAP account" is usually reported as "duplicate mails after connection is recovered".

No copied mail at <move_target_folder> even after connection recovery?

No moved mail at <move_source_folder> after connection recovery?
If step (2) is executed before connection loss, or if step (2) is retried after connection recovery, it happens. Can you see mails by "Just mark it as deleted" at Server Settings as "deleted mail"?
note: if Gmail IMAP, and if step (2) is execute or re-tried, mails are not shown as "deleted mail", because Gmail IMAP automatically expunge "lagged as \Deleted" mail.

Comment 2

[Pander]

(In reply to comment #1)
> "Move mails between two folders of same IMAP account" consists of;
> (1) select <move_soruce_folder>, uid a:b,...x:y copy <move_target_folder>
> (2) uid a:b,...x:y store +flag \Deleted
> (3) Remove entries in .msf for uid=a:b,...x:y, if IMAP delete model=Move to
> trash or Remove it immediately(==not Just mark it as deleted).
> (4) select <move_target_folder>, and fetch headers of new mails(moved mails).
> 
> At which step did you kill connection?

My ADL modem was just about to time out all connections, also in my browerer, when I started the move (mouse drag and drop). Later Thunderbird also informs me my server is off-line.

> If while (1) or (2), problem is that Tb goes to step (3) even though connection
> loss.
> Which mails are really copied by step (1) at IMAP server depends IMAP server.
> Which mails are really marked as \Deleted by step (2) depends on IMAP server.
> If between step (3) and step (4), and if connection is still lost, it's normal,
> because information about new mails at <move_target_folder> can not be obtained
> from server.
> 
> It may be different if <move_target_folder> is offline-use=on folder.
> Do you set offline-use=on? (Folder Properties, Synchronization)

Both folders are set for off-line use and we on-line (at least I assumed until I got the time out).

> In connection loss at any step, copied mail to <move_target_folder> by step (1)
> at IMAP server will be shown after recovery of connection, and deleted mails by
> step (2) will disappear(or shown as deleted mail if Just mark it as deleted)
> after recovery of connection.
> Because step (1) takes far longer than step (2), problem around connection loss
> while "move mails between two folders of same IMAP account" is usually reported
> as "duplicate mails after connection is recovered".
> 
> No copied mail at <move_target_folder> even after connection recovery?

Nope

> No moved mail at <move_source_folder> after connection recovery?

Nope, first I though my network card of operating system was causing the connection problem, so I rebooted. Later I went to see the ADS modem had a red light. Took about 30 minutets before my ISP restored connection. But at that time I already had rebooted the machine.

> If step (2) is executed before connection loss, or if step (2) is retried after
> connection recovery, it happens. Can you see mails by "Just mark it as deleted"
> at Server Settings as "deleted mail"?

Messages also not found in Trash.

> note: if Gmail IMAP, and if step (2) is execute or re-tried, mails are not
> shown as "deleted mail", because Gmail IMAP automatically expunge "lagged as
> \Deleted" mail.

No Gmail was begin used. Even on the server, I could not find the messages on the appropriate ~/Maildir files.

Comment 3

[WADA:World Anti-bad-Duping Agency]

It seems "uid copy" step (2) is not passed to sever, because mails are copied at server if the command is passed.
It possibly problem around "move mails while work offline" because network error happened on any connection at same time in your case.
1. Network error happened, and network interface died, then it's notified to Tb, and Tb goes to Offline Mode automatically.
2. At same time, you tried to move mails betwee IMAP folders.
3. Then similar problem to bug 433806 happened. If you tried another move, bug 567716 might happen.
4. Upon going back to Work Online, mails were lost by bug 433806 and/or bug 567716.

Comment 4

[Pander]

Thank you for looking into this. I hope this contributes to improved off-line moving, deleting, detachting, savedraft, etcetera of email in Thunderbird.

Comment 5

[WADA:World Anti-bad-Duping Agency]

Setting dependency to bug 433806 for ease of search and tracking.

Comment 6

[Wayne Mery (:wsmwk)]

confirming, but will need retest after blockers are fixed.

Comment 7

[Wayne Mery (:wsmwk)]

Pander, can you still reproduce the problem?

Comment 8

[Pander]

Nope