792198 - mail lost when moving from imap to Local Folders

Status: RESOLVED DUPLICATE of bug 462156

(MailNews Core :: Networking: IMAP, defect)

Description

[beanladen]

Attachment: out_stack

User Agent: Mozilla/5.0 (X11; SunOS i86pc; rv:14.0) Gecko/20100101 Firefox/14.0.1
Build ID: 20120714080654

Steps to reproduce:

move a mail with the mouse from an IMAP folder (the second of three different accounts) to a local
folder (subfolder).

OS: Openindiana 151a5 (aka "Opensolaris") x86(=64bit), osol contrib build from mozilla.
Message synchronizing off (since IMAP was invented for doing it in this way!) .



Actual results:

Mail disappears after a longer while (IMAP server is a bit slow), but never shows up in the local subfolder (scanned mail file, there's not a single byte trace anywhere). Instead, cursor still shows 
busy clock, and an attempt to "repair folder" is blocked with "The operation failed because another 
operation is using the folder". This happens often, but not always. A stacktrace of the still live process (attached) shows a zombie, and operations waiting. Error console has some errors:

Before:
Timestamp: 09/18/12 06:08:58 PM
Error: Please do not load stuff in the multimessage browser directly, use the SummaryFrameManager instead.
Source File: resource:///modules/summaryFrameManager.js
Line: 119

Around lost mail event:
Timestamp: 09/18/12 06:12:57 PM
Error: formatURL: Couldn't find value for key: TIME_MAIN
Source File: resource:///components/nsURLFormatter.js
Line: 131

Timestamp: 09/18/12 06:12:57 PM
Error: formatURL: Couldn't find value for key: TIME_FIRST_PAINT
Source File: resource:///components/nsURLFormatter.js
Line: 131

Timestamp: 09/18/12 06:12:57 PM
Error: formatURL: Couldn't find value for key: TIME_SESSION_RESTORED
Source File: resource:///components/nsURLFormatter.js
Line: 131

versioncheck.addons.mozilla.org : server does not support RFC 5746, see CVE-2009-3555

None of the bug reports with this topic are quite close enough to this problem. This did never 
happen in thunderbird 3.x, but is definitely older than version 14.



Expected results:

Mail should appear in Local Folders and should not be sent to byte heaven...
Critical regression, data loss !

Comment 1

[:aceman]

Did your messages appear in any Trash folder?

Comment 2

[beanladen]

No, not at all. As said, no trace, it seems that the thread that should have copied it died
in the middle of the process, while another thread already deleted it from the IMAP server.
That seems also be a dangerous sequence, it should be deleted AFTER the mover thread did his
job completely. Or maybe a wrong error checking when the moving thread dies ?

Comment 3

[beanladen]

Bug #522675 looks promising, maybe a variant of this one, just fixed in TB 15. Downloaded
15.0.1, and could move mails without loosing any. But server is fast now, will test tomorrow
when it's busy again.

Comment 4

[:aceman]

There is also a similar bug 751097.

Comment 5

[beanladen]

Tested with very unreliable server, got a even a message 'Operation did not complete' once,
message disappears completely for a while, but finally pops up at the destination,
mail was never lost again (copying up and downstream, tested several times with large
mail). Don't still know what happens when server goes down while copying upstream, maybe
then mail will be lost again, but that I cannot test.

So far this seems to be resolved for the major cases by the fix of Bug #522675.

Comment 6

[beanladen]

Attachment: stacktrace TB 15.0.1 just after problem reappeared

Comment 7

[beanladen]

Damned, happened again with TB 15.0.1 ! Just the same symptoms, but no zombie thread this time.
New stacktrace attached.

Comment 8

[Wayne Mery (:wsmwk)]

benladen, this is not easy for you to reproduce, correct?  Do you still see it in version 17?

xref bug 462156

Comment 9

[beanladen]

No, I've given up on this, just follow the herd now and synchronize locally, so that I now
have essentially a "POP-IMAP". This is a sad loss of functionality, but I can't bear loosing
mail anymore.
So if you want to really fix this, just find the scheme where the 
'start move->delete on server in the meanwhile->end move with error' 
is made and change that to 
'start move->wait until file is really on client->delete on server'
That should fix all problems where anything unexpected happens in between,
and it should have sufficient long timeouts (3-5 min) if any.
Shouldn't be too hard to change if someone knows the code.

Comment 10

[Wayne Mery (:wsmwk)]

Is this bug 693353 ?

Comment 11

[Wayne Mery (:wsmwk)]

If bug 462156 (or whatever fixes it) doesn't help this bug, then it can be reopened