530709 - composing draft while offline results in apparent dataloss

Status: RESOLVED DUPLICATE of bug 397910

(Thunderbird :: Message Compose Window, defect)

Description

[[e-mail]]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.6pre) Gecko/20091112 Ubuntu/9.10 (karmic) Firefox/3.5.2
Build Identifier: 3.0pre

If you create and save, but don't send a message while offline then go online, that message is going to disappear. It's frightening and quite bad. It seems to reappear if you restart TB. Hate to do it since we already have RCs, but I suggest blocking on TB3 until this is fixed.

Reproducible: Always

Steps to Reproduce:
1. Create a message
2. Go offline (this can be step 0 if you prefer)
3. Save the message
4. Note that it's in the drafts folder
5. Close the message (since it's saved)
6. Go online
7. Note that your message is now gone
8. Curse (loudly)
9. File bug
10. Restart TB, note that the message is restored. 
11. Be confused.
Actual Results:  
The message is saved, then disappears, then reappears.

Expected Results:  
Message should NEVER disappear.

Comment 1

[Mark Banner (:standard8)]

WFM on Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-GB; rv:1.9.1.5) Gecko/20091121 Lightning/1.0pre Thunderbird/3.0

At your step 6 the message briefly disappears then comes back again about 2 seconds later. I believe this is the offline replay doing its work.

I tested this on two different servers (gmail & courier) and got the same results.

I guess this could be an issue on specific IMAP servers, so it would be worth getting an IMAP log.

Comment 2

[Ludovic Hirlimann [:Usul]]

WFM me too.  The message stays in draft - and is always visible.

Michael are you using pop3 and is your draft folder a local folder ?

Comment 3

[Wayne Mery (:wsmwk)]

component=compose until we have a better handle

Comment 4

[:Dossy Shiobara]

WFM Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.4pre) Gecko/20090915 Thunderbird/3.0b4

The draft was saved in my Local Folders>Drafts, then when going online, a _copy_ showed up in my first IMAP account's Drafts folder; the original was still in my Local Folders>Drafts folder.

I'm using Dovecot IMAP as the server.

Comment 5

[Mark Banner (:standard8)]

Not blocking on this as there's a large number of WFM and no clear steps to repeat.

Please feel free to renominate if you can reproduce it.

Comment 6

[[e-mail]]

Hmmm...still not working here, and I just redid it with two messages to make sure nothing strange was happening the first couple times. I'm using gmail servers as well. 

I'm using IMAP, with my draft folder being on the server. After saving, the email shows up in my IMAP draft folder. After coming back online, it's gone. After restarting TB, it's back.

Comment 7

[:Dossy Shiobara]

When offline, shouldn't drafts be saved to the Local Folders>Draft folder?  There's no way it can save directly to the IMAP Drafts folder while offline ... ?

Comment 8

[[e-mail]]

Logically, that must be what it's doing, but that's not what its showing on my system.

Comment 9

[Mark Banner (:standard8)]

Sorry, but this still needs confirmation by someone else and/or clear steps to repeat before we can consider blocking on this.

A protocol log (as I suggested before) would be one way to help this along. See https://wiki.mozilla.org/MailNews:Logging if you need help generating that log.

Comment 10

[[e-mail]]

I made a log of this process with imap log level set to 5, but I don't feel too good posting it here. It indicates my entire folder hierarchy and information about all of my email addresses. 

Is there a better way to handle this?

Comment 11

[[e-mail]]

I also just created a new profile with a new account (on google apps), and it did some strange things, though I wasn't able to reproduce this immediately. Not sure what the difference is. I tried with the drafts folder both selected and unselected for offline use, and I don't /think/ that made a difference.

Hmmm...agree not to block at this point, though something funky is certainly afoot.

Comment 12

[Matthias]

I can confirm an even disastrous data loss during offline work of an IMAP account with Thunderbird 3 which wasn't the case in Thunderbird 2!

I'm using TB 3.0 (installed on four different machines), Windows XP and tested this on two complete different IMAP accounts (GMX, university account - blurry I know). Ad-Ons: GlodaQuilla, MinimizeToTray Plus, Dictionaries, ThreadKey

Reproducible steps:

1. Go offline (automatically or manually)
2. Compose message
3. Save message to draft folder (IMAP account folder)
4. Go online (automatically or manually)
5. Saved messages in draft folders are removed (You can watch this happening if you have several saved mails. The folder is automatically compressed( !!-option in preferences isn't enabled!)) and are not stored on IMAP folder.

These messages are completely removed from the local system and not uploaded to the IMAP account. They are even deleted from the local disk and can't be restored using index (msf file). Messages don't show up again when TB is restarted.

TB 2 didn't show this disastrous behaviour. During composing a quite large mail taking some time several copies of the same mail are stored in the draft folder. Those stayed there when you went online and were copied to IMAP. Now all of them are removed when going online (this of course also happens if a copy of a mail is only stored once).

This is a serious problem for me as it makes TB useless on a laptop if you don't finish your mail and use 'send later'.

Anyone who can reproduce this behaviour with his IMAP account? Any suggestions?

Comment 13

[Wayne Mery (:wsmwk)]

Matti, how do your findings differ from bug 539183, and from comment 0?  (I'm a slow reader this morning)

Comment 14

[Matthias]

(In reply to comment #13)

Hello Wayne,

> Matti, how do your findings differ from bug 539183
Sorry but I don't catch any connection to bug 539183.


, and from comment 0?  (I'm a
> slow reader this morning)
My 'finding' differs from comment 0 only in step to reproduce number 10.
The deleted messages are not restored after restart.

The first time I experienced this behaviour the messages even got completly lost (not stored on harddrive in profile folder).
So far I only could definitely reproduce the removal of the message in the Draft folder. Anyway I could find it on my hdd in the profile folder but wasn't in a position to bring it back using restore index. 

Expected Results:
Behaviour like TB 2.X when composing messages at an IMAP account during offline session. Message is removed and then restored in draft folder marked as unread. By that it's present at the server as well, all sync all fine.


\mat

Comment 15

[u370415]

I experience exactly the same behaviour as Matthias. Can be reproduced with the described steps.

Comment 16

[u370415]

Attachment: IMAP log where this bug occurs

Tried to anonymize as much as possible without loosing too much information. Before the log, I switched to offline line, created a new mail in imap://[e-mail], saved it (checked that it's in INBOX.Drafts) and closed Thunderbird. The log starts with the restart of Thunderbird in online mode. The saved mail disappears from the Drafts!

Comment 17

[Andrea Soster]

Same problem here (Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a1pre) Gecko/20100201 Shredder/3.2a1pre) -> after going back online my draft mail is permanently lost.

Can we do anything to fix this? I think that it's quite common to have an IMAP account on a laptop.

Comment 18

[ecser99]

Same problem as Matthias, 

and this can be reproduced in TB 3.0.4. and Lanikai:
Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.2pre) Gecko/20100302 Lanikai/3.1b1

I can cofirm the data loss. I am using Gmail with imap. The draft folder, I am using is shown in Gmail (browser) as "[IMAP]/drafts". If things like this happen, I start loosing trust in TB and go back to Gmail. 
This is a real show-stopper, this should fixed urgently. I don't understand, how could an error like this pass testing. You shoud rethink your testing strategy, my suggestion: test the basic fuctions (like receiving, sending, saving mails, going working ofline, syncing after offline) especially with the most widespred configurations (Like xp, win7). If this is not working 100 % don't release, fix it and retest!
Othervise TB will shortly loose its reputation beeing a reliable and fast mail client.   

In the Accounts setting, Copies and Folders, I chose "other", then I can see two different "Draft" folders under my Gmail account, but both have the name "Draft" it is not visible, waht is the diference! One of them is the [IMAP]/drafts folder. This should be visible in TB and some guidence should be there (e.g. recommended setting)

Comment 19

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #16)
> IMAP log where this bug occurs

Why .INBOX.Drafts?

> 0[2626140]: considering playing queued url:imap://[e-mail]:993/liteselect>.INBOX.Drafts
> 0[2626140]: creating protocol instance to play queued url:imap://[e-mail]:993/liteselect>.INBOX.Drafts
> 0[2626140]: failed creating protocol instance to play queued url:imap://[e-mail]:993/liteselect>.INBOX.Drafts
> 0[2626140]: considering playing queued url:imap://[e-mail]:993/liteselect>.INBOX.Drafts
> 0[2626140]: creating protocol instance to play queued url:imap://[e-mail]:993/liteselect>.INBOX.Drafts
> 0[2626140]: playing queued url:imap://[e-mail]:993/liteselect>.INBOX.Drafts
> 4772[70c4ac0]: 6c7ec00:mailbox.domain1.tld:S-INBOX.Drafts:ProcessCurrentURL: entering
> 4772[70c4ac0]: 6c7ec00:mailbox.domain1.tld:S-INBOX.Drafts:ProcessCurrentURL:imap://[e-mail]:993/liteselect%3E.INBOX.Drafts:  = currentUrl
> 4772[70c4ac0]: 6c7ec00:mailbox.domain1.tld:S-INBOX.Drafts:SendData: 13 getquotaroot "INBOX.Drafts"

Daniel Minder(IMAP log provider), your case probably has different problem from original issue in addition to original issue. Trying to access with preceding "." in Mbox name is partiqular issue when name space is used and delimiter is ".". As I wrote in bug 528669, same problem occurs on template in Templates folder. Can you open separate bug for problem of preceding "." + draft loss, after checking template case?

Comment 21

[Dan Mosedale (:dmosedale, :dmose)]

Confirmed, given the variety of reports including bug 566655.

Comment 23

[Dan Mosedale (:dmosedale, :dmose)]

I missed comment 19 on my DUPing spree, so it appears that the log in this bug isn't going to help here.  That said, bienvenu mentioned in bug 566655 that the log modules that would be most helpful are:

NSPR_LOG_MODULES=imap:5,imapoffline:5,timestamp

for anyone who wants to help.  

Since shaver and davida have seen this bug and are both on Zimbra, having someone attempt to reproduce against the MoMo test Zimbra instance seems like an extremely valuable endeavor.

Once we can actually reproduce reliably, <http://getsatisfaction.com/mozilla_messaging/topics/imap_draft_composed_offline_deleted_when_going_online-12lf7> suggests that is a regression so adding regression and regressionwindow-wanted keywords.  A good place to start the regression window hunt is likely with the info in bug 397910, since that may well be the same issue.

Comment 24

[Ludovic Hirlimann [:Usul]]

Tested against testzimbra.com - and I get the same results that I reported earlier against gmail. Dmose can you try against's moz's zimbra server ?

Comment 25

[Dan Mosedale (:dmosedale, :dmose)]

I just tested against MoCo's Zimbra instance, and I see the same thing as Ludo.
Which is, copied from bug 566655:

"I'm seeing similar results as standard8 : my draft email disappear for a second
an then they reappear (and status changes from read to unread)."

In other words, we don't yet have a way to reproduce this bug.  This is on Mac.  I have two other things I'm going to try before marking this blocking-.

Comment 27

[David :Bienvenu]

Attachment: fix for windows case

this particular fix only affects windows, because windows is the only nsLocalFile* impl that's caching file sizes, from what I can tell. So it would explain Shaver's problem, but unless davida has come over to the dark side, it wouldn't explain his issue (unless he saw it when the mac nsLocalFile impl was still caching file sizes).

I'll try to do an xpcshell test for this, though offline tests are difficult with xpcshell. But we should definitely get this fix in for 3.1 rc1. And if this bug is in 3.0.x, we should fix that there as well (I suspect it is there, from looking at the code)

Comment 28

[Mark Banner (:standard8)]

Hmm, you confused me, the patch with the tests is on bug 397910.