213827 - IMAP offline stores are deleted when connecting to server fails

Status: RESOLVED FIXED

(MailNews Core :: Backend, defect)

Description

[Marcus Groeber]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624

When connecting to an IMAP server fails, all offline stores are removed without
comment, losing all data in them.

In particular, this recently caused me to lose a mail stored in an IMA Drafts
folder that was composed while in "Work Offline" mode after I restarted Mozilla
in online mode on a connection that had no access to the IMAP server.

Reproducible: Always

Steps to Reproduce:
1. Create an IMAP account
2. Mark some folders for download
3. End Mozilla
4. Run Mozilla again in a situation where the IMAP server is not accessible or down

Actual Results:  
- An error message appears that connecting to the server failed.
- All folders under the IMAP account have disappeared, and their stores have
been removed

Expected Results:  
- An error message appears
- The folders and any headers in them are still available
- An error occurs when accessing any mail that has not been made available for
download

This looks somewhat related to bug 142981 and bug 191990 - these seem to
describe other manifestations of the same effect.

Comment 1

[Jean-Marc Desperrier]

I've met this problem a number of time even if I had no time to enter the bug.
Can't find a duplicate, even thought I expected there would be some :

I'd change the reproduction steps a bit :
Steps to Reproduce:
1. Create an IMAP account
2. Mark some folders for download
3. Click on the left of the folder name to close the view of the subfolders
4. Kill the network connexion (unplug the cable, desactivate it)
5. Click on the left of the folder name to open the view of the subfolders
6. Mozilla gives you an error and there are no subfolder visible anymore for the
account.
7. Restarts your network connexion and click to see the account content

Result :
When connexion is available again, Mozilla redownloads the whole content of the
imap account. It erases local stores, and then starts downloading the message
from the corresponding folders again.

Comment 2

[David :Bienvenu]

That's very bad. What's happening is that when folder discovery is done, we're
deleting the "undiscovered" folders and their offline stores. But that code is
supposed to know that folder discovery failed, and not do that. It's probable
that the network error is not getting through to that code.

Comment 3

[David :Bienvenu]

this worked fine for me (the first set of steps). We never get to the code that
deletes undiscovered folders unless the following condition is true:

    if (!DeathSignalReceived() && (GetConnectionStatus() >= 0))

The second condition means we were able to establish a connection to the server.

The second set of steps are more likely to cause the problem, because we've
already had a connection to that server, and we will re-use it when we try to
rediscover folders. However, in recent 1.5 builds, I added code that detects if
a cached connection is no longer valid, and that should help. Could someone try
a recent 1.5 nightly build? The first set of steps could conceivably cause the
problem if you turned on quick launch, and somehow shutting down partially
didn't close open imap connections...

Comment 4

[Marcus Groeber]

I am definitely seeing the problem I described in the first set of steps every
time with the 1.4 milestone.

In my case, this is caused by using Mozilla to access a corporate IMAP4 server
(Exchange 2000) behind a firewall, so when I dial up to the Internet from home
and try to open the folder tree for that account by accident (a step which I
forgot in my original list), connecting to that server fails, and all the
folders disappear.

The last time I saw this I had definitely stopped Mozilla and even logged off my
NT session since last connecting successfully.

Comment 5

[David :Bienvenu]

two questions:

have you tried a 1.5b build?
what does this mean, exactly? "try to open the folder tree for that account by
accident" - did you try to expand the server hiearchy by clicking on the twisty
next to the account? That might go through a different code path - we do try to
rediscover folders in that case, I think...

Comment 6

[Marcus Groeber]

I haven't tried a 1.5b build yet - I'll try that on Monday when I am back behind
the firewall.

Yes, I mean clicking on the "twisty" (if that's what the triangle next to the
folder name is called). Thinking about it, I have so far not noticed the folders
disappearing when just starting up Mozilla outside the firewall with the folder
tree open, even though I also get the "failed to connect" error.

This would also match the second set of steps given in comment #2 that
explicitly calls for closing and opening the subfolders.

Comment 7

[Jörg Thönnes]

Mozilla 1.3

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.3) Gecko/20030312

I also have the problems described below. I am connecting via IMAP/S to web.de.
Sometimes, all folders are re-built and all already downloaded messages are
reloaded. This is annoying and time wasting.

What is making this really annoying is the fact, that I cannot see neither the
headers nor the bodies until the complete offline cache has been refreshed. This
 is probably another bug (or RFE).

Up to now, I thought that this is connected to changes to folders using the web
front end (some UID issue). But as I read these reports, I also notice that the
reloading happens after some server connection errors.

Will this be fixed in the 1.4 branch?

Comment 8

[David :Bienvenu]

It would be much more likely that this would be fixed in the 1.4 branch if
someone who has this problem could say that it's fixed on the trunk, say in a
1.5b or final build.

Comment 9

[Marcus Groeber]

Mozilla 1.5b

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5b) Gecko/20030827

I can still reproduce this with 1.5b. It took me several attempts of opening and
closing the folder tree of the IMAP4 account via the twisty, and clicking on
various folders (each time receiving an error message "Failed to connect to
server").

Eventually, after opening the folder tree one more time, all the folders were
gone as reported before.

[Note: I did all this while being in an environment where connecting to the
server fails because the name cannot be resolved.]

Comment 10

[Vaclav Dvorak]

I'm seeing this on the Mozilla 2004112206 nightly build that I'm using for some
reason that I can't remember, on WinXP. Very, very annoying. Will try a current
nightly. Is there anything I can do to help debug this?

Comment 11

[Vaclav Dvorak]

*** Bug 264658 has been marked as a duplicate of this bug. ***

Comment 12

[Markus Schiefer]

I am experiencing this bug for about 2 years now... at least versions 1.4 and 1.5 have it. And it looks like there are many duplicates for this bug...

Comment 13

[Ray Booysen]

I experience this at times too.  I saw this a lot in the 1.0.x branch but nothing so far in the latest nightlies.  20051123

Comment 14

[Tom Eicher]

Also happens with version 1.5 (20060119) on linux.

(In reply to comment #5)
> what does this mean, exactly? "try to open the folder tree for that account 
> by accident" - did you try to expand the server hiearchy by clicking on the 
> twisty next to the account? That might go through a different code path - 
> we do try to rediscover folders in that case, I think...

Yes, this is what I do: expand the account by clicking on the twisty. All for a sudden, the folders and their local stores are gone.

Comment 15

[Jo Valentine-Cooper]

I've been experiencing this bug for a while now with Thunderbird. It has driven me quite mad, because my work IMAP server in question (Exchange 5.5, if you're curious, though it's not entirely relevant) doesn't seem to want to store labels on the server itself (which I'm pretty sure is an Exchange issue rather than a mozilla issue), so any time this bug manifests I lose all my labels. Given that I use those labels as a pseudo-todo list, this makes for ugly dataloss and leads to Unkind Thoughts.

For what it's worth, I'm on WinXP and I stopped trying to upgrade after v1.0.8 because I didn't want to go to the trouble if the bug wasn't fixed (I have quite a few old unupdated extensions I can't find replacements for...).

Comment 16

[Vaclav]

There is still this problem in Thunderbird version 1.5.0.9 (20061207).
Once the imap server gets unavailable, all the offline content locally stored by thunderbird is lost.
Offline synchronization has to be started from beginning.
This is a big problem for me - offline content has total volume about 2GB.
Thanks.

Comment 17

[Gearry Judkins]

I can also reproduce this problem in Thunderbird 1.5.0.9 connecting to an Exchange 5.5 server or a Courier IMAP server (not sure of version at the moment, but whatever is in Debian stable).  I have the same problem with Exchange 5.5 as Jo reports in commment 15.  Like other users this problem has been particularly troublesome.  I am not really qualified to offer a patch, but I do have linux, mac , and windows test environments.  If there is anything I can do to help move a fix for this forward, please let me know.  I have not been involved with running nightly builds, but I would be willing to get involved to get this fixed.

Comment 18

[Frank Sauer]

The problem persists in 2.0.0.4. It does not happen every single time though. But there is a good chance that in one out of three connection failures all your mail is lost. This definitely has to be fixed!

Comment 21

[Magnus Melin [:mkmelin]]

This needs sorting out for thunderbird3's better offline imap story.

Comment 24

[Wayne Mery (:wsmwk)]

David, what is next step and what likelihood this may already be improved on trunk?  

"Offline" has lots of reported bugs, so we'll triage and Emre is looking for potential impact on "better faster imap".

Perhaps Steve and Jerome from the recent dupes can help run down leads.
=> dataloss  (eg bug 400624 comments about losing tags)

Comment 25

[David :Bienvenu]

I see an issue in the case of collapse+expand the imap account in the folder pane, and I'll look into fixing that. I would be interested if anyone still sees an issue just on normal folder discovery on startup, as opposed to the case where you collapse+expand the folder hierarchy (which goes through a different code path)

Comment 26

[David :Bienvenu]

Attachment: fix for collapse+expand case

This fixes the case of collapsing and expanding the server, when not connected to the network. I was able to reproduce the loss of the offline store in that case w/o the patch, and didn't lose the offline store w/ the patch.

Comment 27

[David :Bienvenu]

fix checked in - changeset 86:06a64bb4ea73

Comment 29

[Dominique Kaiser]

From the description I'm not sure if this is a 100% duplicate.
Bug 191990 occurs also if you can indeed connect to the IMAP server and successfully login to it. But after logging in no IMAP folderrs are found because of some internal blocking/locking/<whatever> issue within the IMAP server
(For example caused sometimes with GMX email accounts when one is simultaneously logged into the web interface as well - or with MS Exchange server, but not sure what causes it there)

Can it somehow be confirmed that the fix implemented for 213827 also applies to this scenario (I hesistant to install a daily build on my productive machine with thousands of emails in various folders...)?

Comment 34

[herve cimadomo]

i use thunderbird 3.0.15 and this bug is not resolved. When there is a temporary network probleme, if i try to get new message, i have ""Failed to connect to
server" and i lose all offline folder so it's need to reload all offline folders (many Go ...). Thank to reconsiderate it.

Comment 35

[Ludovic Hirlimann [:Usul]]

(In reply to comment #34)
> i use thunderbird 3.0.15 and this bug is not resolved. When there is a

This doesn't exist.

Comment 36

[herve cimadomo]

sorry, i've mistaked with firefox. The version of thunderbird i use is 2.0.0.23( build 20090812)

Comment 37

[Magnus Melin [:mkmelin]]

This is only fixed for thunderbird3.