250004 - Mail folders never synchronise: SSL Failure / Server xxx has disconnected. The server may have gone down or there is a network problem

Status: RESOLVED FIXED

(MailNews Core :: Networking: IMAP, defect)

Description

[Graham Leggett]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7) Gecko/20040623
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7) Gecko/20040623

Upgrading to Mozilla v1.7 causes users IMAP mailboxes to break.

- Start Mozilla/Thunderbird mail
- Immediately, an error message "Server xxx has disconnected. The server may
have gone down or there is a network problem" appears. Click on ok to confirm.
- Open an IMAP mailbox over an SSL connection with many folders over an ethernet
LAN.
- Select a mail folder (not Inbox, Sent, Drafts, or Trash).
- If Mozilla tried to download the messages, select the folder below it until...
- A message "connected to <mailserver>... appears and MailNews appears to not do
anything. No messages are downloaded / the mail folder is not synchronised, the
throbber freezes. It appears to do this after the initial 5 cached IMAP
connections are used up, but sometimes it does it after only 1 connection is used.
- Continue selecting further mail folders. The same "connecting to
<mailserver>"... appears at the bottom of the pane. No mail is downloaded, the
throbber remains inactive.
- Sometimes the message appears on each attempt to connect to the mailserver:
"Server xxx has disconnected. The server may have gone down or there is a
network problem"

At this point Mozilla/Thunderbird is useless - the user cannot synchronise and
therefore read any mail folders. As a result we have temporarily had to move the
users back to Outlook - thus the blocker status on this bug.

Further investigation reveals that if the IMAP server is an SSL server on port
993, the message "server has disconnected" appears on each attempt to select a
mailfolder.

If the IMAP server is a non secure IMAP server on port 143, no "server has
disconnected" message appears, however all the other symptoms are the same: no
synchronisation is possible with the mailserver.

This problem is evident in Mozilla v1.7 for WinXP, Mozilla v1.7 for MacOSX and
Thunderbird 0.7.1 for MacOSX.

We have checked - the folder in MailNews that show empty have definitely got
messages in those folders on the IMAP server.

This represents major breakage of Mozilla's IMAP support, and has effectively
halted our plans to roll out Mozilla/Firebird to replace the security swiss
cheese that is Outlook :(

IMAP server is courier-imap.


Reproducible: Always
Steps to Reproduce:
xxx

Comment 1

[Graham Leggett]

Further investigation.

Turning SSL off (in other words IMAP over 143) allowed one user to synchronise
their mail folders with "offline -> download sync now". This is not possible
with SSL on (the "Server xxx has disconnected" message appears, and the attempt
to synch is aborted).

Another user, the same attempt was made to synch the mail folder by turning SSL
off. In this case, on startup, Mozilla says "connected to <xxx>..." and remains
in this state permanently. Attempts to synch the mail folders are ignored (they
are probably being queued behind whatever Mozilla is trying to do when it
started up). Clicking on "stop" and selecting synchronise again bring the
browser back to the "connected to <xxx>..." state.

In the non SSL case, it is still not possible to synchronise mail folders by
selecting them. An empty mail folder (from Mozilla's perspective) remains empty
regardless of whether there is mail in the folder on the IMAP server, and
whether you select a folder and go back to that folder.

It seems from the above that there are a number of bugs at work here:

1) Complete SSL failure. Mozilla v1.7 / Thunderbird v0.7.1 becomes unoperational
when SSL over IMASP is switched on. Mozilla v1.6 - this worked fine.

2) Flaky message downloads / imap synchronisation. Mozilla v1.7 sometimes seems
to synchronise, sometimes it does not, choosing to hang instead.

Comment 2

[Graham Leggett]

Further testing:

If the ImapMail directory is deleted, Mozilla downloads mail from scratch in
each folder correctly. After this, resynching mail folders goes back to the
original behaviour of not synching each folder with the server properly.

Comment 3

[David :Bienvenu]

is your courier server configured to limit the number of connections to 4 per
user? If so, you can either reconfigure your courier server to allow more
connections, or change thunderbird to only cache 4 connections through the
advanced imap server settings dialog in mail & news account settings.

Comment 4

[Graham Leggett]

Just checked - it is limited to 4 per server. Changed it to 32.

Now when you select each folder MailNews says "Opening folder...". The throbber
stays quiet. There is no evidence that Mozilla is doing anything.

Set SSL back on - it works!!!

Ok, based on this, it seems the bug is in the handling of the situation where
the server refuses to let the client have more connections.

In the SSL case, the message "Server has disconnected..." appears. This message
should say "Server xxx has disconnected. Please make sure your imap server will
allow you to make XXX concurrent connections to the server, or reduce the number
of cached connections to use." (or some better message that explains what is
going on). "The server may have gone down or there is a network problem" is a
completely meaningless and misleading statement.

To fix the non SSL case, can the "contacted <xxx>..." message at the bottom of
the screen be changed to "waiting for imap connection to become available..."
while that particular folder is waiting?

There seems to be very little feedback from Mozilla about what it is doing most
of the time. The throbber is sometimes off while Mozilla is busy, and the
messages at the bottom of the pane disappear too easily (or are not restored if
the mouse moves over a link on the screen by accident).

Proper feedback from Mozilla will avoid us being sent on wild goose chases for
an entire afternoon with a company director over one's shoulder saying "get it
off my machine and put something on that works"...

:(

Comment 5

[David :Bienvenu]

>This message should say "Server xxx has disconnected. Please make sure your
imap >server will allow you to make XXX concurrent connections to the server, or
reduce >the number of cached connections to use." 

I'm sorry about this difficulty. We used to say almost exactly this (I had to
read it twice to make sure you weren't describing what we used to say) but this
stopped working when we switched to synchronous IO. So I either need to switch
the default number of cached connections to 4, or figure out how to get the code
that put up that error message working again with synchronous IO. I used to have
access to a test courier server that I could try this out on, but I don't have
it anymore. I can try to get access to it again...

Comment 6

[David :Bienvenu]

Attachment: potential fix

I don't have a courier server to test this against, but it might work...

Comment 7

[David :Bienvenu]

Attachment: better fix

thx to Hans, I now can test this - I needed to add a case for
NS_NET_ERROR_INTERRUPT, and it works...

Comment 8

[(not reading, please use [e-mail] instead)]

Comment on attachment 152862 [details] [diff] [review]
better fix

a/r=sspitzer, sorry for the delay

Comment 9

[David :Bienvenu]

*** Bug 247407 has been marked as a duplicate of this bug. ***

Comment 10

[Hans Deragon]

Anybody can tell in which version of Mozilla will this fix be part of?  If there
is supposed to be a way to figure this out by ourselves, please explain because
I do not know how to tell by looking at the bugzilla page.

Comment 11

[David :Bienvenu]

we use keywords to indicate if a fix has been checked into the various branches
(e.g., fixed-aviary1.0, fixed-1.7.3) This fix really only puts up an error
message in this case - you still need to reduce your number of cached
connections in the client, or increase the number of the server.

Comment 12

[Mike Kaply [:mkaply]]

Comment on attachment 152862 [details] [diff] [review]
better fix

a=mkaply for 1.7.3

Comment 13

[Hans Deragon]

From your previous comments, I assume that this bug has been solved in 1.7.3.  I
am running 1.7.3 and I suffer of this problem.  Please reopen.  If this bug fix
has not made it into 1.7.3, please specify HOW I can tell which version of
Mozilla this bug will be corrected.  "fixed-aviary1.0" does not tell me
anything.  1.7.3, 1.7.4, etc... that, tells me something.

Currently, with 1.7.3, I can systematically get the problem where I get the
"Server xxx has disconnected. The server may have gone down or there is a
network problem".  My IMAP server is set to serve 4 connections while I have set
my accounts to 3 in Mozilla.  Thus is theory, it should always work, but it does
not.

If you need an imap server with test account, I can provide you one for testing.
 I have 2 imap accounts created for this purpose.

Comment 14

[Bill]

This is a CURRENT (29Aug2018) problem with Thunderbird Daily, v63.01a. So after 14 years, it still has not been fixed?