Closed Bug 196095 Opened 21 years ago Closed 20 years ago

Sending message with IMAP server disconnected produces two error dialogues

Categories

(MailNews Core :: Networking: IMAP, defect)

Product:
MailNews Core
Component:
Networking: IMAP
Platform:
x86
Windows 2000
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: benw, Assigned: Bienvenu)

References

Details

(Keywords: fixed-aviary1.0)

Attachments

(2 files, 1 obsolete file)

fix to silently retry when server disconnects
10.98 KB, patch
mscott
: superreview+
Details | Diff | Splinter Review
missed the idl file changes last time...
12.94 KB, patch
Details | Diff | Splinter Review 
User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3b) Gecko/20030210
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3b) Gecko/20030210

If your Sent folder is on an IMAP server and you spend a long time composing a
message and then try to send it, MailNews throws up two error dialogues about
the server being disconnected.

Reproducible: Always

Steps to Reproduce:
1. Make sure your Sent folder is located on an IMAP server
2. Compose a message and leave the compose window open for more than 10 minutes
(or however long it takes your IMAP server to disconnect)
3. Click the Send button.

Actual Results:  
MailNews throws up two dialogues, the first is an Alert box that reads "Server
<hostname> has disconnected. The Server may have gone down or there may be a
network problem." The second is a Question box that reads "The message was sent
successfully, but could not be copied to your Sent folder. Would you like to
return to the compose window?" The message is sent but not copied to my Sent
folder. I have to click OK, save the message, and then move it to my Sent
folder. I tend to take a long time composing messages so this happens to me
about 70% of the time.


Expected Results:  
Mozilla should reconnect to the IMAP server and copy the message to the Sent
folder. Error dialogue should only appear if there is an error reconnecting.

This is the UW IMAP server.
sounds like your imap server is dropping the connection - it should not do this
in less than 29 minutes, and if it does, it confuses our connection cache.  The
two places that are putting up the error codes don't know about each other, and
reconnecting is difficult in the current code.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Looks like this is caused by my office firewall or something, it doesn't happen
at home. Still, it seems like this should be handled better. Connections will be
dropped for various reasons and this doesn't happen to me in other mailers.
IMAP-server at web.de is timing out in shorter time than 29 minutes, causing
problems with this bug.
dupe of bug 187395?
I'm also encountering this, using MDaemon's IMAP server (not by choice!). I have
moz set to download messages (make avail offline), so its not hitting the server
with every request, and something must be timing out. This only became a problem
with 1.3, it worked fine with 1.2.1 . 

Ideal solution: assume mail servers are buggy/braindead since many are, and
attempt to reconnect silently (perhaps a message in the status bar?). Report an
error only if reconnection fails.
I've worked around this by extending the IMAP session timeout on our mail server
to >30m instead of the default of 10m. Perhaps others might consider this.

Support for IMAP keepalives in mozilla might be a good client-side workaround
for braindead IMAP servers.
we can keep alive a connection if you configure the folder for checking for new
mail, in the folder properties dialog. Then, the folder will be checked for new
mail whenever the normal new mail check is done (e.g., every 10 minutes). So,
you could do this to your sent mail folder.
I can confirm that I get the same behavior as comment #5 (I also use a MDaemon
mail server) and before 1.3 it worked fine.

I gonna try solution provided on comment #7 before changing server timeot
Guy Stalnaker, jstalnak@wisc.edu, University of Wisconsin-Madison

I want to confirm that this behavior exists with Mozilla 1.3 as well, connecting
to IPlanet Messaging Server.  I have verified with our mail guys that there is
no persistant server connection timeout that is configured on the server side.
You should try this with the latest 1.5 trunk build - it check if the connection
is alive before trying to use it, which should help a lot with this problem.
I may say that using moz 1.5a 2003070208 does gives me the same problem. But I
composing mails less that 10 minutes.

Now I'll try to repro it with some testing. 

1 minute - OK.
3 minutes - OK.
does or does not give you the same problem?
Does give, but I leaved the office and the experiment will go on on monday I
believe.

I very often receive the "coping was failed to "Send""...
Our admin has found this information, maybe it helps:

7.19 Why did my POP or IMAP session suddenly disconnect? The syslog has the
message: Killed (lost mailbox lock) user=... host=...
This message only happens when either the traditional UNIX mailbox format or
MMDF format is in use. This format only allows one session to have the mailbox
open read/write at a time. 
The servers assume that if a second session attempts to open the mailbox, that
means that the first session is probably owned by an abandoned client. The
common scenario here is a user who leaves his client running at the office, and
then tries to read his mail from home. Through an internal mechanism called
kiss of death, the second session requests the first session to kill itself.
When the first session receives the "kiss of death", it issues the "Killed (lost
mailbox lock)" syslog message and terminates. The second session then seizes
read/write access, and becomes the new "first" session. 
Certain poorly-designed clients routinely open multiple sessions to the same
mailbox; the users of those clients tend to get this message a lot.
Another cause of this message is a background "check for new mail" task which
does its work by opening a POP session to server every few seconds. They do
this because POP doesn't have a way to announce new mail.

The solution to both situations is to replace the client with a good online IMAP
client such as Pine. Life is too short to waste on POP clients and
poorly-designed IMAP clients. 
This is all very well known. We don't make simultaneous connections to the same
folder - we go to a lot of trouble not to do that. Our very first imap release
6+ years ago sometimes did make simultaneous connections. When we found out that
UW didn't handle that, we fixed it in the 4.5 release, but the UW server person
still likes to claim that we make simultaneous connections all the time.  I
don't believe that's what's happening, unless you have a second client access
the same folders (e.g., Pine :-) )
If it helps all my IMAP accounts are on same server...
Note that this bug deals with the same error messages as bugs:
187395 <http://bugzilla.mozilla.org/show_bug.cgi?id=187395>
and
188004 <http://bugzilla.mozilla.org/show_bug.cgi?id=188004>
I experience the same problem on Mozilla 1.5beta, so please don't close this bug
yet.

The problem is that when any folder has not been accessed for a certain period,
when you try to access it, you get the message "Server xxx has disconnected. The
server may have gone down or there may be a network problem."


When using Mozilla 1.5beta (and all earlier versions) as my IMAP client (courier
IMAP server), I get the message "Server xxx has disconnected. The server may
have gone down or there may be a network problem." After OK'ing this message the
connection is apparently reopened and you can use the folder. The problem is
especially annoying with the sent folder as sending of emails occur without the
message being saved unless you send it several times.

I assume the problem is that Mozilla keeps open connections "cached" and that
mozilla does not notice that the server closes the connection after a certain
time-out period (I seem to recall that the server is required to do this for
IMAP compliance and that the client shouold take notice as the protocol also
defines how the connection is closed). However, I have no way to verify this.

I thought that this could be solved by setting the number of cached connections
to 0 in the advanced server settings, but for some reason mozilla auto
increments this value during use and the error persists. 

The reason for this bug may be the same as Bug#: 115349  
<http://bugzilla.mozilla.org/show_bug.cgi?id=115349>

Timeout can occur not other places in the network, not only in the IMAP server.

If you are sitting behind a firewall running NAT (or rather PAT), all TCP 
connections will eventually timeout and PAT will drop them. If you are running 
Mozilla or Thunderbird at work or in a large organization, I'm 99% sure you 
*will* be behind NAT/PAT (also called IP Masquerading in Linux). The NAT/PAT 
timeout will vary. Here at work, its 20 minutes (configured in the firewall 
globally for all TCP connections)

For that reason, it is imperative that some sort of keep-alive polling takes 
place. Workaround #7 (configuring the "Sent" folder for checking for new mail) 
works for me in avoiding this bug, but it has a nasty side effect: After 
having sent an email, it gets copied to "Sent" properly, but after a few 
minutes, I get a notification that there is new mail. And of course there is, 
in Sent, which I have chosen to check for new mail!

So workaround #7 works somewhat - the sent email is copied to Sent, but it 
isn't a proper workaround.

Just being able to tweak the "30" in 30 minutes as a configuration option 
allowing it to be less than the PAT timeout would probably solve this problem 
for most....
From comment 15:

"When we found out that UW didn't handle that, we fixed it in the 4.5 release,
but the UW server person still likes to claim that we make simultaneous
connections all the time."

Mozilla does make multiple connections to the same folder when you have two
instances open. Having the same folder open in multiple clients is (for me) the
biggest advantage of using IMAP. Problem is that this just doesn't work with
Mozilla. :\

This seems to be related to bug #29782 and to:
http://www.washington.edu/imap/IMAP-FAQs/index.html#7.19
and
http://www.washington.edu/imap/listarch/1997/msg02344.html (indeed, an email
from 1997 :\). 

I really hope that Mozilla and UW are not going to keep pointing fingers to each
other.
please read comment 14 again. This comment says that a single instance of the
netscape/mozilla imap client makes multiple simultaneous connections to the same
mailbox. I'm saying that's not true.  I agree that we should handle getting
disconnected more gracefully, but I'm saying that simply running a single
mozilla client on a single machine should not produce this error, at least as a
result of the mozilla client making simultaneous connections.
Sorry, I should have been clearer about that..

I'm not talking about one mozilla instance opening multiple connections to one
mailbox. I haven't seen that (yet ;)). 

Most problems occur when there are two or more instances of Mozilla accessing an
IMAP mailbox. I'll see if I can find a bit more information about a possible
solution to this problem. It should be possible to access one mailbox from
multiple clients, it's just not working properly with Mozilla at the moment
(well, hasn't been working properly for the past decade or something close to
that ;)).
Hi guys.

I know this bug since ~1 year and i always thought that it is caused by my
webserver. Now i think i learned how to admin the server and started to host
imap/tls boxes for other guys.

I got really annoyed when i recognized that others did not have this problem. So
i spend quite a lot of time to find the cause of it.

Reproducing the problem is really easy:
Open some IMAP folders ... wait 10 minutes ... open them again: Voila "Disconnected"
And yes you are right: the most annoying thing is that he loses his connection
while you are busy with writing emails => Nothing could be saved inside the Sent
folder.

I tried to find the Reason with sniffers, I updated to courier 2.2 ...
I also tried all kinds of Mozillas (1.5, 1.6, 1.7a, Thunderbirds, Netscape ... )

The problem still existed.

But tomorrow morning i head an idea: Tunnel
Simply establish a tunnel between localhost:993 and webserver:993. I used putty
- it has the ability to send keepalive Messages. I tried it with 60sec keepalive.

And voila the problem is gone.

I know this workaround does not help to solve the problem itself, because mostly
the box owners have no ability to establish such a tunnel. And when your server
has such a ability it might be a security risk because putty sessions need a
running Shellacc on the server. (Maybe Stunnel might work, but then there is the
problem, that the server must have Stunnel enabled).

I think mozilla simply needs the possibility to activate keepalive messages. 

A simple option like:
user_pref("mail.server.server1.keepalive", 60); # keepalive time in seconds
would be enough to solve the problem. 

A more intelligent way would be:
When mozilla recognizes regular disconnections: Autoenable keepalive

regards
Cybi
(In reply to comment #24)
> 
> I tried to find the Reason with sniffers, I updated to courier 2.2 ...
> I also tried all kinds of Mozillas (1.5, 1.6, 1.7a, Thunderbirds, Netscape ... )
> 
> The problem still existed.
> 
> But tomorrow morning i head an idea: Tunnel
> Simply establish a tunnel between localhost:993 and webserver:993. I used putty
> - it has the ability to send keepalive Messages. I tried it with 60sec keepalive.
> 
> And voila the problem is gone.

I'm sure you have a firewall that does masquerading inbetween 
you and the imap server, right?


> I think mozilla simply needs the possibility to activate keepalive messages. 
> 
> A simple option like:
> user_pref("mail.server.server1.keepalive", 60); # keepalive time in seconds
> would be enough to solve the problem. 

I think 5 Minutes keepalive as default should be more nice to the 
servers out there.

> A more intelligent way would be:
> When mozilla recognizes regular disconnections: Autoenable keepalive

I would definitely vote for inclusion of such a feature.


(In reply to comment #25)

> I'm sure you have a firewall that does masquerading inbetween 
> you and the imap server, right?

Im not sure, but i think so. (I have no clue which techniques are used by my
provider. But i think there might be one hop doing masquerading.)

> I think 5 Minutes keepalive as default should be more nice to the 
> servers out there.
> I would definitely vote for inclusion of such a feature.

Yes 5 Minutes should be a fine value.

I also tried to find some kind of Workaround. SSH Tunnels can send keepalive
NULL Packages .. Perfect, but not valuable for IMAP Connections.

STunnel: Really easy to establish the connection, but STunnel lacks on keepalive
features.

TCP KeepAliveInterval / KeepAliveTime ... Sounds perfect, but Microsoft
Knowledge Base says that the application itself must enable the keepalive
feature. (Default value would be 2h between keepalive packages.)

cybi
I see exactly the same problem using thunderbird 0.5 on linux at home where I'm
behind a router/firewall and connected to the Univ. of Florida CISE dept. IMAP
mailserver through a cable modem. I *do not* see this problem when I'm in my
office and also using thunderbird 0.5 on linux and directly connected to the UF
CISE IMAP mailserver.

So far, the only obvious way of overcoming this problem is to save the message
being composed, reconnect to the IMAP server and then reopen a Compose window
and send the message. The idea of asking the Sent folder to check for new mail
is quite cool and I'm going to try it.

It would be nice if thunderbird/mozilla could silently reconnect to the IMAP
server when the connection is dropped.
I've only ever had a single message: "The message was sent
successfully, but could not be copied to your Sent folder. Would you like to
return to the compose window?". I've had this problem for months.

On 1.7 RC1, things suddenly seem better (fewer messages of this sort), but today
I did get a new but similar dialog box. It asked whether I want to try again to
save the message. Unfortunately this didn't get me anywhere and seemed to hang
Mozilla, which is worse than 1.6. The get-out previously was to save to
templates (which I store locally), quit Mozilla, go back into Mozilla and
manually move the template to my sent folder. If I don't quit Mozilla at that
point, the manual move action from the templates to the sent folder appears to
work but actually doesn't.

Using Win98.

Alan
This appears to be a possible duplicate of bug 89285, which no-one seems to have
mentioned. That one is now "fixed" with the comment that someone should reopen
it as a new bug (this one?) if it's still a problem. But it is still a problem
on 1.7rc1.
This bug clearly describes what I keep seeing (with 1.7 final on both Linux and
Windows). Connections to the send mail folder often fail; toggling the
online/offline control twice and pressing "Retry" allows them to succeed.

So, a fix would be to do whatever toggles online/offline twice and then
retrying, before popping up that error message :-)

Gerv
This fix makes it so we'll silently try to reconnect if the server disconnects
while we're running a url. It's a bit hard to test, but I've used a program
that terminates tcp connections to disconnect after we've decided that a cached
connection is alive, and it basically seems to work. I've tried a few scenarios
- selecting a message, selecting a folder, and doing an fcc, and they all
seemed to work.
Comment on attachment 158573 [details] [diff] [review]
fix to silently retry when server disconnects

wrong patch
Attachment #158573 - Attachment is obsolete: true
Attachment #158577 - Flags: superreview?(mscott)
Attachment #158577 - Flags: superreview?(mscott) → superreview+
fixed on trunk - will let bake a while before checking in on aviary branch.
Status: NEW → RESOLVED
Closed: 20 years ago
Resolution: --- → FIXED
*** Bug 218100 has been marked as a duplicate of this bug. ***
this is what is checked in, in case we try to port this to 1.7 or something...
Keywords: fixed-aviary1.0
David, you introduced an extra semicolon, causing gcc 3.4.2 bustage:
/mozilla/aviary/mozilla/mailnews/imap/src/nsImapUrl.cpp:1348: Fehler: extra `;'

You fixed it on the trunk, but not on the aviary branch:
http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/mailnews/imap/src/nsImapUrl.cpp&rev=1.164.2.1.4.3&mark=1348#1344
*** Bug 258024 has been marked as a duplicate of this bug. ***
*** Bug 55049 has been marked as a duplicate of this bug. ***
Product: MailNews → Core
*** Bug 238831 has been marked as a duplicate of this bug. ***
Is the fix for this bug part of any released version of Thunderbird yet?
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: