236584 - need to keep alive IDLE connections

Status: RESOLVED FIXED

(MailNews Core :: Networking: IMAP, defect)

Description

[David :Bienvenu]

spun off of bug 141369 - we need to keep-alive imap connections, for folders
other than the inbox. Inbox idle connections can easily be kept alive with check
for new mail, but non-inbox connections can't easily be kept alive (you could
configure mail to check all or individual folders for new mail, but that would
be painful). So, we need to do this on a timer, probably based on the connection
time out for the server (defaults to 29 minutes, but for some servers, can be
less). The timer just needs to issue a NOOP. We could do this from the biff
manager, but then we'd need to keep track of what connections are in the idle
state. We could handle it internally in the imap thread main loop as well by
keeping track of the time we've been idle.

Comment 1

[David :Bienvenu]

keeping in mind that most people only care about the IDLE connection for the
inbox - my server doesn't even seem to generate IDLE responses for folders other
than the inbox.

Comment 2

[Bob Snyder]

I care about IDLE outside of INBOX. INBOX is certainly the priority, but I'd
intuitively expect it to work in any mailbox I'm actively viewing...

Courier-IMAP at least seems to do IDLE outside of the INBOX. I just tested it
with telnet by toggling the flag on a message via a seperate connection:

C: a SELECT INBOX.test
S: * FLAGS (\Draft \Answered \Flagged \Deleted \Seen \Recent)
S: * OK [PERMANENTFLAGS (\Draft \Answered \Flagged \Deleted \Seen)] Limited
S: * 44 EXISTS
S: * 0 RECENT
S: * OK [UIDVALIDITY 1075604011] Ok
S: a OK [READ-WRITE] Ok
C: b IDLE
S: + entering ENHANCED idle mode
S: * 43 FETCH (FLAGS (\Flagged))
S: * 43 FETCH (FLAGS ())

Comment 3

[Ronny Adsetts]

To follow on from comment #2, I'd like to see IDLE work for any folder set to
'Check this folder for new messages' - for those of us doing server side
filtering, pretty much a necessity for making the best IMAP mail client ever
:-). Maybe this would have to be limited by 'Maximum number of server
connections to cache' in server settings? Not sure of the technicalities there...

Comment 4

[David :Bienvenu]

Ronny, that works today, as long as you set your check for new mail interval to
a value less than the server timeout (default is 29 minutes). And yes, you are
limited to the max number of cached connections, because an IDLE connection is a
cached connection.

And in fact, for non-inbox folders not configured for to be checked for new
messages, keeping alive the idle connection is of limited use, because we won't
notify you of new messages in those folders. And also, bear in mind that if we
actually get an idle response before the timeout, the connection will stay alive
because we'll retrieve the new headers, keeping the connection alive. So it's
only connections that don't have any new messages in 29 minutes that will have
the idle connection time out.

Comment 5

[Ronny Adsetts]

OK, that makes sense - thanks for the info David.

I'll give the next reasonably solid thunderbird build a bit of a thrashing then.

Comment 6

[Sam Varshavchik]

Re: comment #4:

Observe that when using IDLE, a "mail check" does absolutely nothing whatsoever,
because as soon as new mail arrives, the client gets notified.  Terminating the
IDLE and then issuing a NOOP will not do anything except in the pathological
situation where the folder is updated immediately after the server exited idle
mode, and before it receives a noop.

The IDLE extension specification requires servers to wait at least 30 minutes
before terminating the connection for inactivity.

Therefore, the easiest apparent solution is to silently enforce an upper limit
(or a fixed limit) of 29 minutes between mail checks when the server supports
the IDLE extension.

Comment 7

[David :Bienvenu]

Sam, if you'll look again at comment 4, you'll see that I say that when we get
notified via idle that there are new messages, we turn around and fetch those
new message headers. That should keep the connection alive.

In any case, I'm a little confused about what you're saying since we'll go out
of idle to issue the noop, fetch the headers, and then go back into idle, which
should restart the timer anyway.

Comment 8

[David :Bienvenu]

Attachment: handle server logging out in idle response

we need to handle the idle connection going bye bye, independent of keeping the
connection alive. This patch is a stab at that, and also attempts to fix a race
condition around m_transport...

Comment 9

[chris hofmann]

Comment on attachment 143716 [details] [diff] [review]
handle server logging out in idle response

a=chofmann for 1.7b

Comment 10

[Frank Wein [:mcsmurf]]

This patch has been checked in, see
http://bonsai.mozilla.org/cvsview2.cgi?diff_mode=context&whitespace_mode=show&file=nsImapProtocol.cpp&branch=&root=/cvsroot&subdir=mozilla/mailnews/imap/src&command=DIFF_FRAMESET&rev1=1.546&rev2=1.547