817008 - Imap idle for incoming mail works for short time then stops working

Status: RESOLVED INVALID

(MailNews Core :: Networking: IMAP, defect)

Description

[Mike Cloaked]

User Agent: Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.95 Safari/537.11

Steps to reproduce:

Set up mail account accessing incoming imap server via starttls - and set to no polling interval.  This is for an arch linux system running 64 bit Thunderbird. 


Actual results:

Initially imap push mail comes in as expected - but after some period any new mail at the server no longer appears in the inbox until the inbox folder is "clicked" again at which point new mail comes in again just fine.


Expected results:

Imap push mail should continue to arrive no matter how long Thunderbird is left unattended but running. The same setup using other mail clients works fine so I guess this is a bug in Thunderbird?

Comment 1

[Mike Cloaked]

I changed the setting to poll every two minutes on the same imap server and it also stops collecting after some period.  However connecting to the same email incoming server via pop works fine when polling.  I would appreciate it if any other user could confirm this behaviour in an independent test?  Thanks.

Comment 2

[Mike Cloaked]

It seems this may be a long running saga:

https://bugzilla.mozilla.org/show_bug.cgi?id=468490

Comment 3

[Mike Cloaked]

I have just found this report too:
https://bugzilla.mozilla.org/show_bug.cgi?id=589731

I will have to check this since I certainly have the situation that I connect to an external imap server but have filters to move mail to an internal imap server in my own machine and highlight the inbox on my internal imap rather than the external imap server.

I will test and report back if behaviour is different if I leave the external imap server inbox as the current folder.

Comment 4

[Mike Cloaked]

It makes no difference if Thunderbird is left with the incoming inbox imap folder as current - it still times out and mail stops arriving.  So this needs to be fixed.

Comment 5

[Mike Cloaked]

Further interesting tests this evening:
1) Connecting to gmail imap continues to work and does not appear to timeout.
2) The server I am connecting to with Thunderbird does time out - mail.demon.co.uk
3) K9Mail on my android phone connecting to mail.demon.co.uk happily continues to receive mail all day and night.

How can I test this further to find what the problem is with the mail.demon.co.uk server when connecting using Thunderbird?

Comment 6

[Mike Cloaked]

I have been running more tests this evening - I changed the connection port to port 993 (tls/ssl) and imap idle now seems to work!  I also checked my connection to gmail where imap idle was working and I was connecting to port 993.

The server that is problematic for me is a Microsoft Exchange IMAP4 server - and this seems to misbehave when connecting TB on port 143 (STARTTLS).

On the other hand if I connect to a Dovecot server that I am running at home on port 143 with STARTTLS then it continues to work with imap idle indefinitely.

So the problem seems to be dependent on which server one is connecting to! Also for android K9mail which continues to work on port 143 it sends an idle refresh every 24 minutes by default.

Does Thunderbird do an idle refresh periodically or not?  How can I find out Thunderbird's behaviour in this regard?

Comment 7

[David Lechner (:dlech)]

Mike, you can log your imap traffic and try to catch it in the act. 

See https://wiki.mozilla.org/MailNews:Logging 

If you get a log where the idle connection drops out, add it to the bug as a text attachment.

Comment 8

[Mike Cloaked]

Thank you David - I didn't know about these logging settings - I will not have free time to run some logs till tomorrow but I will switch on logging then, and try to capture log files to provide some diagnostics to get some pointers.

Comment 9

[Mike Cloaked]

I found some time today to run a test - I updated Thunderbird to yesterday's version and switched on logging and then started Thunderbird - running with the incoming server on starttls on its default port of 143 - and strangely it is working today! The previous version was from the 27th of November and was not working!

Anyway I looked through the log file generated and there are some lines part way down the log file which give failure:

1169983296[7fcc45927370]: considering playing queued url:imap://[e-mail]:143/fetch>UID>/INBOX>146
1169983296[7fcc45927370]: creating protocol instance to play queued url:imap://[e-mail]:143/fetch>UID>/INBOX>146
1169983296[7fcc45927370]: proposed url = INBOX folder for connection Sent has To Wait = FALSE can run = FALSE
1169983296[7fcc45927370]: proposed url = INBOX folder for connection INBOX has To Wait = TRUE can run = FALSE
1169983296[7fcc45927370]: failed creating protocol instance to play queued url:imap://[e-mail]:143/fetch>UID>/INBOX>146

I have put the user as xxxuserxxx for security reasons.

However using the version:

Mozilla/5.0 (X11; Linux x86_64; rv:19.0) Gecko/20121130 Thunderbird/19.0a2 ID:20121130042002

I am not now seeing the problem - so perhaps it was a problematic build of 19a2

I will continue to test over the next few days - if the newer build, and builds for the coming week continue to behave normally then I will close this report.

Comment 10

[Mike Cloaked]

I have been doing further testing - my conclusion that it was working was wrong - because enough mail was arriving at the server to keep the connection alive!  It seems that according to the idle definition the server can disconnect after a timeout period (typically of order 29 minutes) and that the other MUAs that I have been using do issue a refresh in the form of a NOOP before the timeout occurs so that the idle connection is refreshed and the server does not disconnect the imap connection.  It does seem that Thunderbird does not do that!  Hence if an imap connection in the idle mode has no activity for a period beyond the server timeout that has been set then it docsonnects and no further email comes in until the imap connection is refreshed. (so Thunderbird does not have any control unless I missed an advanced setting somewhere to issue a NOOP at a specified frequency) - the definition is discussed in https://bugzilla.mozilla.org/show_bug.cgi?id=468490 but maybe the NOOP refresh was never implemented?  It would be nice to know what the current status is for this provision in both the current release as well as upcoming aurora, central etc?

Thanks