681734 - Thunderbird 5.0+ fails to send mail - connection to SMTP server timed out

Status: RESOLVED WORKSFORME

(MailNews Core :: Networking: SMTP, defect)

Description

[Michael Greis]

User Agent: Mozilla/5.0 (Windows NT 5.1; rv:6.0) Gecko/20100101 Firefox/6.0
Build ID: 20110811165603

Steps to reproduce:

Upon upgrade from 3.1.11 to 5.0, Thunderbird constantly fails to send mail, reporting an SMTP server timeout.    

Because many of the accounts I am connecting to are on a Virtual Distributed Server running Qmail, I researched that first, even though the problem began only when I upgraded to 5.0.  Watching the Qmail log in real time, there are no messages indicating a successful connection (or a connection of any kind) from my Thunderbird client when this happens.  

Here's what I know after a lot of testing...

- I can't reliably set the server timeout settings in Tbird (using the Config Editor). No matter what I try, it always resets (there are several posts on this).
- The problem is specific to Thunderbird (other clients, like an iPhone or a webmail client) can all connect and send mail with no problems
- Mail sent from other systems to the SMTP server is handled fine
- I now have Thunderbird clients set up on two machines.  The one running 3.1.11 has no problems sending mail of all sizes and types.  The one running 5.0 (and now 6.0) consistently fails to complete sending messages (though once in a rare while, it does succeed).
- Various combinations of SMTP parameters (ports, connection security (None, STARTTLS, SSL?TLS) and authentication methods (None, password, etc.) in Tbird don't seem to make a difference.
- The problem may be specific to a Thunderbird account configured with incoming IMAP e-mail. This seems odd to me.
     - I have one account with connects from Tbird 5 to a POP server which is having no trouble sending mail. I initially thought that might be because it's connecting to an ISP (Comcast) rather than my SMTP server. However, a GoDaddy support person set up a Tbird account on my server with incoming POP and had no problem sending mail. When he set up another account with incoming IMAP, he encountered the timeout problem. 

Mozillazine thread:  http://forums.mozillazine.org/viewtopic.php?f=39&t=2273345


Actual results:

Thunderbird reports..
Sending message....
Connected to mail.xxxx.com...
Sending of message failed.  The message could not be sent because the connection to SMTP server mail.xxxx.com timed out.

Comment 1

[Michael Greis]

I have not been able to reproduce successful connection when defining incoming server as POP instead of IMAP (as reported by GoDaddy tech, above).

Comment 2

[Matthias Versen [:Matti]]

Do you have AV software installed ?
In most cases some AV software intercepts the connection and it's probably the reason for the timeout.

Comment 3

[Michael Greis]

That was suggested in the Mozillazine thread.  Although anti-virus software (AVG) is installed,
on the machine running Thunderbird 5.0, e-mail scanning is turned off.  
Interestingly, e-mail scanning is now active on the system running Thunderbird 3.1.11, 
where there are no problems at all sending mail.

Comment 4

[Matthias Versen [:Matti]]

A SMTP log -> https://wiki.mozilla.org/MailNews:Logging or a wireshark protocol could help.

>Although anti-virus software (AVG) is installed, on the machine running Thunderbird 5.0, e-mail scanning is turned off.

Turning off doesn't help in all cases.
It something on your system that blocks the traffic if you really hit a timeout. The incoming server doesn't matter at all if this Report is REALLY about SMTP. You could only get an error message if the sent message can't be stored in your sent IMAP folder but the error message should be different and the email will be sent anyway.

Comment 5

[Michael Greis]

Matthias, thanks.  I've created three SMTP log files per the instructions.  I'll try to attach all three (they're not long).  

The first tracks a series of 10 messages I attempted to send.  4 succeed and 6 fail.  The log seems to reflect that, with a series of 10 communications, 4 of which are longer and give indications of having sent something (although I certainly can't interpret what the messages mean in any detail).  

The second and third logs contain a single transaction each, with one success and one failure.  I sent virtually the same message both times.  The first 27 lines are identical, then the failure log ends.  

I hope this helps.  I would be happy to do any other testing that would be of assistance.

Comment 6

[Michael Greis]

Attachment: SMTP log 1 of 3 as described in comment - 10 transactions

Comment 7

[Michael Greis]

Attachment: SMTP log 2 of 3 as described in comment - single transaction success

Comment 8

[Michael Greis]

Attachment: SMTP log 3 of 3 as described in comment - single transaction failure

Comment 9

[Ludovic Hirlimann [:Usul]]

(In reply to Michael Greis from comment #5)
> Matthias, thanks.  I've created three SMTP log files per the instructions. 
> I'll try to attach all three (they're not long).  

Thanks.

> I hope this helps.  I would be happy to do any other testing that would be
> of assistance.

I don't seen anything particular in them ,not even the failure in the third one :(
I'm not sure adding time in the log would help, david thoughts ?

Comment 10

[David :Bienvenu]

In the third log, the server never responds to the EHLO comand, which means that the communication between Thunderbird and the SMTP server is failing somewhere in between. It doesn't seem to depend on the IP address sent with the EHLO command, which is always the same. I'd suspect the server or a firewall or some anti-virus/anti-spam program geting in the way.

Comment 11

[Michael Greis]

David,

I can see the logic that another program is getting in the way, but if so, why doesn't this happen on 3.1?  The one very clear break is moving to 5.0 and beyond.  The problem first occurred immediately after I upgraded a client to 5.0.  And I mean immediately.  When I upgraded a second machine to 5.0, the exact same thing happened.  When I took that second machine back to 3.1 ,the problem went away.  

Both machines are talking to the same server.  Both machines are using AVG; in fact, I hadn't noticed, but the e-mail scanner on the one now (successfully) running on 3.1 was ON, whereas the one on the other machine has been always been off.  

Just looking at it from the outside, it seems as if something changed in how Thunderbird talks to the SMTP server to send mail between 3.1 and 5.0, and that change is causing the communication to fail somehow.

Comment 12

[David :Bienvenu]

virus checkers can whitelist programs they know about. So version 3.1 doesn't look like version 5 to the virus checker/anti-spam program. We haven't changed the way we send the EHLO argument between 3.1 and 5. And sometimes it gets through and sometimes not, which is very weird. But TB's not doing anything different.

Comment 13

[Michael Greis]

Just installed Thunderbird 6.0 on yet another machine.  This one has just been rebuilt after a hard disk failure and as yet has no anti-virus software installed.  I'm seeing exactly the same behavior with the connection to the SMTP server timing out.

Comment 14

[David :Bienvenu]

So, router, firewall, things like that would be the next thing to check. If we don't get a response from EHLO, there's not much we can do.

Comment 15

[Michael Greis]

None of those things - router, firewall - appear to be the problem.  I've got two machines running side by side - one running Tbird 3.1, the other Tbird 6.02.  The one running Tbird 3.1 communicates flawlessly to the IMAP servers to send mail.  The one running Tbird 6.02 consistently fails.  

There is something different that was introduced in 5.0.  I haven't a clue what it could be, and I hear what you're saying about not changing anything in 5.0.  But what else could it be?  Are there any other diagnostics that would give more information?

Comment 16

[WADA:World Anti-bad-Duping Agency]

No change in "how TCP connection is established", however, small changes are exists in "how communication, such as StartTLS, login, ..., is done" in some releases. e.g. Tb2's "TLS, if avail" is removed by Tb 3, Tb2's TLS is changed to StartTLS and Tb2's SSL is changed to SSL/TLS at server settings in Tb. UI for server setting etc. is changed by some releases. And, in Tb's SMTP setting, there is problem like next;
  When port number is changed to one for SSL/TLS or StartTLS,
    SSL/TLS or StartTLS choice is not changed according to port number change,
  Or, When SSL/TLS or SSL/TLS choice is changed, port number is not changed
  to default port number according to choice change.

Followin log is seen.
> SMTP Send: STARTTLS
> SMTP Response: 220 ready for tls
> SMTP Send: EHLO [192.168.1.100]
Tb3 or later uses STARTTLS only when StartTLS is chosen at server setting.   
If server supports StartTLS but it's not mandatory(depends on connection security level), and if port for SSL is used, StartTLS happens over SSL connection. In this case, something wrong may occur. 

SMTP server setting in Tb should be same as provider's request for SMTP setting. Is your SMTP server setting in Tb correct? Set as provider requests?

Comment 17

[Wayne Mery (:wsmwk)]

Michael ...

> Is your SMTP server setting in Tb correct? Set as provider requests?

Comment 18

[Michael Greis]

I did check the server settings at the time (Oct/Nov 2011).  I also tried several variations with no success.  In desperation, I switched the MTA on the server from qmail to Postfix.  (That caused other problems, but I managed to resolve those.)  That seemed to address the problem, and I was subsequently able to upgrade to new releases of Thunderbird.

A few months ago, I migrated to a new virtual distributed server.  The new server implemented qmail by default.  So far, however, the problem has not recurred.  I am currently using Thunderbird 16.0.2.