1862409 - SMTP Error 421 - Connection timed out

Status: RESOLVED DUPLICATE of bug 1854567

(MailNews Core :: Networking: SMTP, defect)

Description

Attachment: error.png

Another user of my add-on Mail Merge reported a problem, when sending multiple messages, after upgrading from Thunderbird 102 to Thunderbird 115.

The reused SMTP connection is closed after exactly 10 minutes by the server and the following error is reported - also see error.png:

mailnews.smtp: Sending message <xxx> SmtpService.jsm:88:18
mailnews.smtp: Reusing a connection SmtpClient.jsm:130:19
mailnews.smtp: C: MAIL FROM: <xxx> BODY=8BITMIME SIZE=1095 SmtpClient.jsm:624:19
mailnews.smtp: S: 250 2.1.0 Sender OK SmtpClient.jsm:430:17
mailnews.smtp: MAIL FROM successful, proceeding with 1 recipients SmtpClient.jsm:1191:17
mailnews.smtp: Adding recipient... SmtpClient.jsm:1196:17
mailnews.smtp: C: RCPT TO: <xxx> SmtpClient.jsm:624:19
mailnews.smtp: S: 421 4.4.1 Connection timed out. Total session duration: 00:10:09.1333795 SmtpClient.jsm:430:17
mailnews.smtp: Command failed: 421 Connection timed out. Total session duration: 00:10:09.1333795; currentAction=_actionRCPT SmtpClient.jsm:546:19
mailnews.send: Sending failed; An error occurred while sending mail. The mail server responded: Connection timed out. Total session duration: 00:10:09.1333795. Please check the message recipient "[e-mail]" and try again., exitCode=2153066783, originalMsgURI= MessageSend.jsm:337:32 undefined ExtensionCommon.sys.mjs:812
mailnews.smtp: Socket closed. SmtpClient.jsm:518:17

I already mentioned this particular problem briefly in Bug 1854567 Comment 26, but it took a little while to help the user to test the try build from Bug 1854567 Comment 24 and create a SMTP log. The email server in this case seems to be related to Microsoft Exchange (smtp.office365.com and outlook.office365.com).

Bug 1854567 is about a limitation of the number of messages in a reused SMTP connection. This bug is about a limitation of the total session duration of a reused SMTP connection.

We already did two tests with the try build from Bug 1854567 Comment 24: Using the default setting mail.smtpserver.default.max_cached_connections = 3 did not resolve the problem. (I guess this was to be expected.)

Setting mail.smtpserver.default.max_cached_connections = 0, i.e. a single connection for each single message, threw another type of error after about 25 messages:

Send Message Error
Sending of the message failed.
The message could not be sent because the connection to Outgoing server (SMTP) smtp.office365.com timed out. Try again.

The user then had to re-login into the Microsoft account. (Though I am not sure, whether this other error might be related to the try build interfering with the profile of the normal Thunderbird 115.)

I am still hopeful the discussed further changes in the now updated patch in Bug Bug 1854567, i.e. handling 4XX errors gracefully, will resolve this problem as well.

Comment 1

[gene smith]

Alexander,
The patch I've been working on, as is, won't fix the timeout after 10m error you show on comment 0 since it occurred during the RCPT TO response. I might be able to also handle some errors there, so I'll look into it.

However, you then say:

Setting mail.smtpserver.default.max_cached_connections = 0, i.e. a single connection for each single message, threw another type of error after about 25 messages:
Send Message Error
Sending of the message failed.
The message could not be sent because the connection to Outgoing server (SMTP) smtp.office365.com timed out. Try again.

Is this by the same user to the same server as you show in comment 0? (I think it is.)
Do you know which SMTP command (from a log) this error was the response to or occurred during? I.e., was it in response to MAIL FROM, or RCPT TO or maybe DATA?

I'm thinking that for some servers it won't help to go to a new connection when there is a failure or even using a new connection for each message. If they see too many sends within a time frame from your IP they may just shut down or timeout your connection and make you wait a while to try again. (Or, even worse, like gmx did to me, lock/ban the account.)

Comment 2

[gene smith]

Looking at this error closer:

Send Message Error
Sending of the message failed.
The message could not be sent because the connection to Outgoing server (SMTP) smtp.office365.com timed out. Try again.

I'm able to duplicate it by setting a breakpoint in the MAIL FROM response code to stop communication to outlook SMTP server. Looking with wireshark, I see that tb (while stopped at the breakpoint) still sends a keepalive every 30 seconds and outlook responds to it. But after 90 seconds of SMTP inactivity, outlook shuts down the connection (send FIN) and tb pops up the quoted error message.
So it could be that the 25th messages is never really received by outlook for some reason and TB doesn't know it's not received so just waits for a response and then after 90 seconds outlook shuts down the connection. But what is really happening would require a detailed wireshark-style trace to know for sure.

Is this consistent behavior observed multiple times by the user when running the try build?
Does this consistently work OK if user goes back to 102?

The user then had to re-login into the Microsoft account. (Though I am not sure, whether this other error might be related to the try build interfering with the profile of the normal Thunderbird 115.)

Not sure I understand what this means. Did the user get locked out from sending more SMTP to MS similar to what I saw with GMX?
I don't think the try build would have any profile issues...

Comment 3

[gene smith]

Alexander wrote:

The user then had to re-login into the Microsoft account. (Though I am not sure, whether this other error might be related to the try build interfering with the profile of the normal Thunderbird 115.)

I wrote:

Not sure I understand what this means. Did the user get locked out from sending more SMTP to MS similar to what I saw with GMX?

I just now looked in my email Inbox and see the logs you sent, 3.txt and 0.txt.
0.txt shows what happens when 25 messages are sent, each on it's own connection to exchange. TB popped up a message about timeout for the 26th message but it looks like, from the log, that the problem was first caused by a failure to login via OAUTH2. For this, I don't see how a retry would help since TB would just keep trying to use the now invalidated oauth2 token and fail again. So maybe the user really was somehow locked out and had to ask for the account to be unlocked.
But this begs the question, why did the oauth2 login fail on the 26th message? I don't know if it's something about oauth or some other problem detected by the exchange server. I did find that the default max message rate for exchange is 30 messages per minute. But only 25 were sent before it failed and I don't know the time span of the log since timestamps aren't included.

Looking at the 3.txt log, it's as you describe in comment 0. I also found that the default max session time for exchange is 10 minutes so that explains it. So if I change the patch to also handle 4xx errors on RCPT TO response too (not just MAIL FROM) it should handle the 10 minute limit.
But a pref (and ideally a UI) to limit the connection time would also be useful (as you point out in the email).
In addition a pref/UI item to introduce a delay between messages to possibly limit the send rate would be helpful.

Here are some links that discuss the 10 minute max session time and the 30 messages per minute rate limit for exchange:
https://stackoverflow.com/questions/41769476/send-mailmessage-returns-4-4-1-connection-timed-out-every-10-minutes
https://learn.microsoft.com/en-us/office365/servicedescriptions/exchange-online-service-description/exchange-online-limits?redirectedfrom=MSDN#sending-limits-1

Comment 4

[Alexander Bergmann]

The problem was indeed resolved by the patch for Bug 1854567. Thank you!

Comment 5

[Wayne Mery (:wsmwk)]

We use FIXED for bugs that have a patch attached to the bug, DUPE when we know which bug fixed the issue, and WORKSFORME for bugs when neither of those conditions apply