1854567 - SMTP Error 452 - Too many emails sent on this session - Comcast

Status: RESOLVED FIXED

(MailNews Core :: Networking: SMTP, defect)

Description

[Alexander Bergmann]

Two users of my add-on Mail Merge recently reported a problem, when sending multiple messages, after upgrading from Thunderbird 102 to Thunderbird 115. Further investigation indicates there might be a problem in Thunderbird 115, because the same error happens, when Mail Merge is not involved. For example:

Let us assume there are multiple messages, e.g. 30, in the "Local Folders -> Outbox". Then via "File -> Sent Unsent Messages" the first 11 messages are sent just fine; but the 12th message throws an error. Afterwards "File -> Send Unsent Messages" is disabled and Thunderbird needs to be restarted. Then "File -> Sent Unsent Messages" the next 11 messages are sent just fine via "File -> Send Unsent Messages"; but the 24th message throws an error.

Interesting observation: In "Troubleshoot Mode" the 30 messages are continuously processed. The 12th and 24th message still throw an error, but - in contrast to "Normal Mode" - the delivery process does not stop and Thunderbird continues to send the other messages just fine.

The configured SMTP server by both users is Comcast. One user has another SMTP server configured - not Comcast - which works just fine. In Thunderbird 102 the same SMTP server settings for Comcast also work just fine.

Could this be a regression due to the changes regarding the reuse of SMTP connections? We also tried to set mail.smtpserver.default.max_cached_connections = 1, but without success.

Error Dialog:

Send Message Error
Sending of the message failed.
The size of the message you are trying to send exceeds a temporary size limit of the server. The message was not sent; try to reduce the message size or wait some time and try again. The server responded: Too many emails sent on this session.

Error Console:

mailnews.smtp: Command failed: 452 Too many emails sent on this session; currentAction=_actionMAIL SmtpClient.jsm:514:19
mailnews.send: Sending failed: The size of the message you are trying to send exceeds a temporary size limit of the server. The message was not sent; try to reduce the message size or wait some time and try again. The server responded: Too many emails sent on this session., exitCode=2153066794, originalMsgURI= MessageSend.jsm:362:32

Is there anything else we can try or any further information we can share to help fix the problem?

Comment 1

[betterbird.project+10]

TB 115 reuses the SMTP connection, bug 136871, could be related to "Too many emails sent on this session". There is no option to close the SMTP connection to force opening a new one for the next message.

Comment 2

[betterbird.project+10]

Setting mail.smtpserver.default.max_cached_connections to one worsens the issue because then all messages are sent on the same connection. Ideally you'd increase the value to split sending across multiple connections to reduce the number sent on a single connection. It will be a matter of luck though.

Comment 3

[betterbird.project+10]

https://github.com/Betterbird/thunderbird-patches/blob/main/115/bugs/1854567-limit-number-messages-per-connection.patch

Comment 4

[Helposys]

there are following reasons , please do cross check the same

Temporary Size Limit on the Server: The error message suggests that there might be a temporary size limit on Comcast's SMTP server. This limit may be reached after sending a certain number of messages in one session. It's worth checking if Comcast has any specific limits or restrictions on the number of messages sent in a single session.

Session Duration: SMTP servers often have limits on how long a single session can be active. If too much time passes between sending emails, the server may terminate the session. This could explain why sending messages in batches (as observed in your description) sometimes works.

SMTP Server Settings: Since you mentioned that switching to a different SMTP server works fine, it's possible that Comcast's SMTP server settings are more restrictive or have been updated. Double-check the SMTP server settings, and if possible, compare them to the settings in Thunderbird 102 to see if there are any differences.

Thanks
Regards
Manoj Kumar
Server admin at MigoSMTP

Comment 5

[Magnus Melin [:mkmelin]]

https://www.xfinity.com/support/articles/limitations-on-sending-email?ts=2 says Comcast has a limit of 100 recipients per message and max 1000 recipients per day.
It sounds possible that they are (incorrectly perhaps) counting the 100 per session, which isn't that hard to break if you're mass sending.

Comment 6

[betterbird.project+10]

We had reports that the specific Comcast issue "Too many emails sent on this session" has been fixed by our proposal. Gene has a Comcast account.

Comment 7

[Magnus Melin [:mkmelin]]

Attachment: Bug 1854567 - Handle SMTP Error 452 properly, and avoid reusing clients after 100 recipients. r=leftmostcat,gds

According to rfc5321 error 452 is not about the message size, but the number of recpients.

Comment 8

[betterbird.project+10]

According to comment #0, sending the messages prepared by Mail Merge in the outbox, each addressed individually, fails on the 12th message, long before the limit of 100 is reached.

Comment 9

[gene smith]

I'm not sure how TB smtp implementation works. If I have 60 messages in my outbox all going to the same SMTP server and the smtp cached connections is 3 (default), will it create 3 connection and send 20 messages on each connection?
Anyhow, per this https://www.xfinity.com/support/articles/email-errors, comcast allows up to 25 connection per ip address, so what happens if we set cached connections to 25 and try to send 13 messages via outbox. Will TB make 13 connections to comcast/xfinity SMTP server and send 1 message per connection? (Probably not but I haven't looked at the code yet.)
There are other limits at their site such as 1000 messages per session. Not sure if a "session" is one connection or the combined up to 25 connections. But it's a lot more than the observed limit of 11 that is reported in comment 0.
My guess, without looking at the code, is that the "cached" connection of 3 is how many simultaneous connections to different SMTP server are allowed; but there will typically just be one. So a correct fix is to limit the number of messages sent during a connection. I don't think the issue has anything to do with the number of recipients in one message.

Comment 10

[Magnus Melin [:mkmelin]]

(In reply to gene smith from comment #9)

I'm not sure how TB smtp implementation works. If I have 60 messages in my outbox all going to the same SMTP server and the smtp cached connections is 3 (default), will it create 3 connection and send 20 messages on each connection?

On average yes, it's async so ~20 per message, depending on timing issues.

My guess, without looking at the code, is that the "cached" connection of 3 is how many simultaneous connections to different SMTP server are allowed; but there will typically just be one.

Correct.

So a correct fix is to limit the number of messages sent during a connection. I don't think the issue has anything to do with the number of recipients in one message.

It's AFAICT a conflation of what the number of RCPTs mean, and they apparently count all from a connection as one. Which hit's their 11 recipient limit.... The number of RCPTs can come from one message or many.

Comment 11

[gene smith]

(In reply to Alexander Bergmann from comment #0)

Let us assume there are multiple messages, e.g. 30, in the "Local Folders -> Outbox". Then via "File -> Sent Unsent Messages" the first 11 messages are sent just fine; but the 12th message throws an error. Afterwards "File -> Send Unsent Messages" is disabled and Thunderbird needs to be restarted. Then "File -> Sent Unsent Messages" the next 11 messages are sent just fine via "File -> Send Unsent Messages"; but the 24th message throws an error.

Did each of the 30 message have just one recipient or more than one?

Interesting observation: In "Troubleshoot Mode" the 30 messages are continuously processed. The 12th and 24th message still throw an error, but - in contrast to "Normal Mode" - the delivery process does not stop and Thunderbird continues to send the other messages just fine.

Do you know which custom prefs were not at default to maybe have this effect? (Of course, all addons are disabled in troubleshoot mode.)

(In reply to Magnus Melin [:mkmelin] from comment #5)

https://www.xfinity.com/support/articles/limitations-on-sending-email?ts=2 says Comcast has a limit of 100 recipients per message and max 1000 recipients per day.

I'm not following how this relates to the observed limit of 11 messages sent. Maybe if each message in outbox has 100 recipients, you could only send 10 before hitting the 1000 recipient per day limit, which is close to 11. But I suspect each message in the reporter's outbox only had one recipient.

Comment 12

[Magnus Melin [:mkmelin]]

I suspect the documentation may be incorrect.

Comment 13

[gene smith]

https://www.xfinity.com/support/articles/email-errors
This shows similar specs (scroll down to Non-Specific Error Codes)

A remote possibility is that the bug is caused by receiving messages on one connection at too fast a rate as mentioned in section RL000001. As "trust" builds up they let you go faster. But this reads like it's about relaying messages from a non-comcast server to comcast addresses via their server -- not what we are doing.

Anyhow, I don't see a way to independently verify a fix for this without a comcast/xfinity account, which I don't have.

Comment 14

[Alexander Bergmann]

(In reply to gene smith from comment #11)

Did each of the 30 message have just one recipient or more than one?

I have asked the two users and both just confirmed that the individual messages in the "Outbox" had just one single recipient each.

Do you know which custom prefs were not at default to maybe have this effect? (Of course, all addons are disabled in troubleshoot mode.)

The two users are not aware of any unusual (SMTP) settings. During our initial testing we made sure to rule out any involvement of an add-on, in particular my own add-on Mail Merge. One of the users even created about 30 messages individually by hand in "Troubleshoot Mode" - with the same error.

Honestly, for me it doesn't look like a problem with the total number of recipients, but rather a problem with too many individual messages per single SMTP connection:

One of the two users was in contact with the Betterbird team and already tested the proposed patch by Betterbird, see:
https://github.com/Betterbird/thunderbird-patches/blob/main/115/bugs/1854567-limit-number-messages-per-connection.patch

He confirms that without the patch the first 11 messages can be sent just fine, the 12th message throws an error; the next 11 messages can be sent just fine, the 24th message throws an error; and so on. With the patch by Betterbird, the user is able to send 100+ individual messages without a problem!

Comment 15

[gene smith]

(In reply to Alexander Bergmann from comment #14)

One of the two users was in contact with the Betterbird team and already tested the proposed patch by Betterbird, see:
https://github.com/Betterbird/thunderbird-patches/blob/main/115/bugs/1854567-limit-number-messages-per-connection.patch

He confirms that without the patch the first 11 messages can be sent just fine, the 12th message throws an error; the next 11 messages can be sent just fine, the 24th message throws an error; and so on. With the patch by Betterbird, the user is able to send 100+ individual messages without a problem!

Thanks for the info, just wanted to make sure of all the details.
Yes, I'm aware of the proposed patch. However, I would prefer a fix that maybe didn't require adding a new pref to take care of the problem, but maybe not possible.

Do you or one of your users or possibly commenter Helposys in comment 4 know of a way to get a temporary consumer grade SMTP account with comcast/xfinity to actually duplicate the problem and test a possible fix?

Comment 16

[betterbird.project+10]

Please note that we've reworked our patch based on Magnus' ideas from D189676. Now the connection gets closed properly via QUIT.

We noticed a bug in Magnus' patch. Before calling client.quit(); the client needs to be removed from the busy pool. Otherwise it will be used soon after which leads to a hang since this particular client has just sent QUIT.

Comment 17

[betterbird.project+10]

Another issue with Magnus' patch. When the connection is closed via timeout, you need to reset your recipient count. We do something similar here. You should test your patch by reducing the threshold from 100 to 2 and see how it behaves.

Comment 18

[gene smith]

Reporter Alexander,
Want to let you know that Magnus has a solution that should resolve the comcast issue. There will be a new pref you will be able to set false called mail.smtpserver.default.cache_connections. This will cause each message to go out on its own TCP/SMTP connection as occurred with 102. The change should be in daily fairly soon but has not yet landed.
So I probably don't need a comcast account to test this on.

Comment 19

[gene smith]

Attachment: Bug 1854567 - On SMTP 4xx error, do retry on a new connection. r=mkmelin

This just uses the existing mail.smtpserver.default.max_cached_connection set
to less than 1 to indicate that each message should go out on its own
connection. Otherwise, it includes all the other changes from MM's main patch.

Comment 20

[Alexander Bergmann]

To answer your earlier question: Unfortunately I don't know how to get a temporary account for Comcast - neither do my users.

A (new) preference to disable cached connections as a fallback mechanism sounds like a very good idea. This will make debugging these kind of problems much easier, when the old behavior of up to and including Thunderbird 102 can be simulated.

I wonder whether there is a possibility to handle this specific kind of error dynamically at runtime: Use (multiple) cached connections by default and - once this error occurs - close the existing connection and create a new connection. Or fallback to disable cached connections. Is that possible?

Comment 21

[gene smith]

(In reply to Alexander Bergmann from comment #20)

I wonder whether there is a possibility to handle this specific kind of error dynamically at runtime: Use (multiple) cached connections by default and - once this error occurs - close the existing connection and create a new connection.

I updated my proposed patch (linked to in comment 19 above) to do this. With this solution there should be no need to adjust any of the SMTP specific preferences away from default.
I'm not familiar with doing networking directly with JS code so hopefully my proposed changes are "legal". I've verified that the fix works by watching the network with Wireshark.

Or fallback to disable cached connections.

I didn't try to use an idle cached connection (typically, there aren't any just hanging around). I'm just closing the active connection when it produces the 452 error in response to the MAIL FROM command if at least one message has already been sent on the connection. Then I create a new connection within the same "Client" instance and try to send the failed message again on the new connection.

I still don't have an SMTP server that actually produce a 452 error after sending "too many" message on a connection. So in the code I had to simulate the error with the code block in SmtpClient.jsm marked with temp test ... which I've left in the WIP patch for now.

I didn't add anything to respond to errors on RCPT TO commands since I don't think that is as much of an issue. Also, my proposed patch still contains Magnus' change for starting, on the next message, a new connection after 100 or more recipients have been successfully sent.

Comment 22

[Alexander Bergmann]

Gene, thank you for working on this issue! :-)

Just a clarification: It will still be possible to disable the reuse of cached connections by manually setting mail.smtpserver.default.max_cached_connections = 0, correct? Then Thunderbird 115 will behave just like previous versions of Thunderbird, i.e. one (new) message via one (new) connection, correct?

If you can create a try build based on Thunderbird 115.3.1, then I can ask the two users to test the proposed patch.

In the meantime I am in contact with (new) other users, who report two (new) other SMTP related problems with the new Thunderbird 115. A try build based on Thunderbird 115.3.1 would be handy in these cases as well, because I could let them set mail.smtpserver.default.max_cached_connections = 0 to further narrow down the problem they report.

Comment 23

[gene smith]

(In reply to Alexander Bergmann from comment #22)

Gene, thank you for working on this issue! :-)

Just a clarification: It will still be possible to disable the reuse of cached connections by manually setting mail.smtpserver.default.max_cached_connections = 0, correct? Then Thunderbird 115 will behave just like previous versions of Thunderbird, i.e. one (new) message via one (new) connection, correct?

Yes, setting that to zero still does that with my proposed change.

If you can create a try build based on Thunderbird 115.3.1, then I can ask the two users to test the proposed patch.

In the meantime I am in contact with (new) other users, who report two (new) other SMTP related problems with the new Thunderbird 115. A try build based on Thunderbird 115.3.1 would be handy in these cases as well, because I could let them set mail.smtpserver.default.max_cached_connections = 0 to further narrow down the problem they report.

Yes, I will try to make "try" based on the latest release version (115.3.1, I presume). However, sometime "try" server won't build "older" versions and requires it to be built against daily or maybe latest beta. (I assume the users can't test with a daily or beta since your addon is limited to 115?)

Comment 24

[gene smith]

Alexander,
The try build with patch on top of 115.3.1 is building now. I'm building for all platforms since I don't know what you need. Since you sound familiar with try builds, I suppose you know where to find the installers after it is done.
Here's the link: https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=f0979f536e781f3d45b02261e61535dc0018cd43
All I did was comment out my error insertion code and tried to debug it again "mentally", so no certainty it will do what is expected with comcast servers.
When it runs, it is still marked as "daily" but will show version 115.3.1 in about TB.

Comment 25

[Alexander Bergmann]

Gene, thank you for creating the try build! :-)

Yesterday I did a few tests with the try build myself and I am very pleased with the results:

When using the default setting of mail.smtpserver.default.max_cached_connections = 3, then the connection is reused - just as in Thunderbird 115. (No unintended regression!) Additionally the number of recipients is now counted and - once 100 recipients on a single connection are reached - the connection is closed and a new connection is opened.

When using mail.smtpserver.default.max_cached_connections = 0, then a new connection is used for every outgoing message. This is actually the very same behavior as in Thunderbird 102. I really like this new possibility to fallback to the old behavior, which has worked very well for many years.

Unfortunately I am not able to reproduce the error 452, so I cannot test the new automatic error handling in this case. I will contact the two users, who have experienced this error initially, and report back their results.

Comment 26

[Alexander Bergmann]

I wonder, whether other type of errors (or all other errors) can be handled in the same way: Whenever a cached connection is reused and an error occurs, then close the cached connection and try again with a new connection. Only if the error persists in a new connection, then stop the delivery and report the error to the user.

As I hinted before, I am in contact with another user, who reports another timeout error: The server stops to accept new messages after exactly 10 minutes. With the current patch to automatically handle error 452, the problem will probably not be solved. But I am confident that setting mail.smtpserver.default.max_cached_connections = 0 will be successful.

With an automatic error fallback mechanism as described above, this other timeout error might be handled automatically and gracefully as well.

Comment 27

[gene smith]

(In reply to Alexander Bergmann from comment #22)

If you can create a try build based on Thunderbird 115.3.1, then I can ask the two users to test the proposed patch.

Hi Alexander,
I was wondering if the users you mention have had a chance to test the patch for the 115.3.1 try build? With this they should be able to send unlimited messages via comcast using default settings (I hope).

In case you need it, here's the link again to the try build: https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=f0979f536e781f3d45b02261e61535dc0018cd43

In the meantime I am in contact with (new) other users, who report two (new) other SMTP related problems with the new Thunderbird 115. A try build based on Thunderbird 115.3.1 would be handy in these cases as well, because I could let them set mail.smtpserver.default.max_cached_connections = 0 to further narrow down the problem they report.

Yes, that may also help for the "new" problems but should not be needed for the "limit of 11 message via comcast" problem.

Comment 28

[Alexander Bergmann]

I am very sorry for the delay... But I only received the results of the try build from one user last Thursday and wasn't able to relay them right away, because I was AFK over the weekend for personal reasons.

One of the two users did some extensive tests; the other user was unfortunately not able to do any tests with the try build, yet. Here are the very promising results of the first user:

At the beginning we made sure the error is still happening with the official version Thunderbird 115.3.2 Stable - to rule out any interim configuration changes of the email server.

Afterwards we installed the try build and did a first test with the default setting mail.smtpserver.default.max_cached_connections = 3. In this case all 150+ individual messages were sent successfully without a single problem. The user also created an SMTP log - after 11 messages were already sent successfully, the error is now handled gracefully:

...
mailnews.smtp: Sending message <xxx> SmtpService.jsm:88:18
mailnews.smtp: Reusing a connection SmtpClient.jsm:130:19
mailnews.smtp: Total RCPTs during this connection: 11 SmtpClient.jsm:1257:19
mailnews.smtp: Message sent successfully. SmtpClient.jsm:1318:21
...
mailnews.smtp: Sending message <xxx> SmtpService.jsm:88:18
mailnews.smtp: Reusing a connection SmtpClient.jsm:130:19
mailnews.smtp: C: MAIL FROM: <xxx> BODY=8BITMIME SIZE=1898302 SmtpClient.jsm:624:19
mailnews.smtp: S: 452 4.1.0 Too many emails sent on this session SmtpClient.jsm:430:17
mailnews.smtp: Socket closed. SmtpClient.jsm:518:17
mailnews.smtp: Client ready for reuse SmtpClient.jsm:561:19
mailnews.smtp: Connection to smtp.comcast.net closed SmtpClient.jsm:176:19
mailnews.smtp: Connecting to smtp://smtp.comcast.net:465 SmtpClient.jsm:135:19
mailnews.smtp: Connected SmtpClient.jsm:407:17
mailnews.smtp: S: 220 comcast.net comcast.net ESMTP server ready SmtpClient.jsm:430:17
mailnews.smtp: C: EHLO <xxx> SmtpClient.jsm:624:19
mailnews.smtp: S: comcast.net hello <xxx>, pleased to meet you
250-HELP
250-AUTH LOGIN PLAIN XOAUTH2
250-SIZE 36700160
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 OK
SmtpClient.jsm:430:17
mailnews.smtp: Possible auth methods: PLAIN,LOGIN SmtpClient.jsm:911:17
mailnews.smtp: Current auth method: PLAIN SmtpClient.jsm:664:17
mailnews.smtp: Authentication via AUTH PLAIN SmtpClient.jsm:679:21
mailnews.smtp: C: AUTH PLAIN <xxx> SmtpClient.jsm:624:19
mailnews.smtp: S: 235 2.7.0 ... Authentication succeeded SmtpClient.jsm:430:17
mailnews.smtp: Authentication successful. SmtpClient.jsm:1134:17
mailnews.smtp: C: MAIL FROM: <xxx> BODY=8BITMIME SIZE=1898302 SmtpClient.jsm:624:19
mailnews.smtp: S: 250 2.1.0 <xxx> sender ok SmtpClient.jsm:430:17
mailnews.smtp: MAIL FROM successful, proceeding with 1 recipients SmtpClient.jsm:1191:17
mailnews.smtp: Adding recipient... SmtpClient.jsm:1196:17
mailnews.smtp: C: RCPT TO: <xxx> SmtpClient.jsm:624:19
mailnews.smtp: S: 250 2.1.5 <xxx> recipient ok SmtpClient.jsm:430:17
mailnews.smtp: Total RCPTs during this connection: 1 SmtpClient.jsm:1257:19
mailnews.smtp: RCPT TO done. Proceeding with payload. SmtpClient.jsm:1258:19
mailnews.smtp: C: DATA SmtpClient.jsm:624:19
mailnews.smtp: S: 354 OK SmtpClient.jsm:430:17
mailnews.smtp: Sending 65536 bytes of payload 28 SmtpClient.jsm:592:17
mailnews.smtp: Sending 63294 bytes of payload SmtpClient.jsm:592:17
mailnews.smtp: S: 250 2.0.0 <xxx> mail accepted for delivery SmtpClient.jsm:430:17
mailnews.smtp: Message sent successfully. SmtpClient.jsm:1318:21
...
mailnews.smtp: Sending message <xxx> SmtpService.jsm:88:18
mailnews.smtp: Reusing a connection SmtpClient.jsm:130:19
mailnews.smtp: Total RCPTs during this connection: 2 SmtpClient.jsm:1257:19
mailnews.smtp: Message sent successfully. SmtpClient.jsm:1318:21
...

We did another test with mail.smtpserver.default.max_cached_connections = 0, in which case a new connection is used for every single outgoing message. Again all 150+ individual messages were sent successfully without a single problem - although the delivery was noticeably slower (Which seems to be expected I guess). In this case the SMTP log looks like:

...
mailnews.smtp: Sending message <xxx> SmtpService.jsm:88:18
mailnews.smtp: Connecting to smtp://smtp.comcast.net:465 SmtpClient.jsm:135:19
mailnews.smtp: Connected SmtpClient.jsm:407:17
mailnews.smtp: Total RCPTs during this connection: 1 SmtpClient.jsm:1257:19
mailnews.smtp: Message sent successfully. SmtpClient.jsm:1318:21
mailnews.smtp: Socket closed. SmtpClient.jsm:518:17
...
mailnews.smtp: Sending message <xxx> SmtpService.jsm:88:18
mailnews.smtp: Connecting to smtp://smtp.comcast.net:465 SmtpClient.jsm:135:19
mailnews.smtp: Connected SmtpClient.jsm:407:17
mailnews.smtp: Total RCPTs during this connection: 1 SmtpClient.jsm:1257:19
mailnews.smtp: Message sent successfully. SmtpClient.jsm:1318:21
mailnews.smtp: Socket closed. SmtpClient.jsm:518:17
...

I can send you the full SMTP logs privately - if you need them.

Comment 29

[Alexander Bergmann]

Last Thursday I received yet another problem report:

A user reported an issue with Ionis; after exactly 20 messages he receives the following error:
"Requested action aborted. Mails per session limit exceeded."

He further elaborates this limitation might have been introduced by Ionos back in December 2020; though he didn't find any kind of official confirmation. At the moment it is also unclear, which exact error number is returned by the server. He thinks it might be error 451. Unfortunately he uses Thunderbird 115.3.2 Portable and doesn't seem to be able to run tests with the try build.

I assume mail.smtpserver.default.max_cached_connections = 0 will solve the problem. However, may I ask you another time, whether an automatic fallback mechanism for all errors - as described in comment #26 - might be a viable and more general solution?

Comment 30

[Alexander Bergmann]

I was able to reproduce the Ionos problem, i.e. after 20 messages the error 451 is thrown. This affects gmx.de and web.de as well. Unfortunately this specific limitation is not mentioned in the official FAQ at Ionos:
https://www.ionos.com/help/email/general-topics/what-are-the-mass-mailing-limits/

Send Message Error
Sending of the message failed.
An error occurred while sending mail. The mail server responded: Requested action aborted
Mails per session limit exceeded. Please verify that your email address is correct in your account settings and try again.

With the try build and the default mail.smtpserver.default.max_cached_connections = 3 the problem persists:

mailnews.smtp: Message sent successfully. SmtpClient.jsm:1318:21
mailnews.smtp: Sending message <xxx> SmtpService.jsm:88:18
mailnews.smtp: Reusing a connection SmtpClient.jsm:130:19
mailnews.smtp: C: MAIL FROM: <[e-mail]> BODY=8BITMIME SIZE=358 SmtpClient.jsm:624:19
mailnews.smtp: S: 451-Requested action aborted. 451 Mails per session limit exceeded. SmtpClient.jsm:430:17
mailnews.smtp: Command failed: 451 Requested action aborted. Mails per session limit exceeded.; currentAction=_actionMAIL SmtpClient.jsm:546:19
mailnews.send: Sending failed; An error occurred while sending mail. The mail server responded: Requested action aborted. Mails per session limit exceeded. Please verify that your email address is correct in your account settings and try again., exitCode=2153066722, originalMsgURI= MessageSend.jsm:337:32 undefined ExtensionCommon.sys.mjs:812
mailnews.smtp: Closing connection to mail.gmx.net... SmtpClient.jsm:172:21
mailnews.smtp: Socket closed. SmtpClient.jsm:518:17

With the try build and mail.smtpserver.default.max_cached_connections = 0 the problem is solved; 50+ messages are created just fine.

The user originally reporting the issue also discovered that Buhl Data Service - one of the more prominent tax software developers in Germany - released this FAQ entry just a few months ago:
https://www.buhl.de/shop/faqs?article=2422

Because this is pretty much the same bug as the Comcast issue - just with different error codes - I would really like to suggest to develop a more general approach than only to handle error 451 and 452 in particular. Either via an automatic fallback mechanism for all kind of errors - as described in comment #26 - or via another dedicated preference to limit the number of messages per single connection per SMTP server.

That being said I am also totally fine with shipping the patch as is or with an automatic error handling for 451 on top - and working on a more general approach afterwards. (We urgently need the possibility to disable the reuse of cached connections via mail.smtpserver.default.max_cached_connections = 0 for users to easily get Thunderbird 115 working again.)

Comment 31

[gene smith]

Hi Alexander,
Good to hear the "try" patch made improvements, at least on comcast.
Yes, if you have full logs available, I would like to see them.
I haven't looked close at it yet, but it may be possible to handle "any error" and try to re-do the connection like the patch handles just 452.

You mention other providers like "ionis", gmx.de and web.de that have similar problems or produce error 451 and not just 452. Would it be possible to provide me a temporary account on one of those where I can duplicate and test a complete fix? But, even if you can't, I see your point about going ahead and releasing the patch as-is.

Comment 32

[gene smith]

I made a free account on gmx and tried to cause the problem by sending emails via LF/Outbox to about 23 recipient. Actually just sent them to invalid addresses like [e-mail], [e-mail], ... etc and gmx accepted the first 19 and then said error 550, "Mailbox not available" for the rest. Then they locked my account due to "suspicious activity". So I guess I should have used "real" addresses.

I made another gmx account and tried the send again with valid addresses (mostly just test accounts I have for testing TB). I had to send more than one to most addresses since I don't have 20 test accounts. This also sent the first 19 OK and then I received error 550 on the ones after the first 19. (I had 23 total.) At least this didn't cause my new gmx account to be locked.

What I saw in both cases is, as I said, the first 19 are sent OK on a single connection. Then the 20th message is attempted to be sent on the connection and fails with 550 on the "RCPT TO" response causing the connection to be terminated. Then a new connection is established and authenticated and message 21 is attempted to be send and also fails with the same error on RCPT TO. So I end up with 4 messages still in Outbox that never got successfully sent and produced error 550 "mailbox unavailable".

It appears that my fix for errors like my 452 during the "MAIL FROM" response won't help in this case since a new connection doesn't even allow the next message in the Outbox queue to be sent.

I'm thinking maybe that since my gmx accounts are new and I'm not yet a "trusted" user that they are limiting the number of SMTP transactions I can do to 19 within a time span. I'll try again and see if I can send more than 19 today.
Finally, FWIW, gmx has now unlocked my original account and now I have two gmx accounts.

Comment 33

[gene smith]

Finally, FWIW, gmx has now unlocked my original account and now I have two gmx accounts.

Now they are both locked again after re-testing the sends. Similar to before but this time 20 messages sent OK. On MAIL FROM for message 21, got 451 error (mails per session limit exceeded) and I had changed the code so that any 4xx would make a new connection on the same "client" and retry the failed message send. But all the additional attempts to send failed with 550 "mailbox unavailable" during RCPT TO phase.

Comment 34

[gene smith]

(In reply to gene smith from comment #33)

Finally, FWIW, gmx has now unlocked my original account and now I have two gmx accounts.

Now they are both locked again after re-testing the sends.

This time gmx refused to unlock my test accounts and permanently banned them. Oh well, I guess sending 20 or so messages from Outbox for a brand new GMX user looks like a violation of terms of service to them.

I did some more testing by simulating errors on one of my existing SMTP account and it works OK so went ahead and submitted the patch for review.
It keeps all of Magnus' changes except that no additional pref is needed and responds to error codes 4xx by retrying the failed send on a new connection. The patch is mostly the same as Alexander and his user tested except it responds to any 4xx code (not just 452) on the MAIL FROM response by closing the connection, creating a new connection and re-sending the message on the new connection.

Comment 35

[gene smith]

Alexander,
The patch has been approved for checkin. However, since it will only appear initially in daily it probably won't be helpful to you there. So let me know if you want me to merge the patch with a try build based on latest 115.* so you and your users can test it in real world.
The primary change from the last try build is that it now handles the 4xx errors in the the response to RCPT TO and not just in MAIL FROM response. I haven't added a limit on the amount of time the SMTP connection can live or a limit on the number of messages to send on a connection other than the hardcoded 100 recipient limit* that Magnus added in his original patch, which I kept.
Thanks.
Edit:
*) To try to clarify this, you can have more than 100 recipients in messages queued in Outbox. But once messages are sent and the total recipient count exceeds 99, the current SMTP connection will be closed and a new connection established to send the remaining messages in Outbox, again with the 100 recipient limit. Example: 1000 messages in Outbox each with 1 recipient. This will use use 10 SMTP connections.

Comment 36

[Alexander Bergmann]

Yes, another try build based on Thunderbird 115 would indeed be very nice to have - mainly to bridge the time until the patch finally lands in Stable. Additionally it would also give me and my users a chance to test the patch in a real-world setting beforehand. Thanks!

Comment 37

[gene smith]

It's almost done now: https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=70e7ffc1374a9bb631fd492010a0e83eb0572927
This is the patch merged into 115.4.1.

Comment 38

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/7d987e318488
On SMTP 4xx error, do retry on a new connection. r=mkmelin

Comment 39

[Alexander Bergmann]

How is the current time schedule for this patch regarding Beta and Stable? Is there a chance the patch can be uplifted in the near future so that the problem is fixed in time for the Christmas emails?

If there is no chance to uplift the whole patch: Do you think you can at least uplift the part of the patch, which restores the old behavior of Thunderbird 102 (and earlier) to only use a single SMTP connection for each outgoing message via mail.smtpserver.default.max_cached_connections = 0?

Comment 40

[Wayne Mery (:wsmwk)]

This only just became available on beta 121.0b1 yesterday. How soon it is taken on 115 depends on progress with the localization (language) change in the patch. => Rob?

Comment 41

[Rob Lemley [:rjl]]

Strings are not translated yet. Give it at least 4 weeks from the 27th.

Comment 42

[Wayne Mery (:wsmwk)]

(In reply to Rob Lemley [:rjl] from comment #41)

Strings are not translated yet. Give it at least 4 weeks from the 27th.

Can we uplift based on the following?
I spot checked 8 languages and all were translated - mostly three weeks ago. And we are at four weeks since landing.
Sorted on latest activity and most have recent activity. Most of those don't have recent activity, have no activity for months.
So I wouldn't expect much more for many weeks.

Comment 43

[Rob Lemley [:rjl]]

(In reply to Wayne Mery (:wsmwk) from comment #42)

Can we uplift based on the following?
I spot checked 8 languages and all were translated - mostly three weeks ago. And we are at four weeks since landing.
Sorted on latest activity and most have recent activity. Most of those don't have recent activity, have no activity for months.
So I wouldn't expect much more for many weeks.

I look at the top 10 locales as shown on stats.thunderbird.net. Then I add in "nl" because the top 10 are in flux sort of. This results in a top 13 (which includes en-US and ja-JP-mac so really top 11):
cs
de
en-GB
en-US
es-ES
fr
it
ja
ja-JP-mac
nl
pl
pt-BR
ru
zh-CN

Looking at that list, all have translated this new string so uplift is good to go.

Comment 44

[Wayne Mery (:wsmwk)]

Comment on attachment 9356771 [details]
Bug 1854567 - On SMTP 4xx error, do retry on a new connection. r=mkmelin

[Triage Comment]
Approved for esr115

Comment 45

[Rob Lemley [:rjl]]

Thunderbird 115.6.0:
https://hg.mozilla.org/releases/comm-esr115/rev/929f52ae8642
https://hg.mozilla.org/releases/comm-esr115/rev/fe0d3990d4cb

Comment 46

[Rob Lemley [:rjl]]

Backout Thunderbird 115.6.0:
https://hg.mozilla.org/releases/comm-esr115/rev/473897dfbb2f

Comment 47

[Rob Lemley [:rjl]]

Thunderbird 115.6.0:
https://hg.mozilla.org/releases/comm-esr115/rev/ce88b4a3964d