136871 - mail opens new smtp connection for each item sent. Should reuse server connection, especially for slow or overloaded network.

Status: RESOLVED FIXED

(MailNews Core :: Networking: SMTP, defect)

Description

[David Hagood]

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.9+) Gecko/20020402
BuildID:    2002040211

If there are many mails queued for transmission (i.e. in disconnected mode) and
then the queued mails are sent, Mozilla opens a fresh connection to the mail
server on each message.

This is a real problem if the mail server is using authentication and is slow to
authenticate - our mail server requires about 4 seconds to authenticate a user,
so this is 4 seconds per message wasted.


Reproducible: Always
Steps to Reproduce:
1. Go offline
2. Compose N messages, where N > 1
3. Go online & force a send of unsent messages

Actual Results:  Mozilla will open a new mail server connection on each message

Expected Results:  Mozilla would open the socket to the MTA, and then send all
pending messages.

Comment 1

[Mikael Parknert]

Reporter: Can you reproduce this bug with a recent build of Mozilla (for
example, 1.4rc2)? 

If so, then please comment again with details. 
If not, then please resolve this bug as WORKSFORME. Thanks.

Comment 2

[david.hagood]

Verified with build 2003050714 - Queued several messages offline, started
Ethereal capturing SMTP traffic, and put Mozilla into on-line mode.

For each queued message, Mozilla:
1) Opened the connection.
2) Authenticated to the server.
3) Send the mail
4) Closed the socket - full TCP shutdown.

Comment 3

[Mikael Parknert]

yes, I see the same

Comment 4

[Jo Hermans]

*** Bug 228638 has been marked as a duplicate of this bug. ***

Comment 5

[David :Bienvenu]

I'd fix this by adding the ability to cache a single connection to the smtp
server object and add a boolean to the smtp url that says whether or not to
cache the connection when done. Default would be false. That flag would be
checked when the smtp url was done, and if true, we would leave the connection
alive and store it in the smtp server. The send later code would set it to true
for each msg sent until the last one, where it would be set to false. The code
that gets an smtp protocol object would need to learn to try to get it from the
server.

Comment 6

[OstGote!]

I think this should be Hardware/OS All/All. See the dupe.

Comment 7

[Mark Shuttleworth]

I've put a $100 bounty on this one, details at:

  http://www.markshuttleworth.com/bounty.html

Comment 8

[Christian Eyrich]

David

But the connection can only be reused when sending mails through the same SMTP
server as the mail(s) before. So we've three options:
1. Hold the connection until reaching a message that has to be send using
another server. That might lead to re-establishing a former connection again
when reaching another message.
2. Holding multiple connections at a time, each for a once used SMTP server in
case we need it later. They're only closed after all messages have been sent.
3. Sorting the messages to blocks that going to be send using the same SMTP server.

And the SMTP protocol code has to know if it has already sent a message to jump
right to the MAIL FROM point instead waiting for the servers initial 220. But
that should be quite trivial.

Comment 9

[David :Bienvenu]

>But the connection can only be reused when sending mails through the same SMTP
>server as the mail(s) before. So we've three options:

that's why I said to put the connection in the smtp server object and to
retrieve it from the stmp server. There's one of those per smtp server, so you
should be fine there.

We'll only ever cache smtp connections when doing a send unsent messages, imo.
At no other time do we really know we're going to send multiple messages to the
same server. So we'll only cache connections in that case.

Comment 10

[David :Bienvenu]

Oh, I see what you mean, Christian. A single unsent messages folder can have
messages destined for multiple smtp servers...well, a simple way of doing this
would just be to iterate over the smtp servers and have each of them close their
cached connection, if any, when the send later operation is done. In the normal
case (a single stmp server), this doesn't do any extra work, and has the
advantage of simplicity. There can only be one unsent messages folder, so I
don't think this would cause any problems. Only sending unsent messages should
cause a connection to get cached...

Comment 11

[Christian Eyrich]

Ok, that's option 2, yes? Having messages destined for 3 different smtp servers
will produce three cached connections until they're closed at the end.

I wouldn't feel that good with it, but it's a simple and functional solution.

Comment 12

[David :Bienvenu]

yes, that's option 2. It's not great, but Option 1 has the drawback you mention.
I'm not sure Option 3 is doable easily w/o reading all the messages into memory
- I don't believe which smtp server to use is stored in the db/msg hdr.

Comment 13

[Christian Eyrich]

> don't believe which smtp server to use is stored in the db/msg hdr.

No, have to read the id key from the X-Identity-Key line, translate into
identity and from there to the smtpserver. Surely that most difficult and costly
solution.

Comment 14

[Vaclav Dvorak]

Is this really specific to PC/Linux?

Comment 15

[David :Bienvenu]

all/all

Comment 16

[Colin Slater]

I've been working on this for a while, and I got the protocol details worked out
and tested already, but i'm not sure how to get through the myriad of classes
between where each email is picked up out of the outbox and where it's actualy
put on the wire. Currently, I have NS_MsgLoadSmtpUrl() passing an array of URL's
to nsSmtpProtocol, and nsSmtpProtocol stores the list and sends each one in
order. But if I implement it this way, there are still 5 more functions that
need to be modified to pass all their arguments in arrays. This sees like an
ugly hack to me. 

The code that currently sends mail is simply not condusive to an easy fix for
this bug. I guess it might be possible to reuse the same nsSmtpProtocol object,
but I do not know how to go about that. If someone could help me figure out what
the best solution is, i'd appreciate it.

Comment 17

[Christian Eyrich]

I must admit that I never got out of nsSmtpProtocol.
But couldn't NS_MsgLoadSmtpUrl build a list of aUrl's and determine wheter it
already created a SmtpProtocol object with this URL? If yes, reuse it and its
connection.
Of course SmtpProtocol must not close the socket when one message has been sent
and the object must not get destroyed until the last message has been sent.

This way no function has to know all URL at once.
Care has to be taken for servers closing a cached connection (timout) while
we're sending a message through another one. Reusing such a "connection" and
object would be tricky.

Comment 18

[Colin Slater]

I just had an idea. My code currently uses a list of url's, and processes them
sequentialy. I could just write a function in nsSmtpProtocol to add URL's to
this list one by one, and when all the URL's are added, a different function
will kick off sending all the URL's at once. The server won't time out, because
we won't start sending untill we have all the URL's, and if the server doesn't
support multiple messages at the same time, we can just keep creating more
connections.

My only problem is how do I save the same instance of nsSmtpProtocol. I'm not
very familiar with Mozilla code, but might XPCOM do something of this nature?
I'm reading the docs for it now, an XPCOM service I guess is what this would be?

Comment 19

[Christian Eyrich]

(In reply to comment #18)
> I just had an idea. My code currently uses a list of url's, and processes them
> sequentialy. I could just write a function in nsSmtpProtocol to add URL's to
> this list one by one, and when all the URL's are added, a different function
> will kick off sending all the URL's at once.

I can't see why this could help against a timeout. See my comment #8, except you
sort the URL's by server, you can't be sure you send all mails for server x
continuously.
If you don't, connection to server x could time out while you're sending a
message to server y before returning back to send another message to server x.

Comment 20

[Colin Slater]

> I can't see why this could help against a timeout. See my comment #8, except you
> sort the URL's by server, you can't be sure you send all mails for server x
> continuously.

Heh, i'm not that far yet. I misread your previous post. I think that having
multiple messages going to multiple servers is niche case, and that if we had to
revert to the current behaviour when multiple servers are involved, it wouldn't
be too terrible a loss. I'll worry about that after it works for a single server.

Comment 21

[Christian Eyrich]

Well, if you think that will do it, it's ok. I only wanted to prevend us/your
from changing the whole strategy if multiple servers come into play.

Comment 22

[:aceman]

I would vote for solution 3 from comment 8. There you don't need to worry about 
any timeouts or caching connections.

Comment 23

[Andrew Hagen]

I agree with comment 22. Most people will only be using one SMTP server. 

Comment 24

[David Ascher (:davida)]

This seems like a good win.

Comment 25

[David Ascher (:davida)]

The asynchronous smtp sending (bug 440794) will use a queue in the outbox, and should fix this. Moving off of blocking to wanted as per driver discussion, but I feel optimistic that it'll happen soonish.

Comment 26

[Magnus Melin [:mkmelin]]

Leaving wanted‑thunderbird3+, but low priority.

Comment 28

[Wayne Mery (:wsmwk)]

bienvenu, is it reasonable that bug 511079 would deliver this?
or does this bug come first?
(return to reality)

Comment 29

[David :Bienvenu]

(In reply to Wayne Mery (:wsmwk) from comment #28)
> bienvenu, is it reasonable that bug 511079 would deliver this?
> or does this bug come first?

They are essentially independent.

Comment 30

[Philip Prindeville]

(In reply to David Ascher (:davida) from comment #25)
> The asynchronous smtp sending (bug 440794) will use a queue in the outbox,
> and should fix this. Moving off of blocking to wanted as per driver
> discussion, but I feel optimistic that it'll happen soonish.

You still feeling it? :-)

Comment 31

[John Navas]

This is a fairly serious issue for those of us working primarily offline with infrequent slow connections (e.g., low-speed wireless, satellite) to send queued messages, and it's been open for 10 years (!), so please elevate the Priority. Thank you.

Comment 33

[Alex Ihrig [:Thunderbird_Mail_DE]]

In Bug 1301865 I reported the following issue:

In our support forums a user reported a problem by using filters to forward messages. By this filter Thunderbird has to forward 20-30 messages per day. It seems the filter is sending these 20-30 messages in parallel, which leads to 20-30 parallel SMTP connections. The provider has a limit of 3 parallel SMTP conntections. This leads always to SMTP errors and failing the filter (the messages are marked as forwarded, but in fact they are not).

It would be necessary to have an option to limit Thunderbirds parallel SMTP connections and/or parallel related filter actions.

Comment 34

[Alex Ihrig [:Thunderbird_Mail_DE]]

Attachment: smtp.log

SMTP log related to my last comment

Comment 35

[Paul Schauble]

I have the same problem as comment 33. More and more I'm seeing SMTP servers that have limits on the number of concurrent connections.

The solution proposed in this bug, one connection per server, solves the problem, but at the cost of being slower. Do you think it's worth considering having more than one connection per server, but a settable limited number?

Comment 36

[Howard Chu]

Making it settable may be a good idea. Fwiw, on a slow connection (e.g. 2G modem, 3G modem) opening multiple connections in parallel will only slow things down further.

Comment 37

[u677327]

Opening multiple connections also has privacy implications for Onion hosts on the Tor network in addition to the 3+ second latency per message to connect.

Comment 38

[Wayne Mery (:wsmwk)]

rnons, do you know if this would still be an issue with smtp-js? Perhaps this should be changed to ENH?

Comment 39

[Ping Chen [:rnons] (away)]

It's still an issue, we always create a new connection. I guess it's a bit easier to support this now, but still not that easy.

Comment 40

[Alexandros Diamantidis]

Apart from being slower, an MSA with rate limits might block a sender for some time when opening many short-lived SMTP connections in a short period - I hit this case after creating many unsent outgoing messages with a mail merge plugin, while being on a fast network. The mail server stops accepting connections after 20 messages or so, and I have to wait for a short while. Also, this causes a new error modal to appear for each message that couldn't be sent, which must be closed separately.

Comment 41

[Ping Chen [:rnons] (away)]

OK, I will take this next month.

Comment 42

[Ping Chen [:rnons] (away)]

Attachment: Bug 136871 - Use class syntax in SmtpService.jsm and SmtpServer.jsm. r=mkmelin

Comment 43

[Ping Chen [:rnons] (away)]

Attachment: Bug 136871 - Reuse SMTP connections when sending mails. r=mkmelin

Comment 44

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/b68d61082ca1
Use class syntax in SmtpService.jsm and SmtpServer.jsm. r=mkmelin
https://hg.mozilla.org/comm-central/rev/880b61ab6557
Reuse SMTP connections when sending mails. r=mkmelin