244210 - slow mail transfer speed to server

Status: RESOLVED WORKSFORME

(MailNews Core :: Networking: SMTP, defect)

Description

[alanmoz]

User-Agent:       Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.6) Gecko/20040113
Build Identifier: Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.6) Gecko/20040113

Mail server is on local network, 10/100 lines between server and work-station. 
Send a large attachment (say over 10 mb), and transfer from Mozilla client on
work-station to server is VERY slow.  Same with Netscape v7.  However, Netscape
4.7 email client is 10 times faster - as is Eudora.  Same situation for
receiving from server to work-station.  Netscape 4.7/Eudora go 10 times the
speed of  Mozilla 1.6 OR Netscape 7 email client.  Mozilla takes minutes,
4.7/Eudora take seconds.  Same whether a Win98 machine or a Win2k machine
running Mozilla.

Reproducible: Always
Steps to Reproduce:
1. Described exactly above
2.
3.


Expected Results:  
Same speed at least as Netscape 4.7, not one tenth the speed.

Comment 1

[Gervase Markham [:gerv]]

This is an automated message, with ID "auto-resolve01".

This bug has had no comments for a long time. Statistically, we have found that
bug reports that have not been confirmed by a second user after three months are
highly unlikely to be the source of a fix to the code.

While your input is very important to us, our resources are limited and so we
are asking for your help in focussing our efforts. If you can still reproduce
this problem in the latest version of the product (see below for how to obtain a
copy) or, for feature requests, if it's not present in the latest version and
you still believe we should implement it, please visit the URL of this bug
(given at the top of this mail) and add a comment to that effect, giving more
reproduction information if you have it.

If it is not a problem any longer, you need take no action. If this bug is not
changed in any way in the next two weeks, it will be automatically resolved.
Thank you for your help in this matter.

The latest beta releases can be obtained from:
Firefox:     http://www.mozilla.org/projects/firefox/
Thunderbird: http://www.mozilla.org/products/thunderbird/releases/1.5beta1.html
Seamonkey:   http://www.mozilla.org/projects/seamonkey/

Comment 2

[alanmoz]

The issue was fixed in the INCOMING direction, but the same issue still remains
in the OUTGOING direction.

Alan

Comment 3

[Ray Booysen]

*** Bug 314733 has been marked as a duplicate of this bug. ***

Comment 4

[(not reading, please use [e-mail] instead)]

sorry for the spam.  making bugzilla reflect reality as I'm not working on these bugs.  filter on FOOBARCHEESE to remove these in bulk.

Comment 5

[Wayne Mery (:wsmwk)]

Alan,
Andrei,

Do you run an antivirus program?  try without.
 http://kb.mozillazine.org/Thunderbird_:_FAQs_:_Anti-virus_Software / bug 378360    

if you still see a problem please create an smtp log
<http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#smtp>

one similar bug - bug 374459

Comment 6

[Andrei Boros]

Wayne : in my case, no antivirus.

I will monitor the issue and will try to obtain a log

Comment 7

[Wayne Mery (:wsmwk)]

(I should have asked) 
Do you see this with more than one server/ISP?
Does this happen on same scale with large messages with no attachments, or does it only affect mail with attachments?


Alan says he still see this also using SM 1.1.4 "Transmit = 266 kb/sec, receive same file 2.6 mb/sec."

Comment 8

[Wayne Mery (:wsmwk)]

Attachment: 9MB text message, zipped to 51k

>Since it shows up with 3-plus megs of attachments (I test with 5 to 10
>meg), it's a bit difficult to create a "large message with no
>attachments" of that size!

not at all.  use the zipped message to create large test message without attachments - select all and paste to make larger test message as needed.

I see no send issues for very large messages, hence the question of whether you tested with a different server to rule out some variable within the server.


>Since 3 or 5 megs is beyond most ISP email limits, I doubt your normal
>user ever sees the effects - or cares, for that matter.

many people sending emails with large pictures that easily exceed 1, 2, 3MB.  though probably not many people would see or care if it were slow.

If send big text is much different from sending message of similar size with large attachments, then please attach smtp log for both scenarios.  Also, turn off spellcheck on send and copy message to "sent" folder.

Comment 9

[Wayne Mery (:wsmwk)]

(In reply to comment #0)
> Send a large attachment (say over 10 mb), and transfer from Mozilla client on
> work-station to server is VERY slow.  Same with Netscape v7.  However, Netscape
> 4.7 email client is 10 times faster - as is Eudora.  

and comment 7
>  SM 1.1.4 "Transmit = 266 kb/sec, receive same file 2.6 mb/sec."

David, do you have any insight to the above... i.e. 4.7 being faster than anything since?

unfortunately reporter not likely to be of any further help: "Doesn't really matter ... Matter closed."

Comment 10

[David :Bienvenu]

in cases like this, 9 times out of 10, it's a virus checker slowing things down. I don't think networking in general is 10x slower in Thunderbird vs 4.7

Comment 11

[Wayne Mery (:wsmwk)]

That would have been my first guess too. But Alan states these were from same machine to same server, the "only variable" being the mail client.

Comment 12

[David :Bienvenu]

virus checkers have been known to slow down one executable and not an other - seems to happen all the time

Comment 13

[alanmoz]

Just add to the record - virus checkers are NOT involved.

Comment 14

[Andrei Boros]

I've just run a test on this. 

I followed the instructions at http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#smtp to the letter, but got a log file that was 0 bytes in size. 

My test message had a 17MB file attached. Progress bar went up fast until about 31%, and from then on, it took almost 2 minutes to send the message. 

Mailserver being sendmail, a linux box (pentium, about 1G), on the local network (100M) and no other load on the server machine, the sending machine or the network. 

Any other suggestions on how to dig deeper into this ?

used this from a command prompt opened in the Mozilla thunderbird installation dir : 
==================
set NSPR_LOG_MODULES=protocol:5
set NSPR_LOG_FILE=c:\tmp\filename
start mozilla

Comment 15

[alanmoz]

Refer to 2007-09-27 - the problem was fixed for receiving, but not for sending.  Prior to this, it was both directions.

Alan

Comment 16

[alanmoz]

Sorry - 2005-09-27, not 2007....

Alan

Comment 17

[Wayne Mery (:wsmwk)]

(In reply to comment #14)
> I've just run a test on this. 
> 
> I followed the instructions at
> http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#smtp to the
> letter, but got a log file that was 0 bytes in size. 
>
> ==================
> set NSPR_LOG_MODULES=protocol:5
> set NSPR_LOG_FILE=c:\tmp\filename
> start mozilla
 
protocol must be changed to smtp (going to server) or imap (coming from server).
suggest changing 
 start mozilla
to
 mozilla


> My test message had a 17MB file attached. Progress bar went up fast until about
> 31%, and from then on, it took almost 2 minutes to send the message. 

I'm not a mail expert, but I know some sites throttle the sending of large messages - not sure how that is done.

Comment 18

[Wayne Mery (:wsmwk)]

(In reply to comment #15)
> Refer to 2007-09-27 - the problem was fixed for receiving, but not for sending.
>  Prior to this, it was both directions.

works for you at which end - SERVER receiving, or client PC receiving?

Comment 19

[alanmoz]

Client (the offending browsers) receiving.  As I said, the original problem was both ways, then it became the current issue - client receiving OK, client sending poor.

Alan

Comment 20

[alanmoz]

To put this into perspective, the slow speed experienced from client to server is still roughly equivalent to T1 speed.  So anyone using T1, ADSL or slower to reach their ISP's server will likely never notice the issue.  It only shows up when the connection is 100 meg/sec.

Comment 21

[Wayne Mery (:wsmwk)]

Attachment: log smtp for thunderbird

two choices:

1. obtain protocol log - copy attached file to thunderbird directory, stop thunderbird, run file to start thunderbird, kill send part way through - 30 seconds? (we don't need a 2meg log)

2. sniff both clients mail send with http://www.wireshark.org/ and compare the logs - zip and attach to this bug report

until then, there's nothing here.

Comment 22

[alanmoz]

Choice 3 - I can simply forget about it.

It's been 4 years.  I select option 3.

Comment 23

[Andrei Boros]

I'm gonna try creating a report with wireshark this weekend. 

Comment 24

[Andrei Boros]

Attachment: TB generated log

Log file generated with the proposed .BAT file. 
TB client, linux/sendmail server, 100mbit lan (no other traffic). 

The first message was small, attachement <0.5M. 
Second message was larger, attachement ~9M. 
Having a controlled environment and no other traffic, I was able to measure traffic (iptraf) at about 900kbps for this transfer.

Comment 25

[Andrei Boros]

Just after that, I did a new test. Same messages, same sizes, all traffic captured with Ethereal. 

1) 0.5M message, ethereal says 4M avg throughput
ftp://ftp.andrix.ro/pub/tests/test-small.libpcap.gz (485k)

2) ~8M message, ethereal says 0.6M avg throughput and shows a _very_ constant throughput graph. I don't know where the limitation originates from
ftp://ftp.andrix.ro/pub/tests/test-large.libpcap.gz (7884k)

Hope it helps. 
Otherwise please tell me what other tests should I perform.
Thank you.

Comment 26

[Wayne Mery (:wsmwk)]

qawanted

Comment 27

[Andrei Boros]

I can confirm this issue on many installations, both linux and windows, and versions since 2.0 up to the most recent one ... 

As a comparison, on the same computer, same network, same server, MS Outlook sends the file to the server much faster then TB. 

To me it looks as if there would be something like a token bucket traffic shaper in the sending code of TB, at least the behaviour is similar to something like this. 

Comment 28

[Ludovic Hirlimann [:Usul]]

(In reply to comment #27)

> To me it looks as if there would be something like a token bucket traffic
> shaper in the sending code of TB, at least the behaviour is similar to
> something like this. 

What kind of server are you talking to ? Are you authenticating against that server (if yes , how) ? Is the issue still present with recent builds of Thunderbird 3.0 (http://www.mozillamessaging.com/en-US/thunderbird/early_releases/downloads/)

Comment 29

[Andrei Boros]

I'm talking to a linux based sendmail, without authentication or SSL/TLS. 
Issue affected both win32 and linux builds of TB. 

Haven't tested TB-3.0 yet, but I'll give it a try.

Comment 30

[Wayne Mery (:wsmwk)]

Andrei, can your reproduce using version 3? 

If so ...
0. retest with all firewall and antivirus disabled/off
1. please get version and name of system you are sending to
2. please get log using "timestamp,smtp:5" per https://wiki.mozilla.org/MailNews:Logging

Comment 31

[Andrei Boros]

I have prepared a new test case in the following context : 

Server : Linux Slackware 12.2, sendmail 8.14.1, located on the same network and tested at an hour when network load and server load are low. During tests, server loadavg was < 0.3 (intel P4@2.66GHz, 1GB RAM, lan100)
Temporarily disabled maximum message limit. 

Client 1 : Linux Slackware 12.0, Thunderbird 2.0.0.23, intel P4@2.0GHz, 1GB RAM, lan100, connected on the same LAN and same 8 port switch with the mailserver.
Accessing mailserver on port 25, pure smtp

Client 2 : WInXP, Thunderbird 3.0.4 (intel core2 @2.0GHz, 2GB RAM, lan100), connected on same network, though accessing the mailserver via a NAT router.
Accessing mailserver on port 465, smtp/SSL

Client 1 :
Prepared the env, started TB. Started wireshark. 
Send message with 10MB attachment. 
Send message with 48MB attachment. See I/O graph from wireshark. 
(saving of message to Sent folder, via IMAP to same server was visibly much faster then the sending was, though I didn't time it)

Client 2 : 
Prepared env, started TB. Started wireshark. 
Send message with 10MB attachment. 
Send message with 48MB attachment. See I/O graph from wireshark. 

Note : I edited the logs to mask server and client IP addresses.
See attached files for more info. I've put them all into a ZIP for grouping.

Comment 32

[Andrei Boros]

Attachment: New comparative speed tests on SMTP, TB2, TB3

Attachment with test results.

Comment 33

[Wayne Mery (:wsmwk)]

(FTR, I withdraw my comment 11)

Andrei, do you still see this when using version 7 or newer?

Comment 34

[Andrei Boros]

I have made several tests with TB-7.0.1 and the mail transfer speed to the SMTP server seems ok. 
Transfer speed is constant and uses best available speed according to the bandwidth with the server. (tested with speed >23Mbit with a 35Mbyte attachment). 

IMO this bug seems to have been FIXED. 
(or other changes in the codebase have indirectly fixed the issue). 

Thank you.

Comment 35

[Joshua Cranmer [:jcranmer]]

RESOLVED WORKSFORME per comment 34.