620123 - Freeze on send with PDF attachment

Status: RESOLVED WORKSFORME

(Thunderbird :: General, defect)

Description

[dcm-mozilla]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7

Win 7 64 bit, Thunderbird 3.1.7, IMAP email with 1and1. 

Sending an email with a PDF attachment sometimes results in Thunderbird freezing, requiring a process kill to terminate Thunderbird. This has been a problem for a while. About four weeks ago, on my last attempt to find a solution, I saw a web report that it only occurred with PDF attachments, and that has been my observation since then. (Before I had assumed that it could happen with any attachment, but I now buy into the idea that it is only with PDFs.) 

Reproducible: Sometimes

Steps to Reproduce:
1. Compose an email.
2. Attach a PDF.
3. Hit send.
4. Repeat until problem occurs. It is possible that starting composition on another email or doing a search on an email folder may increase the odds of a freeze, but I have certainly seen it happen where the only thing I do is press send and wait.

Actual Results:  
Thunderbird freezes and requires process kill. Doesn't appear to be a consistent point in the process of sending the email (length of green progress bar), but I think it is probably true it is during the actual PDF file transmssiion, not the body of the email, since my email body tens to be short (few paragraphs) and the PDFs tend to be larger (20k to MBytes) and the green progress bar % is pretty random at the crash point.

Expected Results:  
Sent the email and not frozen.

Comment 1

[WADA:World Anti-bad-Duping Agency]

For SMTP sending by Tb and SMTP receiving by SMTP server, attachment or not is absolutly irrevant. It's because Tb merely sends 4KB data until whole mail data is sent, and SMTP server merely receives 4KB data until whole mail data arrives. So, if issue during sending, it's relevant to total mail data size only.

If PDF only problem, it is usually caused by anti-virus/anti-scam processing.
In you case, phenomenon is client side freeze(perhaps CPU 100% like one), it may be caused by "outbound mail scanning of your anti-virus software", if PDF only problem.

Really PDF only problem? No problem if multiple jpeg file attachments, even if large mail? Size of PDF is always larger than other files in your case, isn't it?

Mail sending roughly consists of;
(1) Generation of mail data stream
(2) SMTP send of mail data stream
(3) Save sent maill copy to Sent folder.

Check next, please.
(1) Create mails of different size.
    1-1. Compose a mail-64KB, attach some jpeg files, "Send Later",
         move mail in Outbox of "Local Folders" to folder XXX(mail size=64KB)
    1-2. Repeat 1-1. for 128KB mail, 256KB mail, 512KB mail, 1MB mail.
         (if required, create 2MB mail, 4MB mail, ...)
(2) Send mails.
    2-1. Edit As New at XXX, change From:/To:, send mail-64KB to yourself.
    2-2. Repeats 2-1. for for 128KB mail, 256KB mail, 512KB mail, 1MB mail.

Is your "freeze" reproduced by above test?
When your "freeze" happens, what is CPU utilization of system and process of Tb? 100%?
Does you SMTP server requests SSL? If yes, does your problem occur with SMTP server of non SSL?
When did your problem start to occur? After upgrade from Tb 2 to Tb 3.0 or Tb 3.1? Or after upgrade from Tb 3.0 to Tb 3.1? 

If not PDF only problem, and if "CPU 100%" problem, and if SSL only problem, and if after upgrade from Tb 2.x to Tb 3.0 or Tb 3.1, is your problem similar phenomenon to bugs listed in Dependency tree for Bug 538283 and Bug 538283 itself?

If similar phenomenon, is you problem reproduced with network.tcp.sendbuffer=4096 and network.tcp.sendbuffer=65536?

Comment 2

[WADA:World Anti-bad-Duping Agency]

Woops. Correction of step 2-1.
2-1. Edit As New at XXX, change From:/To:, "Send Latr" of mail-64KB to yourself,
     "Send Unsent Messages".

Comment 3

[slm]

I have seen similar send freezes when trying to send attachments where file size was about 10MB. File type was not pdf, so my guess it's more to do with file size than file type. These freezing sends can be succesfully sent later once TB process has been killed first (I use WinXP SP3).  

Then some guessing: Could it be that if you have several mail accounts, automatic send/receive (that happens say every 10 minutes) for these mail accounts begins while a send is in progress and this somehow gets stuck?

Comment 4

[dcm-mozilla]

That is an interesting idea ("several mail accounts, automatic send/receive (that happens say every 10 minutes) for these mail accounts begins while a send is in progress and this somehow gets stuck").

I have 7 accounts and check for email every 5 minutes. So if there is some type of conflict between checking and sending, it would happen randomly and could be in proportion to file size.

Regarding file size, I have had cases where the file size has been small (under 2 MB.)

I have not yet had a chance to run the tests in your 2010-12-18 21:03:46 PST message.

Comment 5

[slm]

In my case sending those big attachments does take quite a while, easily more than a minute. So it is quite possible that send/receive can start during this long send. Nowadays I try to remember to press Send/Receive first, then Send.

Comment 6

[Wayne Mery (:wsmwk)]

dcm, Do you still see this problem?  If no, the key questions are still unanswered afaict, so we still need more of your great feedback ...

(In reply to WADA from comment #1) 
> Check next, please.
> (1) Create mails of different size.
>     1-1. Compose a mail-64KB, attach some jpeg files, "Send Later",
>          move mail in Outbox of "Local Folders" to folder XXX(mail size=64KB)
>     1-2. Repeat 1-1. for 128KB mail, 256KB mail, 512KB mail, 1MB mail.
>          (if required, create 2MB mail, 4MB mail, ...)
> (2) Send mails.
>     2-1. Edit As New at XXX, change From:/To:, send mail-64KB to yourself.
>     2-2. Repeats 2-1. for for 128KB mail, 256KB mail, 512KB mail, 1MB mail.
> 
> Is your "freeze" reproduced by above test?
> When your "freeze" happens, what is CPU utilization of system and process of
> Tb? 100%?
> Does you SMTP server requests SSL? If yes, does your problem occur with SMTP
> server of non SSL?
> When did your problem start to occur? After upgrade from Tb 2 to Tb 3.0 or
> Tb 3.1? Or after upgrade from Tb 3.0 to Tb 3.1? 
> 
> If not PDF only problem, and if "CPU 100%" problem, and if SSL only problem,
> and if after upgrade from Tb 2.x to Tb 3.0 or Tb 3.1, is your problem
> similar phenomenon to bugs listed in Dependency tree for Bug 538283 and Bug
> 538283 itself?
> 
> If similar phenomenon, is you problem reproduced with
> network.tcp.sendbuffer=4096 and network.tcp.sendbuffer=65536?

Comment 7

[Joshua Cranmer [:jcranmer]]

RESOLVED INCOMPLETE due to lack of response to previous question. If you feel this change has been made in error, please respond to this bug with your reason why.

Comment 8

[dcm-mozilla]

It still does it. I'll try to run the requested tests this weekend.

Comment 9

[dcm-mozilla]

> Check next, please.
> (1) Create mails of different size.
>     1-1. Compose a mail-64KB, attach some jpeg files, "Send Later",
>          move mail in Outbox of "Local Folders" to folder XXX(mail size=64KB)
>     1-2. Repeat 1-1. for 128KB mail, 256KB mail, 512KB mail, 1MB mail.
>          (if required, create 2MB mail, 4MB mail, ...)
> (2) Send mails.
>     2-1. Edit As New at XXX, change From:/To:, send mail-64KB to yourself.
>     2-2. Repeats 2-1. for for 128KB mail, 256KB mail, 512KB mail, 1MB mail.
> 
> Is your "freeze" reproduced by above test?

Yes. If froze on the 4 MB attachment (one 4.1 MB jpg). That is with STARTTLS.

> When your "freeze" happens, what is CPU utilization of system and process of
> Tb? 100%?

On an i7, the TB CPU perf is 0% and the memory is not changing (147MB on this occurrence). Total CPU perf for the system is 4% from Chrome. (I have seen no correlation between network activity from other processes and the TB freeze. It freezes when I am doing nothing else on the computer but sending the email.) My email provider is 1and1 and I'm using IMAP.

> Does you SMTP server requests SSL? If yes, does your problem occur with SMTP
> server of non SSL?

Normally my SMTP is StartTLS. Freezes are frequent with file sizes of roughly 2, 3, 4 MB  and larger. 

'None' appears to work better, but it eventually froze on the fourth 16MB email I tested. Unlike the case with "StarTTLS" (CPU at 0%), when 'None' froze, the CPU % for TB was 13% (on a i7 = 4 cores, 8 threads, so that is equivalent to 1 thread pegged at 100%.) I tried to repeat this failure but four more 16 MB ran OK and I'm out of time tonight.



> When did your problem start to occur? After upgrade from Tb 2 to Tb 3.0 or
> Tb 3.1? Or after upgrade from Tb 3.0 to Tb 3.1? 

I don't remember for sure, but I think it started with TB 3. (It still happens (TB 9.0.1). I happens on netbooks through i7's - cpu performance and memory does not appear to matter. It happens on Win XP and Win 7. 
 
> If not PDF only problem, and if "CPU 100%" problem, and if SSL only problem,
> and if after upgrade from Tb 2.x to Tb 3.0 or Tb 3.1, is your problem
> similar phenomenon to bugs listed in Dependency tree for Bug 538283 and Bug
> 538283 itself?

I seem to have two things going on - CPU 0% when it freezes on send with STARTLS and the thread being pegged at 100% when it freezes on send with 'None'.

SSL seems to also be pretty stable - I will test more in the next few days but I was able to send multiple 16 MB files.

> If similar phenomenon, is you problem reproduced with
> network.tcp.sendbuffer=4096 and network.tcp.sendbuffer=65536?

I'll check.

Comment 10

[WADA:World Anti-bad-Duping Agency]

Mail sending consists of;
(1) Create mail data stream when Send is requested.
(2) Login to SMTP server and sends mail data to server and waits for OK from server.
(3) Copy sent mail data to Sent folder.

To surely rule out issues in (1) and (3), please do next.
At (1), execute "Send Later" always in checking.
At (3), disable "save sent mail copy to Sent folder", or always use local mail folder as "folder to save sent mail copy".

And, at step (2) by "Send Unsent Messages", always get SMTP log with timestamp and always check SMTP log, please.
  "S" in SMTP == Simple, so SMTP flow is simple.
  Login, declare mail size, notify mail sender's mail adress and mail
  recipiet's mail address to SMTP server, and ask for permisstion of mail
  data transfer, and when server says "Go ahead", sends all mail data to server,
  sends "." only line as "End of data indicator", waits for "OK" response
  from server which is indicator of "mail is acceppted", and logout.
Because actual mail size is written in SMTP log, you don't need to write down attached file type nor attached file size any more. And, with timestamp, when all mail was passed to server, when OK response was returned to server, ..., is always/easily be known by SMTP log.
Because mail data itself is not written in SMTP log, SMTP log is not so large, so you can always check it by text editor.

> I seem to have two things going on 
> - CPU 0% when it freezes on send with STARTLS
> and the thread being pegged at 100% when it freezes on send with 'None'.

Same SMTP server(supports both StartTLS & None)?  Or different SMTP server?
In both cases, was same auto-save setting, same Drfats folder, same Sent folder, used in your mail composition and mail sending?

If different SMTP server, route from yor PC to server is different. So, if router's bug exists at server side, problem due to router's bug can occur on one SMTP server only.
If router's bug is relivant, small tcp.sendbuffer=4096 is an effective workaround, because related router's bug is problem when more than 64KB data is sent at once, and if tcp.sendbuffer is less than 64KB, less than 64KB data is passed to server at once.
Rule out this case first please, before repeated try of very big mail sending test

Comment 11

[Wayne Mery (:wsmwk)]

10dcm-mozilla see questions in comment

Comment 12

[dcm-mozilla]

I am now on TB 10.0.2 and unable to reproduce the problem. (It was very easily reproduced before.) Don't close it yet - I want to run some more tests to be sure that 10.0.2 doesn't have it.

Comment 13

[dcm-mozilla]

I have checked further and a system with TB 7.0.1 that always had the problem is now fine. Also, a system with TB 9.0.1 that always had the problem is now fine. On that system, I tried TB 9.0.1 using NONE, STARTLS and SSL and all were OK. The 10.0.2 system is also OK (and always has been, I think.) All these systems are running Windows 7, but we've had the problem for a long time, including on Windows XP.

The only reasons I can see for why the 7.0.1 and 9.0.1 systems are now OK are either 1and1 (the email provider) fixed something causing it on their end, or the regular Microsoft patches to Windows 7 finally changed something that made the problem go away. 

All my tests were with the same email provider, 1and1, and that was the same provider that we used when we had problems. Since they handle all our email, I don't know if the problem was repeatable with other providers. 

Anyway, something changed since Feb. 6 to make this long standing problem go away (we've had the problem for years). 

The issue RESOLVED for me. Thanks.