1083233 - Composed message then during "Sending Mail" dialog, progress meter stalled/hung near end of "Mail sent successfully" sometimes followed "(Not Responding)" in title bar with blank screen. Issue mostly when sending message. Worse with attachments.

Status: RESOLVED INVALID

(Thunderbird :: Message Compose Window, defect)

Description

[Mark Sicignano]

User Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:32.0) Gecko/20100101 Firefox/32.0
Build ID: 20140923175406

Steps to reproduce:

I'm using Windows 8.1 on a Core i7. Also run Thunderbird on a Windows 7 Core i7 machine as well. Plenty of RAM on both 16 and 8 respectively.

This bug happens spuriously, most often when I am sending an email.


Actual results:

 I see the progress bar that says that the message is sending. If I click on a window, my main TB window goes white and the dialog box often just hangs, but looks like it's not being drawn correctly. In the screen case it appears to have disappeared while the main TB window remains white for a while. Within 30-60 seconds, TB finally responds and redraws the main window. 

I have no idea what is hanging it up.

I have seen occasional hangs when typing messages that will cause TB to go unresponsive for similar times 30 seconds or longer. I cannot come up with a reasonable list of steps to reproduce it reliably.  In the most recent case, I was replying to an email and there were no attachments. Just text in my reply.

http://screencast.com/t/EAcNQV8gl8wP


Expected results:

It should have quickly sent the message, and returned me back to the main Thunderbird window so I can continue working.

Comment 1

[Mark Sicignano]

I just want to add another screencast which shows mostly similar, but slightly different behavior, on the same laptop mentioned above.

http://screencast.com/t/6CABYU93nVL

Keep in mind, this happened, then I started up the screen capture, so this starts about 15 seconds into the "hang". Again, mainly just a short textual message. Not consistent behavior. But about 1 time out of every 10? Maybe less frequent than that.

Comment 2

[Wayne Mery (:wsmwk)]

Start *Windows'* safe mode with networking enabled
- win8 http://windows.microsoft.com/en-US/windows-8/windows-startup-settings-including-safe-mode
- win7 http://windows.microsoft.com/en-US/windows7/Start-your-computer-in-safe-mode
- XP http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/boot_failsafe.mspx

Still In Windows safe mode, start thunderbird in safe mode
- http://support.mozillamessaging.com/en-US/kb/safe-mode

Does problem go away?

Also, how big is your sent folder?

Comment 3

[Mark Sicignano]

Wayne,

The problem is intermittent enough that I would have to spend 2-3 hours in safe mode with networking and that would hinder me getting any of my real work done. ;-)  And if all went well, it wouldn't tell us anything. I would have to go a full day or two to be comfortable that it wasn't happening in Safe Mode.

When it happened today, I fired up Procexp, and created a "full dump", but that file is kind of big to send to you (350MB) and it contains a lot of personal information.

I'm not set up with developer tools at he moment either, and I'm not a Thunderbird development contributor (other than reporting bugs as I hit them).

BTW, my folders in general on this computer are pretty small. This laptop is not my main PC, so I have my sent folders set to flush out emails after 30 days or so, automatically.  One is 11 MB. Another is 8.5 MB.

I am using Avast (Free), and I also run CCleaner every so often.

I'm hoping somebody else can duplicate this problem and be of a bit more assistance. If you think there is anything else I can do to help you, let me know.

Comment 4

[Wayne Mery (:wsmwk)]

(In reply to Mark Sicignano from comment #3)
> ...
> I am using Avast (Free), and I also run CCleaner every so often.

any third party firewall?  proxy?

> I'm hoping somebody else can duplicate this problem and be of a bit more
> assistance. 

There are many possible causes and, despite your great details, we don't know enough about your situation or your cause to know how to reproduce it.

> If you think there is anything else I can do to help you, let me
> know.

Run Thunderbird safe mode is a start

Comment 5

[Mark Sicignano]

No third party firewall. No proxy. It happens spuriously with different email servers on different hosting companies.

I appreciate everything you're saying and I will try to find more information for you, and I will try to find time to run Thunderbird in safe mode as you described.

Comment 6

[Mark Sicignano]

Wayne,

OK, I just did this in Thunderbird Safe Mode and duplicated it on the first email I tried.

I did attach a 196 KB jpeg to the email.

It hung there. As soon as it hung, I started up Jing and recorded this screencast of it hanging there and eventually it goes away. Pretty boring screencast, but you'll see that it's just sitting at 98% for some time, and finally closes.

http://screencast.com/t/2C8v8bC3Azov

I will try to go into Windows 8.1 in Safe Mode AND Thunderbird Safe Mode to let you know if I can duplicate it there as well.

-mark

Comment 7

[Mark Sicignano]

More results: 

So I went into Windows 8.1 safe mode with networking, connected to my network and started Thunderbird in Safe mode as well. I sent about 5-10 emails with attached file. They all seemed to go through without a hitch. 

I restarted and came back into Windows 8.1 normally.  Went into TB in safe mode again. Got it to fail pretty reliably. Not every single time, but about 50% of the time. It may have something to do with attachments, or larger sized emails. 

Just for kicks I suspended my Avast shield for 10 minutes, and it still hung. I had suspected that might have been a factor. Not sure that it is.

Here is a full window (TB) screen cast where I show a number of aspects of the bug. Note that the first email went through, but the second time, it hung.

I'm sending out via comcast.net. So if you need more information, let me know.

http://screencast.com/t/aDnHxy5B

And my machine configuration is here: http://speccy.piriform.com/results/tXn43mNBVRwpNIbLOMlluAS

Hope this helps.

Comment 8

[Mark Sicignano]

**UPDATE**: I just had TB hang for about 30 seconds about 4 times in 10 minutes, all while working with the same mail. I'm currently on 31.2.0, but it's been happening to me in other recent versions as well.

1) I created an email to somebody, and attached a 4 MB file to it. I then switched out of the window to go to a Firefox window, and TB went into hang mode with a spinning cursor for about 30 seconds.

2) Later on the same email, I was editing it and switched windows again, and it hung the same way.

3) I disabled Avast's Active Protection (File System Shield) for 10 minutes. Did a bunch of window switching and didn't see any problems. But then I thought I would File->Save the message. It hung on me again with Avast temporarily disabled.

4) By now I have sent that message, but I created another one while I'm entering in this information. A plain text message. Enter some junk in the subject, enter some junk in the body. Then just hitting Ctrl+S to do a save... over and over. Eventually, TB hangs on me again in the same way. But having an attachment (I show a < 1 MB attachment that immediately creates the problem on the very next save) seems to make it easier to duplicate.

http://screencast.com/t/ikihtZVF

Comment 9

[Mark Sicignano]

So I ran procmon and captured the events where I send a couple of PSD files (about 5 MB) via email. Hoped it would catch some function taking a looooong time. It did.

The ProcMon.exe log file. Native format so you can open it in ProcMon.exe:

http://sicinivs.s3.amazonaws.com/tb/HangOnSendLogfile.PML

When you open the file in ProcMon it will show you filter criteria; uncheck the one for result is "SUCCESS" because this function call takes 77+ seconds to finish, but the result is SUCCESS.

Fastest way to get to the line in the log file is to search for 77.1 (the number of seconds it took)

I really hope this helps. It is a constant pain.

If you don't to or can't run procmon.exe, here is the vital information:

https://www.evernote.com/shard/s2/nl/633458/0b6d0a02-8d3e-402e-b219-be851cc8e63c

Comment 10

[Wayne Mery (:wsmwk)]

You may have multiple issues if disabling AVAST did not help, but Windows safe mode did help.  I have also seen cases where disabling the antivirus is not sufficient to help, because the package has Thunderbird addons or other hooks.

But regardless, if you cannot reproduce when in Windows Safe Mode, then the problem is not THunderbird.

Comment 11

[Mark Sicignano]

Some more information on this one that should be helpful.

I wouldn't say that Thunderbird doesn't have a problem here, but I believe I managed to find the issue.

At first I suspected that it had something to do with Avast scanning the larger attachment file, but that really wouldn't account for 77 seconds in delay and hang for Thunderbird. Especially since I think many of the files that would hang are ones excluded by Avast anyway, and I went as far as making sure Avast had all of the email related folders and temp folders excluded to try to eliminate the problem.

What I ultimately found was that Windows Search Indexing was grabbing the file.

The email was being sent. In fact, in one case, I was talking to somebody on the phone. They received the file and were looking at it, meanwhile, Thunderbird was hanging for more than a minute AFTER the file was successfully sent and received by somebody else.

I have an SSD for a primary drive, and a fast conventional hard drive for a secondary drive. My SSD being more limited in space, I moved my mail folder to the D: drive. But I never excluded my mail folders from being indexed. This appears to be the issue.  I didn't go back and forth to test it out. I merely excluded the folders and said, "Let's see if this is part of the problem."  Since then I have not seen this problem. 

If a developer wants to test it out, give this a try to see what happens. 

I think it still could be a bug in Thunderbird. You have to wonder why Thunderbird sends the email and then when it goes to delete the temporary attachment or email file, it hangs for 77 seconds (in my example). The search indexer... It can't be holding onto a file for this long, can it? I don't know.

Good luck. I hope this information is helpful.

Comment 12

[Wayne Mery (:wsmwk)]

Nice job, and thanks for doing that detective work.  My fault for not pointing you to https://wiki.mozilla.org/Thunderbird:Testing:Memory_Usage_Problems  See item 6, second item.  

So when a profile or mail folder is moved from the default location, there's nothing Thunderbird can do to protect you against Microsoft's or other vendor's change in behavior. In other words if your data is not in %appdata% then all bets are off. There can also be bad consequences if %temp% has been moved, but I forget what they are just now.  In both cases, you can get hammered by both AV and other software.

aceman, do you know if we warn users in any way when they specify a non-default location, and if not do we have a bug report on it?

Comment 13

[:aceman]

(In reply to Wayne Mery (:wsmwk) from comment #12)
> aceman, do you know if we warn users in any way when they specify a
> non-default location, and if not do we have a bug report on it?

I don't think we do and I do not see any reason to warn about that (I'd even think outside of profile is a safer location for emails). The OS indexing would happen regardless of the location. Or do you think the Windows indexer excludes TB profiles by default? In that case the user can exclude the real email location too.

I think the reporter should check if the Indexer is holding onto the file while it is in the Temporary files. I think Temp should be excluded from indexing.

Comment 14

[Wayne Mery (:wsmwk)]

(In reply to :aceman from comment #13)
> (In reply to Wayne Mery (:wsmwk) from comment #12)
> > aceman, do you know if we warn users in any way when they specify a
> > non-default location, and if not do we have a bug report on it?
> 
> I don't think we do and I do not see any reason to warn about that 

I'm not sure yet either, or how we would do it if we wanted to. Just exploring, because we offer the users a footgun here with no guidance.

> (I'd even think outside of profile is a safer location for emails). 
Safer how?

> The OS indexing would happen regardless of the location. Or do you think the Windows indexer
> excludes TB profiles by default? 

It is documented that the Windows OS does not index %appdata% directories. This can be seen at setting in Advanced Properties for each folder. for %appdata% it is unchecked by default. %temp% is part of the %appdata% structure, so it too is not indexed.
 
(I'm not sure data on networked drives gets indexed)

> In that case the user can exclude the real email location too.
Yes, definitely. 

> I think the reporter should check if the Indexer is holding onto the file
> while it is in the Temporary files. I think Temp should be excluded from
> indexing.

see above. However, AV software have been known to scan %temp%

Comment 15

[:aceman]

(In reply to Wayne Mery (:wsmwk) from comment #14)
> > (I'd even think outside of profile is a safer location for emails). 
> Safer how?
Unless you also put your profile outside of appData, I feel appdata is considered a second class location:
1. it is hidden from the user by default (at least on Windows XP). So user is not even aware there is something interesting in there.
2. most apps put disposable data there. Not really temporary, but still possible to be recreated, like some settings, UI placements, etc. Yes, like FF/TB profile, except some important files there.
3. I've seen corporate IT does not consider it to be worthy user's data and does not back it up or migrate it with the user (e.g. to new machine).

So all of this increases the risk for somebody somewhere to forget to migrate/not delete his messages hiding in appdata.

> > The OS indexing would happen regardless of the location. Or do you think the Windows indexer
> > excludes TB profiles by default? 
> 
> It is documented that the Windows OS does not index %appdata% directories.
> This can be seen at setting in Advanced Properties for each folder. for
> %appdata% it is unchecked by default. %temp% is part of the %appdata%
> structure, so it too is not indexed.
Hmm, did I say something about appdata being second class?:) So even Windows does not think there is useful user's data in there. So then, why do we like to store messages there? Important user data that must be protected/backed up and be visible to the user? Do we want help MS to train users to be unaware where their data really is? Yeah, maybe there are statistics that the mbox files must be hidden from casual user as he would more likely just delete them as unknown junk than keeping them safe? Maybe. Does MS expect all programs that put useful data into appdata to provide hooks/exceptions/adapters into the Indexer? Maybe.

But I consider the emails important so they should be on the same safety class (filesystem/location) as any other created data/documents/photos. But maybe because I am not on Windows so I am not hit by these Windows catches:) And I have only POP3 so the messages are important, not disposable data still kept on the server (like IMAP).

> (I'm not sure data on networked drives gets indexed)
> 
> > In that case the user can exclude the real email location too.
> Yes, definitely. 
> 
> > I think the reporter should check if the Indexer is holding onto the file
> > while it is in the Temporary files. I think Temp should be excluded from
> > indexing.
> 
> see above. However, AV software have been known to scan %temp%
Sure, AV should, as malware can unpack anywhere. But indexing should not.

Comment 16

[Wayne Mery (:wsmwk)]

(In reply to :aceman from comment #15)
> (In reply to Wayne Mery (:wsmwk) from comment #14)
> > > (I'd even think outside of profile is a safer location for emails). 
> > Safer how?
> Unless you also put your profile outside of appData, I feel appdata is
> considered a second class location:
> 1. it is hidden from the user by default (at least on Windows XP). So user
> is not even aware there is something interesting in there.

Welcome to the world of Windows. And true for every version.


> 2. most apps put disposable data there. Not really temporary, but still
> possible to be recreated, like some settings, UI placements, etc. Yes, like
> FF/TB profile, except some important files there.

Beware 
- confusing %appdata% with c:\users\blah\appdata
  %appdata% is c:\users\blah\appdata\roaming
- not all data in c:\users\blah\appdata is alike, eg
  roaming vs c:\users\blah\appdata\local

local is considered to be disposable, non-persistant.  
and roaming considered to be very much durable, which is where TB profile and folders are kept.


> 3. I've seen corporate IT does not consider it to be worthy user's data and
> does not back it up or migrate it with the user (e.g. to new machine).
>
> So all of this increases the risk for somebody somewhere to forget to
> migrate/not delete his messages hiding in appdata.

Yup. Also, users have even blown away their profiles accidentally after they move them somewhere else and forgotten they had moved it.

> > > The OS indexing would happen regardless of the location. Or do you think the Windows indexer
> > > excludes TB profiles by default? 
> > 
> > It is documented that the Windows OS does not index %appdata% directories.
> > This can be seen at setting in Advanced Properties for each folder. for
> > %appdata% it is unchecked by default. %temp% is part of the %appdata%
> > structure, so it too is not indexed.
> Hmm, did I say something about appdata being second class?:) So even Windows
> does not think there is useful user's data in there. So then, why do we like
> to store messages there? Important user data that must be protected/backed
> up and be visible to the user? Do we want help MS to train users to be
> unaware where their data really is? Yeah, maybe there are statistics that
> the mbox files must be hidden from casual user as he would more likely just
> delete them as unknown junk than keeping them safe? Maybe. Does MS expect
> all programs that put useful data into appdata to provide
> hooks/exceptions/adapters into the Indexer? Maybe.

If by "second class" you mean it's treated as being less important, I don't think that's a fair statement nor a fair conclusion.  Windows *and applications* hides all kinds if things from Windows users - being hidden doesn't make it less or more important (except in the case of system files). And on linux, well, not the same.


> But I consider the emails important so they should be on the same safety
> class (filesystem/location) as any other created data/documents/photos. But
> maybe because I am not on Windows so I am not hit by these Windows catches:)
> And I have only POP3 so the messages are important, not disposable data
> still kept on the server (like IMAP).

yes

Comment 17

[:aceman]

(In reply to Wayne Mery (:wsmwk) from comment #16)
> Beware 
> - confusing %appdata% with c:\users\blah\appdata
>   %appdata% is c:\users\blah\appdata\roaming
> - not all data in c:\users\blah\appdata is alike, eg
>   roaming vs c:\users\blah\appdata\local
> 
> local is considered to be disposable, non-persistant.  
> and roaming considered to be very much durable, which is where TB profile
> and folders are kept.
Yeah, sorry. I live in the C:\Documents and Settings\user\Application Data\ word, where there is no Roaming dir :)
So I put user's home and data on a different partition for apps where it is possible and useful. Easy to contain it and backup.

> > > > The OS indexing would happen regardless of the location. Or do you think the Windows indexer
> > > > excludes TB profiles by default? 
> > > 
> > > It is documented that the Windows OS does not index %appdata% directories.
> > > This can be seen at setting in Advanced Properties for each folder. for
> > > %appdata% it is unchecked by default. %temp% is part of the %appdata%
> > > structure, so it too is not indexed.
> > Hmm, did I say something about appdata being second class?:) So even Windows
> > does not think there is useful user's data in there. So then, why do we like
> > to store messages there? Important user data that must be protected/backed
> > up and be visible to the user? Do we want help MS to train users to be
> > unaware where their data really is? Yeah, maybe there are statistics that
> > the mbox files must be hidden from casual user as he would more likely just
> > delete them as unknown junk than keeping them safe? Maybe. Does MS expect
> > all programs that put useful data into appdata to provide
> > hooks/exceptions/adapters into the Indexer? Maybe.
> 
> If by "second class" you mean it's treated as being less important, I don't
> think that's a fair statement nor a fair conclusion.  Windows *and
> applications* hides all kinds if things from Windows users - being hidden
> doesn't make it less or more important (except in the case of system files).
> And on linux, well, not the same.
Yes, that is what I mean. Hiding it from the user means he does not need to care about it as it is not important. Maybe it isn't so, but from my experience the users think so. I do not feel good on a PC unless I can unhide hidden folders :) I need to know where stuff is. So I hate e.g. Android for this reason (hides things unless you root it). But I am probably too old already :)

On linux TB does it similarly. The profile is in a .thunderbird directory, which is also hidden by default in most file managers.

Comment 18

[Wayne Mery (:wsmwk)]

Mark, does the updated summary accurately capture/summarize the situation? 
And, do you see problems at other times when not sending a message?

Composed message then during "Sending Mail" dialog, progress meter stalled/hung near end of "Mail sent successfully" sometimes followed "(Not Responding)" in title bar with blank screen.  Issue mostly when sending message. Worse with attachments.

Comment 19

[Mark Sicignano]

Wayne,

Read through everything with this bug and it all makes sense to me. I moved my mail folder off of my cramped SSD C: Drive and onto a regular drive with more space, D:\Users\mark\Mail\Profiles.

Windows Search Indexing was scanning that folder by default because it wasn't excluded, and I don't realize it by making the connection. Appears that when I would send an email, and in particular with an attachment, that Windows Search would grab onto files... maybe outbox, maybe the attachment files, and would be indexing them while Mozilla had to wait for a lock to be released before it could finish the attachment and send the message. This appears to be what was happening and is my "theory". I didn't put any heavy debugging into this or sic any sysinternals tools on them to find out more details. 

However, since excluding my mail profile from the Windows Search Index, I have had NO instances of this happening anymore, so I'm quite confident that this was my problem. I was not haveing this problem on other PC's because I didn't move the data mail profiles on those machines.

If this doesn't address all of your questions Wayne, just buzz me back and I will give you whatever other information you need.

Comment 20

[Wayne Mery (:wsmwk)]

Thanks for all the good information