Closed Bug 621083 Opened 13 years ago Closed 13 years ago

Very slow, nonresponsive (hourglass) displaying any screens (many minutes response) caused by Microsoft Security Essentials (MSE)

Categories

(Thunderbird :: General, defect)

x86
Windows XP
defect
Not set
major

Tracking

(Not tracked)

RESOLVED INVALID

People

(Reporter: gep2, Unassigned)

Details

(Keywords: perf, Whiteboard: [support])

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.59 Safari/525.19
Build Identifier: thunderbird.exe is 12,584,112 bytes long... 12/14/2010 filedate, 12:02am...trouble gettting to Help/About menu (or anything)

Have many filters, numerous (IMAP) mail accounts, local folders for mail archiving.  Thunderbird erases everything below the blue title bar in its frame, and can take many minutes to redisplay the frame.  Checking with task manager, thunderbird.exe is using negligible CPU time, about 27Mb of memory, is not generating page faults, has 452 handles (seems reasonable enough), 21 threads, and I/O read/writes/other and read bytes/write bytes/other writes all are not indicating any large amount of activity.  Looking at the local mail folders subdirectory, it does seem that Thunderbird is copying (but VERY VERY slowly) messages to a local mail folder (on the order of 200K bytes after several minutes!!).  So thunderbird.exe is not burning CPU cycles (this is a quad-core AMD Opteron 3.0+GHz processor), isn't generating an unreasonable number of page faults (10 in 30 seconds, just passed?)... it's almost as if it isn't getting dispatched by Windows, or it's waiting on something that isn't happening.  (I even tried setting the execution priority of thunderbird.exe to "high" and that will sometimes get a right-away screen update, once, but doesn't keep it going quickly).  Presently, thunderbird.exe has used 20:08 of CPU time, about 280M of RAM, 310M of peak RAM, 785,000 page faults, VM size 270M, 404 handles, 17 threads, 863,142 I/O reas, 667,442 I/O writes, 42,689,175 other I/O, 4.1Gb of I/O read bytes, 679Mb of I/O write bytes, 608Mb of I/O other bytes.  I can't imagine what it's doing, but it's not burning CPU or disk resources, as far as I can see.  Exiting and restarting Thunderbird sometimes helps a little, for a while... although it does take a long time (several minutes) to bring it back up, too.

Reproducible: Always




I update the software regularly, I think this is 3.1.7
odd problem. Thanks for the detail.

1. What happens if you start in safe mode? 
https://support.mozillamessaging.com/en-US/kb/Safe+Mode
2. what antivirus software do you run?
3. how much memory on your system?
Keywords: perf
Starting Thunderbird in safe mode is not significantly different.  For example, closing a message window tab might take 1m45s to redisplay another message, or to return to the message list.  The list might stay on the screen for 10s-30s before (without me clicking on anything) the window is erased, and it will take again 1m45s to display messages again.  Clicking the Help menu item, for example, might take 1m30s to 1m45s to display the Help menu.  I'm running Microsoft Security Essentials, RUbotted (from Trend Micro), Spybot: Search & Destroy Resident, and Super AntiSpyware (4.47.1000).  All show that my system is clean, other than the usual tracking cookies.  I have 8Gb of memory in this machine (found out after adding the extra 4Gb that XP only supports half of that, sigh).  

When I took down Thunderbird to bring it back up in Safe Mode (which it's still in), Task Manager showed for thunderbird.exe (again 3.1.7) about 45 mins of CPU used, 450Mb of memory approximately being used, 3.2 million page faults, 598 handles, 31 threads, 2.7 million reads totalling 14.7Gb, 2.1 million writes totalling 4.5Gb, 34 million I/O other totalling 451Mb.  (This had been up for most of a day).  CPU utilization (quad-core 3.0+GHz AMD 64-bit processor) is running typically 15%.  Since bringing it up in Safe Mode, taskmgr reports 4m33 CPU time, 272Mb memory in use, 301Mb peak memory, 360,575 page faults, VM size 257Mb, 519 handles, 30 threads, 453k reads totalling 2.27Gb, 77k writes totalling 204Mb, 2.1M I/O other totalling 44Mb.  I'm not seeing these numbers building rapidly (or any others for other programs for that matter) while I'm waiting for Thunderbird to respond.

I'm running S.M.A.R.T. on my C: hard drive, and it's showing the drive to be OK.  I have run defrag on this hard drive (a week or two ago) and that ran normally.  I checked Event Manager and am not seeing warnings of hard drive errors etc etc.

I did notice a warning about Thunderbird not being able to connect to localhost (I'm not sure which account that would be for, I have several mail accounts defined) and that might be related to some of the security software I'm running (so they can monitor incoming mails?).  Notwithstanding, I'm able to send and receive e-mails on my main accounts, at least, so I wouldn't think that's the problem on these accounts.  (My main two accounts are IMAP accounts).  

I do have numerous filters, and I can go into my lcoal mail folders subdirectory (on a Windows 2003 server mirrored drive, connected by 1Gb Ethernet) and can see messages being regularly archived there based on filters I have defined, so I know that filters are running.

One of my mail account Inbox files (under the Thunderbird/profiles/--/ImapMail/etc subdirectory) is 9.8Gb, which is large but the file has been larger and it seems that Thunderbird manages it OK.

I am a heavier user than many, I'm sure... I have in my local mail folders subdirectory tree some 250 mail folders totalling 180+ Gb... (but again, that's why I'm using a "serious" mail client like Thunderbird!)
Last night I took a video showing what the screen looks like... showing the screen with messages eventually appearing and then disappearing after just a few seconds... to reappear in typically about 1m45s later... the .AVI file is almost a gigabyte, nearly ten minutes long.  I'll be glad to upload it somewhere you direct, if it will help to understand what I'm seeing.
As a starting point, I suggest working through https://wiki.mozilla.org/Thunderbird:Testing:Memory_Usage_Problems
I went ahead and uploaded the video to Youtube... here's the link:
http://www.youtube.com/watch?v=9x7PICSh1PQ

I will go through the memory usage suggestions... I'm not sure there's any way to disable (temporarily) Microsoft Security Essentials without completely uninstalling it... I'll see that I can find.  But the memory usage (currently 288M, with a peak memory usage of 343M and about 1.6M page faults) don't seem particularly unreasonable based on the mail folders and accounts I have defined.

Is there a way to temporarily turn off my mail filters, without going and disabling each filter individually (which would be a pain, since there are so many...)?  I don't *think* that the filters are the main problem, since I would expect that those would show as either or both of CPU activity or disk activity, and those don't seem to be busy (based on Task Manager, anyhow) while I'm waiting for Thunderbird to respond.
note: much of the document covers non-memory performance issues.

> Is there a way to temporarily turn off my mail filters, without going and
disabling each filter individually (which would be a pain, since there are so
many...)?

rename the msgFilterRules.dat files that you find in ImapMail account folders and Mail account folders.

how many filters do you estimate you have?
The filters files (for my two main mailboxes) are about 35K and 80K bytes respectively.

I did remove several (4?) filter criteria which were based on message body content rather than header data (I don't know whether these result in rereading the entire folder for each such filter processed, or processing all the filters as the folder is processed in one pass sequentially... but I did anyhow use header criteria before the body criteria, which should have helped avoid a lot of unnecessary churning).

What finally seemed to fix the problem was that I configured Microsoft Security Essentials to not scan files in the Thunderbird temporary profile directory tree, and the local mail folders tree.  Apparently the way Thunderbird opens/closes/uses files results in Microsoft Security Essentials spending a LOT of time examining these (large) mail folder files (and apparently, over and over again).  I also told it to not scan the files in the thunderbird program directory (including thunderbird.exe).  I hope that by so doing I'm not preventing MSE from finding any malicious software perhaps arriving in e-mails.  

But I had to do SOMETHING, because Thunderbird was all but unusable the way it was.

BTW, it would be VERY helpful if Thunderbird would timestamp somehow when calling and returning from system services (opening, reading, writing, closing files, in particular), waiting on locks or thread coordination, (etc etc) and warn when things take unreasonably long times to return (things that should take dozens or hundreds of milliseconds and take more than 5 or 10 seconds or some such).  This could help diagnose and fix problems like this.

It would also be very helpful if (for diagnostic purposes, at least) a tasks/threads window could be opened to see what lengthy tasks are concurrently running... filters, compacting folders, downloading mails, moving messages, and so forth.... especially when certain operations are pending, waiting for other operations to finish.
thanks for that great update.  Gordon, did you also see "(not responding)" in the title bar of the thunderbird window?


> I hope that by so doing I'm not preventing MSE from finding any malicious software perhaps arriving in e-mails. 

essentially no, depending on your POV. see 
http://kb.mozillazine.org/Email_scanning_-_pros_and_cons
http://kb.mozillazine.org/Thunderbird_:_FAQs_:_Anti-virus_Software
https://wiki.mozilla.org/Thunderbird:Testing:Antivirus_Related_Performance_Issues#Problem_Antivirus_Packages
Status: UNCONFIRMED → RESOLVED
Closed: 13 years ago
Resolution: --- → INVALID
Summary: Very nonresponsive (hourglass) displaying any screens (many minutes response) → Very slow, nonresponsive (hourglass) displaying any screens (many minutes response) caused by Microsoft Security Essentials (MSE)
Whiteboard: [support]
You need to log in before you can comment on or make changes to this bug.