817335 - Slow UI response time. High CPU. Horrible jank in NtClose (ntdll.dll). High main thread IO

Status: RESOLVED INVALID

(Thunderbird :: General, defect)

Description

[Gregory Szorc [:gps]]

I am seeing horrible jank in Earlybird. Steps to reproduce are essentially "Use Earlybird."

Profile: http://people.mozilla.com/~bgirard/cleopatra/?1354400502787#report=468bd94527331340bf7f923a6c87abc2cc9cff0a

I'd had this problem for weeks and just got around to capturing a profile. I believe Earlybird is downloading a number of messages in the background while this is happening.

Comment 1

[Gregory Szorc [:gps]]

I'm not having the problem in Thunderbird 17. I /think/ it's contained in the profile, but I was using Earlybird 2012-11-27. The problem reproduced as soon as I restarted the application. It made Earlybird unusable for me.

Comment 2

[Gregory Szorc [:gps]]

Now I'm seeing this in Thunderbird 17:

https://people.mozilla.com/~bgirard/cleopatra/?1354422681571#report=edd4003b148a3f373b08667024f90fa531364828

I've used Thunderbird in Safe Mode (read: add-ons disabled) and the problem persists. While this was occurring, I noticed Thunderbird appears to be bulk fetching the Gmail "All Email" folder via IMAP. I've seen it do this periodically. Not sure why it refetching it all the time.

This jank makes me want to ditch Thunderbird. Constant and 870ms average responsiveness. Yuck.

Comment 3

[Gregory Szorc [:gps]]

A bugzilla search for NtClose reveals bugs 710705, 711086, 717386, 726119, 748856. I'm not sure what the C++ function is for this jank because the profiler isn't showing the symbols. Perhaps I need to download a special build of Earlybird?

Anyway, CC'ing Taras out of jank principle.

Comment 4

[Gregory Szorc [:gps]]

I removed and re-added my gmail account and the issue seems to have gone away. As part of re-adding this account, I deselected some rather large folders (like All Mail) from the download list.

I also noticed Microsoft Security Essentials had recently automatically cleaned a threat out of a mailbox file in my profile. Perhaps it changed the file in such a way to cause Thunderbird to flip out?

Comment 5

[Wayne Mery (:wsmwk)]

(In reply to Gregory Szorc [:gps] from comment #4)
> I also noticed Microsoft Security Essentials had recently automatically
> cleaned a threat out of a mailbox file in my profile. Perhaps it changed the
> file in such a way to cause Thunderbird to flip out?

certainly could be. anecdotaly, I recently helped a user with bad delete message performance. And the problem was found to be Defender

Comment 6

[Wayne Mery (:wsmwk)]

I don't think this is caused by TBird
And I've had other user(s) report performance issue with windows defender.
Disable AV and report results. FWIW MSE sholdn't be touching your mail.

Comment 7

[Gregory Szorc [:gps]]

I'm still seeing occasional jank in Thunderbird. However, I'm not seeing symbols in the crash reporter so I have no clue if it's in NtClose or some other function. Maybe it's because I'm running the stable channel of Thunderbird or an old version of the profiler?

Comment 8

[Mike Conley (:mconley) (:⚙️)]

You'll want to use the add-on as it exists in Github:

https://github.com/bgirard/Gecko-Profiler-Addon/raw/master/geckoprofiler.xpi

This is because AMO is being a bit slow getting the newer revision of the add-on available. I'm not even sure BenWa is still making much of an effort to get the recent builds of the add-on up there anymore.

Anyhow, the GitHub copy of the add-on should give you symbols.

-Mike

Comment 9

[Gregory Szorc [:gps]]

I just upgraded to Earlybird and saw jank in NtClose a few seconds after startup. http://people.mozilla.com/~bgirard/cleopatra/?1367347965091#report=2438f971d3e7980248f3c49f6da6c19572786b49

I have since disabled MSE real-time protection and will see if that does anything.

Comment 10

[Wayne Mery (:wsmwk)]

(In reply to Gregory Szorc [:gps] from comment #0)
> I am seeing horrible jank in Earlybird.

Can you describe please, how is your interaction with Thunderbird impacted?


> I'd had this problem for weeks and just got around to capturing a profile. 

But it wasn't happening at some point in time X in the past, with the same setup?


> believe Earlybird is downloading a number of messages in the background
> while this is happening.
> I have since disabled MSE real-time protection and will see if that does
> anything.

Interesting that you mention this. Because MSE generally is seen as quite stable and benign working with Thunderbird. In fact it's a top recommendation for AV software. However, there are issues cited at https://wiki.mozilla.org/Thunderbird:Testing:Antivirus_Related_Performance_Issues#Problem_Antivirus_and_Anti-Malware_Packages .. and missing is a citation where I recently helped someone and to my surprise we identified MSE. (Fx must have crashed before I could save wiki)

With MSE enabled do you see CPU usage for msmpeng.exe. And with it disabled, or tweaked for Thunderbird, do you have good results?

ref: http://answers.microsoft.com/en-us/protect/forum/mse-protect_scanning/msmpengexe-thread-2548-ntdlldll-uses-25-of-quad/57e9690c-5208-4396-a6c4-0fca46ac9ac7

Comment 11

[Wayne Mery (:wsmwk)]

I see this today. No significant amount of download activity at the time, just a few messages. memory usage nominal. 

http://people.mozilla.com/~bgirard/cleopatra/?1375823268636#report=5b04fd0a63e0c37b9417e213c5aa40079fe18a87

Comment 12

[Gregory Szorc [:gps]]

Adding a profile directory exclusion to MSE makes the jank go away.

It's sad I have to do this, as MSE has discovered malware emailed to me in the past. I guess now I'll trust Thunderbird won't execute attachments without my knowledge and I'll let MSE find them if they ever escape my profile.

Comment 13

[Wayne Mery (:wsmwk)]

welcome to the world of AV induced problems - https://bugzilla.mozilla.org/buglist.cgi?v4=antivirus%20AV&f1=short_desc&o3=anywords&list_id=9966625&v3=antivirus%20AV&j2=OR&o1=nowordssubstr&classification=Client%20Software&classification=Components&f4=status_whiteboard&query_format=advanced&f3=short_desc&o4=anywords&f2=OP&product=MailNews%20Core&product=Thunderbird

(In reply to Gregory Szorc [:gps] (not actively checking bugmail) from comment #12)
> Adding a profile directory exclusion to MSE makes the jank go away.

No AV software should be scanning the thunderbird profile directory. The files of the profile are large compared to an individual message. And the files are *frequently* accessed and updated.  Scanning them is not a performance option.  I too have had to exclude my Thunderbird profile.

- https://wiki.mozilla.org/Thunderbird:Testing:Antivirus_Related_Performance_Issues
- http://kb.mozillazine.org/Thunderbird_:_FAQs_:_Anti-virus_Software (somewhat outdated)

> It's sad I have to do this, as MSE has discovered malware emailed to me in
> the past. I guess now I'll trust Thunderbird won't execute attachments
> without my knowledge 

TB will not

> and I'll let MSE find them if they ever escape my profile.

If the AV is functioning properly, it is sufficient for the AV to scan attachments when they are opened or saved from a message.