1304479 - long delayed UI response once per session changing/viewing messages (win10 on SSD) if "Allow Windows Search to search messages" is enabled

Status: RESOLVED DUPLICATE of bug 1249056

(Thunderbird :: General, defect)

Description

[Adam Kessel]

User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 Safari/537.36

Steps to reproduce:

In nearly every Thunderbird session, the client locks up for anywhere from 5 to 60 seconds. I have been experiencing this problem for many years, and it still exists in today's daily build. The UI freeze appears to happen when moving from one email to the next. Typically, but not always, if it has occurred once in a given usage session, it doesn't occur again.

I have seen several other bugs and forum/support discussions about this or a similar problem, but I'm not sure if I have the same issue so I'm opening it as a new bug.

I have tried starting in safe mode. Anecdotally, this seems to reduce but definitely does not eliminate the problem.

I have seen threads suggesting disabling antivirus software (I have ESET 6.2.2) or Windows Firewall; however, on a corporate device, I don't have the ability to disable these features, and all other software works fine with them so I think it's something Thunderbird has to fix. I've also seen this same issue on Mac and Linux (although I am not using either platform regularly now for email) so that again suggests it's not really an AV issue.

I'm currently running fully patched Windows 10 on latest gen fully-loaded X1 Carbon, but have experienced this issue continuously at least since Windows 7.

I've got a large IMAP mail base. Syncing both to a private IMAP server as well as gmail.

Happy to help troubleshoot/debug. This problem has existed for long enough, and is apparently widespread enough, that it really ought to be fixed.

Comment 1

[Wayne Mery (:wsmwk)]

(In reply to Adam Kessel from comment #0)
> User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML,
> like Gecko) Chrome/53.0.2785.116 Safari/537.36
> 
> Steps to reproduce:
> 
> In nearly every Thunderbird session, the client locks up for anywhere from 5
> to 60 seconds. I have been experiencing this problem for many years, and it
> still exists in today's daily build. The UI freeze appears to happen when
> moving from one email to the next. Typically, but not always, if it has
> occurred once in a given usage session, it doesn't occur again.

So the issue is not moving messages, but viewing them?

> I have seen several other bugs and forum/support discussions about this or a
> similar problem, but I'm not sure if I have the same issue so I'm opening it
> as a new bug.

that should be a hint that you are probably not seeing something new :)

> I have tried starting in safe mode. Anecdotally, this seems to reduce but
> definitely does not eliminate the problem.

What addons do you have installed?
 
> I have seen threads suggesting disabling antivirus software (I have ESET
> 6.2.2) or Windows Firewall; however, on a corporate device, I don't have the
> ability to disable these features, and all other software works fine with
> them so I think it's something Thunderbird has to fix. 

> I've also seen this
> same issue on Mac and Linux (although I am not using either platform
> regularly now for email) so that again suggests it's not really an AV issue.

Were the Mac and Linux Thunderbird profiles (accounts, settings etc) the same?

(fair warning, the user population is not homogenous. So one can't assume there is one problem across all users.)
 
> I'm currently running fully patched Windows 10 on latest gen fully-loaded X1
> Carbon, but have experienced this issue continuously at least since Windows
> 7.
> 
> I've got a large IMAP mail base. Syncing both to a private IMAP server as
> well as gmail.

define large in terms of numbers please.
 
> Happy to help troubleshoot/debug. This problem has existed for long enough,
> and is apparently widespread enough, that it really ought to be fixed.

https://wiki.mozilla.org/Thunderbird:Testing:Memory_Usage_Problems is your starting point for gathering information about your environment.

Comment 2

[Adam Kessel]

Some additional troubleshooting info on top of responses I just provided by email:

- I have rebuilt %appdata% from scratch several times without noticing an impact on this problem
- global-messages-db.sqlite file is about 1.2GB
- panacea.dat is 60KB
- Task manager shows RAM usage about 160MB
- no proxies
- everything stored locally on an SSD
- no newsgroups
- gmail not holding much mail at all

Comment 3

[Adam Kessel]

Oops, sorry. I see now I can't respond by email. Here is what I added:

On 9/21/2016 3:15 PM, Bugzilla@Mozilla wrote:
>> User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML,
>> like Gecko) Chrome/53.0.2785.116 Safari/537.36
>>
>> Steps to reproduce:
>>
>> In nearly every Thunderbird session, the client locks up for anywhere from 5
>> to 60 seconds. I have been experiencing this problem for many years, and it
>> still exists in today's daily build. The UI freeze appears to happen when
>> moving from one email to the next. Typically, but not always, if it has
>> occurred once in a given usage session, it doesn't occur again.
> So the issue is not moving messages, but viewing them?
As best I can tell, the answer is yes. It can happen when I'm just scrolling from one message to the next, and not moving anything. It also happens when I archive a message, but it looks like the message archives fine, and then it freezes before it's able to render the next message.
>> I have seen several other bugs and forum/support discussions about this or a
>> similar problem, but I'm not sure if I have the same issue so I'm opening it
>> as a new bug.
> that should be a hint that you are probably not seeing something new
Understood, although I've taken flack before for adding on to an existing bug before it has been determined that it really is the same bug.
>> I have tried starting in safe mode. Anecdotally, this seems to reduce but
>> definitely does not eliminate the problem.
>>
>> What addons do you have installed?
Compact Menu 2
CompactHeader
Search Results By Date Not Relevance
Toggle Headers

But, noting again I get the behavior at least sometimes in safe mode as well.
>> I have seen threads suggesting disabling antivirus software (I have ESET
>> 6.2.2) or Windows Firewall; however, on a corporate device, I don't have the
>> ability to disable these features, and all other software works fine with
>> them so I think it's something Thunderbird has to fix.
>> I've also seen this
>> same issue on Mac and Linux (although I am not using either platform
>> regularly now for email) so that again suggests it's not really an AV issue.
> Were the Mac and Linux Thunderbird profiles (accounts, settings etc) the same?
They were the same mail servers and mail server settings, and same plugins.
> (fair warning, the user population is not homogenous. So one can't assume there
> is one problem across all users.)
>
>> I'm currently running fully patched Windows 10 on latest gen fully-loaded X1
>> Carbon, but have experienced this issue continuously at least since Windows
>> 7.
>>
>> I've got a large IMAP mail base. Syncing both to a private IMAP server as
>> well as gmail.
> define large in terms of numbers please.
My IMAP account is about 13 GB, or 336,000 messages. I don't think it matters, but the messages are in maildir format on the server-side for my IMAP server.
>> Happy to help troubleshoot/debug. This problem has existed for long enough,
>> and is apparently widespread enough, that it really ought to be fixed.
> https://wiki.mozilla.org/Thunderbird:Testing:Memory_Usage_Problems is your
> starting point for gathering information about your environment.
>
>
I'll take a look at that.

I just got this Visual Studio debug capture during a freeze. I'm not sure if this helps or it just shows that the display is stuck while something else is going on under the hood:
     [External Code]
>    xul.dll!D3DVsyncSource::D3DVsyncDisplay::VBlankLoop() Line 1883    C++
     xul.dll!mozilla::detail::RunnableMethodImpl<enum nsresult (__thiscall nsUrlClassifierDBServiceWorker::*)(void),1,0>::Run() Line 766    C++
xul.dll!MessageLoop::RunTask(already_AddRefed<mozilla::Runnable> aTask) Line 347    C++
xul.dll!MessageLoop::DeferOrRunPendingTask(MessageLoop::PendingTask && pending_task) Line 357    C++
     xul.dll!MessageLoop::DoWork() Line 429    C++
xul.dll!base::MessagePumpDefault::Run(base::MessagePump::Delegate * delegate) Line 37    C++
     xul.dll!MessageLoop::RunHandler() Line 226    C++
     xul.dll!MessageLoop::Run() Line 206    C++
     xul.dll!base::Thread::ThreadMain() Line 183    C++
     xul.dll!ThreadEntry(void * arg) Line 256    C++
     [External Code]

        this    Error reading register value.
-        flushTime    {mValue={mGTC=21221850846284 mQPC=21222272943000 mHasQPC=true ...} }    mozilla::TimeStamp
-        mValue    {mGTC=21221850846284 mQPC=21222272943000 mHasQPC=true ...}    mozilla::TimeStampValue
        mGTC    21221850846284    unsigned __int64
        mQPC    21222272943000    unsigned __int64
        mHasQPC    true    bool
        mIsNull    false    bool
-        longVBlank    {mValue=84524468 } mozilla::BaseTimeDuration<mozilla::TimeDurationValueCalculator>
        mValue    84524468    __int64
-        vsync    {mValue={mGTC=21221841631308 mQPC=21222272418058 mHasQPC=true ...} }    mozilla::TimeStamp
-        mValue    {mGTC=21221841631308 mQPC=21222272418058 mHasQPC=true ...}    mozilla::TimeStampValue
        mGTC    21221841631308    unsigned __int64
        mQPC    21222272418058    unsigned __int64
        mHasQPC    true    bool
        mIsNull    false    bool
        hr    S_OK    HRESULT
-        flushDiff    {mValue=42263000 } mozilla::BaseTimeDuration<mozilla::TimeDurationValueCalculator>
        mValue    42263000    __int64
-        now    {mValue={mGTC=21221850846284 mQPC=21222272943000 mHasQPC=true ...} }    mozilla::TimeStamp
-        mValue    {mGTC=21221850846284 mQPC=21222272943000 mHasQPC=true ...}    mozilla::TimeStampValue
        mGTC    21221850846284    unsigned __int64
        mQPC    21222272943000    unsigned __int64
        mHasQPC    true    bool
        mIsNull    false    bool

Comment 4

[Wayne Mery (:wsmwk)]

visit http://www.eicar.org/86-0-Intended-use.html Using a text editor, create your own fake virus file in the thunderbird profile.  If it gets flagged then eset is scanning your thunderbird profile

> - global-messages-db.sqlite file is about 1.2GB
That's probably OK. 
(mine was about 1.3 before I blew it away so I could remove some folders from sesarching. now it's 450mb)

> - Task manager shows RAM usage about 160MB
160MB just for Thunderbird? That sounds too low unless you've only got one or two small acounts.

> My IMAP account is about 13 GB, or 336,000 messages. 
So that's one account? 
How big is the Inbox, Drafts, and Spam (or Junk)?  (look in thunderbird folder properties)

Comment 5

[Adam Kessel]

> visit http://www.eicar.org/86-0-Intended-use.html Using a text editor, create your own fake virus file in the thunderbird profile.  If it gets flagged then eset is scanning your thunderbird profile

I'm sure eSet is scanning, since it has triggered on and quarantined malware emails before. It may be at the end of the day that Thunderbird just doesn't work with eSet, but that would be unfortunate since it would exclude corporate users who can't make exceptions. I'd hope there would be some workaround. I'm also not convinced the behavior is pointing in that direction, though. Why would it usually just freeze up once per session, and then as long as I keep the client open, not freeze up again?

> > - Task manager shows RAM usage about 160MB
> 160MB just for Thunderbird? That sounds too low unless you've only got one or two small acounts.

That is what I was seeing when I checked earlier. Now it's more up around 350MB although it bounces around a lot--down to 340MB and up to 370MB. (FWIW, laptop has 8GB RAM, and problem occurs when nothing else is running.)

> > My IMAP account is about 13 GB, or 336,000 messages. 
> So that's one account? 
> How big is the Inbox, Drafts, and Spam (or Junk)?  (look in thunderbird folder properties)

I'm an inbox zero guy. So inbox is nearly always < 10 messages.

Drafts is similar.

Spam/Junk is regularly cleaned out by a cron job, so at most it's a few hundred emails.

My archive folders are sorted by year, going back about 12 years. A typical archive folder is about ~20,000 messages and 1.5 GB.

Comment 6

[Wayne Mery (:wsmwk)]

(In reply to Adam Kessel from comment #5)
> > visit http://www.eicar.org/86-0-Intended-use.html Using a text editor, create your own fake virus file in the thunderbird profile.  If it gets flagged then eset is scanning your thunderbird profile
> 
> I'm sure eSet is scanning, since it has triggered on and quarantined malware
> emails before. It may be at the end of the day that Thunderbird just doesn't
> work with eSet, but that would be unfortunate since it would exclude
> corporate users who can't make exceptions. 

Part of the job of AV software is to play nice with others. If it isn't set to exclude TB automatically then it must be done manually. Therefore, the details I provided above so we know precisely from your doing the test what is happening.

> I'd hope there would be some
> workaround. I'm also not convinced the behavior is pointing in that
> direction, though. Why would it usually just freeze up once per session, and
> then as long as I keep the client open, not freeze up again?

Sorry, don't have an answer to that question.
 
> > > - Task manager shows RAM usage about 160MB
> > 160MB just for Thunderbird? That sounds too low unless you've only got one or two small acounts.
> 
> That is what I was seeing when I checked earlier. Now it's more up around
> 350MB although it bounces around a lot--down to 340MB and up to 370MB.
> (FWIW, laptop has 8GB RAM, and problem occurs when nothing else is running.)

memory size changing is OK. And 400-500MB is not uncommon. 

When TB gets slow, that's precisely when to check what memory and cpu thunderbird is using, and also check what the rest of the system is using

> > > My IMAP account is about 13 GB, or 336,000 messages. 
> > So that's one account? 
> > How big is the Inbox, Drafts, and Spam (or Junk)?  (look in thunderbird folder properties)
> 
> I'm an inbox zero guy. So inbox is nearly always < 10 messages.
> 
> Drafts is similar.
> 
> Spam/Junk is regularly cleaned out by a cron job, so at most it's a few
> hundred emails.

cool. But that doesn't tell us the size on local disk - which is what impacts thunderbird performance. (should be the same size as server, but who knows)

Comment 7

[Adam Kessel]

> Part of the job of AV software is to play nice with others. If it isn't set to exclude TB automatically then it must be done manually. Therefore, the details I provided above so we know precisely from your doing the test what is happening.

OK, I will experiment and report back.

> cool. But that doesn't tell us the size on local disk - which is what impacts thunderbird performance. (should be the same size as server, but who knows)

%appdata%/thunderbird is ~344,000 files and about 36.5GB on disk.

Comment 8

[Adam Kessel]

By the way, was my Visual Studio breakpoint information above useful at all?

Comment 9

[Adam Kessel]

Here's another call stack from another "lock" incident. I did notice the RAM and CPU usage were very low at the time, that seems to have no correlation with the lock-ups.

	xul.dll!nsLocalFile::CopySingleFile(nsIFile * aSourceFile, nsIFile * aDestParent, const nsAString_internal & aNewName, unsigned int aOptions) Line 1871	C++
 	xul.dll!nsLocalFile::CopyMove(nsIFile * aParentDir, const nsAString_internal & aNewName, unsigned int aOptions) Line 1987	C++
 	xul.dll!nsLocalFile::MoveTo(nsIFile * aNewParentDir, const nsAString_internal & aNewName) Line 2163	C++
 	xul.dll!_NS_InvokeByIndex() Line 57	Unknown
 	xul.dll!XPCWrappedNative::CallMethod(XPCCallContext & ccx, XPCWrappedNative::CallMode mode) Line 1349	C++
 	xul.dll!XPC_WN_CallMethod(JSContext * cx, unsigned int argc, JS::Value * vp) Line 1143	C++
 	xul.dll!js::InternalCallOrConstruct(JSContext * cx, const JS::CallArgs & args, js::MaybeConstruct construct) Line 458	C++
 	xul.dll!InternalCall(JSContext * cx, const js::AnyInvokeArgs & args) Line 503	C++
 	xul.dll!Interpret(JSContext * cx, js::RunState & state) Line 2922	C++
 	xul.dll!js::RunScript(JSContext * cx, js::RunState & state) Line 404	C++
 	xul.dll!js::InternalCallOrConstruct(JSContext * cx, const JS::CallArgs & args, js::MaybeConstruct construct) Line 479	C++
 	xul.dll!InternalCall(JSContext * cx, const js::AnyInvokeArgs & args) Line 503	C++
 	xul.dll!js::Call(JSContext * cx, JS::Handle<JS::Value> fval, JS::Handle<JS::Value> thisv, const js::AnyInvokeArgs & args, JS::MutableHandle<JS::Value> rval) Line 522	C++
 	xul.dll!JS_CallFunctionValue(JSContext * cx, JS::Handle<JSObject *> obj, JS::Handle<JS::Value> fval, const JS::HandleValueArray & args, JS::MutableHandle<JS::Value> rval) Line 2777	C++
 	xul.dll!nsXPCWrappedJSClass::CallMethod(nsXPCWrappedJS * wrapper, unsigned short methodIndex, const XPTMethodDescriptor * info_, nsXPTCMiniVariant * nativeParams) Line 1212	C++
 	xul.dll!nsXPCWrappedJS::CallMethod(unsigned short methodIndex, const XPTMethodDescriptor * info, nsXPTCMiniVariant * params) Line 614	C++
 	xul.dll!PrepareAndDispatch(nsXPTCStubBase * self, unsigned int methodIndex, unsigned int * args, unsigned int * stackBytesToPop) Line 85	C++
 	xul.dll!SharedStub() Line 113	C++
 	xul.dll!nsMsgFolderNotificationService::NotifyMsgsMoveCopyCompleted(bool aMove, nsIArray * aSrcMsgs, nsIMsgFolder * aDestFolder, nsIArray * aDestMsgs) Line 125	C++
 	xul.dll!nsImapMailFolder::CopyMessagesOffline(nsIMsgFolder * srcFolder, nsIArray * messages, bool isMove, nsIMsgWindow * msgWindow, nsIMsgCopyServiceListener * listener) Line 7423	C++
 	xul.dll!nsImapMailFolder::CopyMessages(nsIMsgFolder * srcFolder, nsIArray * messages, bool isMove, nsIMsgWindow * msgWindow, nsIMsgCopyServiceListener * listener, bool isFolder, bool allowUndo) Line 7597	C++
 	xul.dll!nsMsgCopyService::DoNextCopy() Line 317	C++
 	xul.dll!nsMsgCopyService::DoCopy(nsCopyRequest * aRequest) Line 261	C++
 	xul.dll!nsMsgCopyService::CopyMessages(nsIMsgFolder * srcFolder, nsIArray * messages, nsIMsgFolder * dstFolder, bool isMove, nsIMsgCopyServiceListener * listener, nsIMsgWindow * window, bool allowUndo) Line 544	C++
 	xul.dll!_NS_InvokeByIndex() Line 57	Unknown
 	xul.dll!XPCWrappedNative::CallMethod(XPCCallContext & ccx, XPCWrappedNative::CallMode mode) Line 1349	C++
 	xul.dll!XPC_WN_CallMethod(JSContext * cx, unsigned int argc, JS::Value * vp) Line 1143	C++
 	xul.dll!js::InternalCallOrConstruct(JSContext * cx, const JS::CallArgs & args, js::MaybeConstruct construct) Line 458	C++
 	xul.dll!InternalCall(JSContext * cx, const js::AnyInvokeArgs & args) Line 503	C++
 	xul.dll!Interpret(JSContext * cx, js::RunState & state) Line 2922	C++
 	xul.dll!js::RunScript(JSContext * cx, js::RunState & state) Line 404	C++
 	xul.dll!js::InternalCallOrConstruct(JSContext * cx, const JS::CallArgs & args, js::MaybeConstruct construct) Line 479	C++
 	xul.dll!InternalCall(JSContext * cx, const js::AnyInvokeArgs & args) Line 503	C++
 	xul.dll!js::Call(JSContext * cx, JS::Handle<JS::Value> fval, JS::Handle<JS::Value> thisv, const js::AnyInvokeArgs & args, JS::MutableHandle<JS::Value> rval) Line 522	C++
 	xul.dll!JS_CallFunctionValue(JSContext * cx, JS::Handle<JSObject *> obj, JS::Handle<JS::Value> fval, const JS::HandleValueArray & args, JS::MutableHandle<JS::Value> rval) Line 2777	C++
 	xul.dll!nsXPCWrappedJSClass::CallMethod(nsXPCWrappedJS * wrapper, unsigned short methodIndex, const XPTMethodDescriptor * info_, nsXPTCMiniVariant * nativeParams) Line 1212	C++
 	xul.dll!nsXPCWrappedJS::CallMethod(unsigned short methodIndex, const XPTMethodDescriptor * info, nsXPTCMiniVariant * params) Line 614	C++
 	xul.dll!PrepareAndDispatch(nsXPTCStubBase * self, unsigned int methodIndex, unsigned int * args, unsigned int * stackBytesToPop) Line 85	C++
 	xul.dll!SharedStub() Line 113	C++
 	xul.dll!nsMsgFilterAfterTheFact::OnEndExecution() Line 366	C++
 	xul.dll!nsMsgFilterAfterTheFact::AdvanceToNextFolder() Line 432	C++
 	xul.dll!nsMsgApplyFiltersToMessages::RunNextFilter() Line 1094	C++
 	xul.dll!nsMsgFilterAfterTheFact::AdvanceToNextFolder() Line 460	C++
 	xul.dll!nsMsgFilterService::ApplyFilters(int aFilterType, nsIArray * aMsgHdrList, nsIMsgFolder * aFolder, nsIMsgWindow * aMsgWindow, nsIMsgOperationListener * aCallback) Line 1121	C++
 	xul.dll!_NS_InvokeByIndex() Line 57	Unknown
 	xul.dll!XPCWrappedNative::CallMethod(XPCCallContext & ccx, XPCWrappedNative::CallMode mode) Line 1349	C++
 	xul.dll!XPC_WN_CallMethod(JSContext * cx, unsigned int argc, JS::Value * vp) Line 1143	C++
 	xul.dll!js::InternalCallOrConstruct(JSContext * cx, const JS::CallArgs & args, js::MaybeConstruct construct) Line 458	C++
 	xul.dll!InternalCall(JSContext * cx, const js::AnyInvokeArgs & args) Line 503	C++
 	xul.dll!Interpret(JSContext * cx, js::RunState & state) Line 2922	C++
 	xul.dll!js::RunScript(JSContext * cx, js::RunState & state) Line 404	C++
 	xul.dll!js::InternalCallOrConstruct(JSContext * cx, const JS::CallArgs & args, js::MaybeConstruct construct) Line 479	C++
 	xul.dll!InternalCall(JSContext * cx, const js::AnyInvokeArgs & args) Line 503	C++
 	xul.dll!js::Call(JSContext * cx, JS::Handle<JS::Value> fval, JS::Handle<JS::Value> thisv, const js::AnyInvokeArgs & args, JS::MutableHandle<JS::Value> rval) Line 522	C++
 	xul.dll!js::fun_call(JSContext * cx, unsigned int argc, JS::Value * vp) Line 1249	C++
 	xul.dll!js::InternalCallOrConstruct(JSContext * cx, const JS::CallArgs & args, js::MaybeConstruct construct) Line 458	C++
 	xul.dll!InternalCall(JSContext * cx, const js::AnyInvokeArgs & args) Line 503	C++
 	xul.dll!Interpret(JSContext * cx, js::RunState & state) Line 2922	C++
 	xul.dll!js::RunScript(JSContext * cx, js::RunState & state) Line 404	C++
 	xul.dll!js::InternalCallOrConstruct(JSContext * cx, const JS::CallArgs & args, js::MaybeConstruct construct) Line 479	C++
 	xul.dll!InternalCall(JSContext * cx, const js::AnyInvokeArgs & args) Line 503	C++
 	xul.dll!js::Call(JSContext * cx, JS::Handle<JS::Value> fval, JS::Handle<JS::Value> thisv, const js::AnyInvokeArgs & args, JS::MutableHandle<JS::Value> rval) Line 522	C++
 	xul.dll!JS::Call(JSContext * cx, JS::Handle<JS::Value> thisv, JS::Handle<JS::Value> fval, const JS::HandleValueArray & args, JS::MutableHandle<JS::Value> rval) Line 2836	C++
 	xul.dll!mozilla::dom::EventHandlerNonNull::Call(JSContext * cx, JS::Handle<JS::Value> aThisVal, mozilla::dom::Event & event, JS::MutableHandle<JS::Value> aRetVal, mozilla::ErrorResult & aRv) Line 259	C++
 	xul.dll!mozilla::dom::EventHandlerNonNull::Call<nsISupports *>(nsISupports * const & thisVal, mozilla::dom::Event & event, JS::MutableHandle<JS::Value> aRetVal, mozilla::ErrorResult & aRv, const char * aExecutionReason, mozilla::dom::CallbackObject::ExceptionHandling aExceptionHandling, JSCompartment * aCompartment) Line 361	C++
 	xul.dll!mozilla::JSEventHandler::HandleEvent(nsIDOMEvent * aEvent) Line 215	C++
 	xul.dll!mozilla::EventListenerManager::HandleEventSubType(mozilla::EventListenerManager::Listener * aListener, nsIDOMEvent * aDOMEvent, mozilla::dom::EventTarget * aCurrentTarget) Line 1133	C++
 	xul.dll!mozilla::EventListenerManager::HandleEventInternal(nsPresContext * aPresContext, mozilla::WidgetEvent * aEvent, nsIDOMEvent * * aDOMEvent, mozilla::dom::EventTarget * aCurrentTarget, nsEventStatus * aEventStatus) Line 1286	C++
 	xul.dll!mozilla::EventListenerManager::HandleEvent(nsPresContext * aPresContext, mozilla::WidgetEvent * aEvent, nsIDOMEvent * * aDOMEvent, mozilla::dom::EventTarget * aCurrentTarget, nsEventStatus * aEventStatus) Line 374	C++
 	xul.dll!mozilla::EventTargetChainItem::HandleEvent(mozilla::EventChainPostVisitor & aVisitor, mozilla::ELMCreationDetector & aCd) Line 275	C++
 	xul.dll!mozilla::EventTargetChainItem::HandleEventTargetChain(nsTArray<mozilla::EventTargetChainItem> & aChain, mozilla::EventChainPostVisitor & aVisitor, mozilla::EventDispatchingCallback * aCallback, mozilla::ELMCreationDetector & aCd) Line 382	C++
 	xul.dll!mozilla::EventDispatcher::Dispatch(nsISupports * aTarget, nsPresContext * aPresContext, mozilla::WidgetEvent * aEvent, nsIDOMEvent * aDOMEvent, nsEventStatus * aEventStatus, mozilla::EventDispatchingCallback * aCallback, nsTArray<mozilla::dom::EventTarget *> * aTargets) Line 716	C++
 	xul.dll!mozilla::EventDispatcher::DispatchDOMEvent(nsISupports * aTarget, mozilla::WidgetEvent * aEvent, nsIDOMEvent * aDOMEvent, nsPresContext * aPresContext, nsEventStatus * aEventStatus) Line 777	C++
 	xul.dll!nsINode::DispatchEvent(nsIDOMEvent * aEvent, bool * aRetVal) Line 1306	C++
 	xul.dll!nsContentUtils::DispatchXULCommand(nsIContent * aTarget, bool aTrusted, nsIDOMEvent * aSourceEvent, nsIPresShell * aShell, bool aCtrl, bool aAlt, bool aShift, bool aMeta) Line 6163	C++
 	xul.dll!nsXBLPrototypeHandler::DispatchXULKeyCommand(nsIDOMEvent * aEvent) Line 536	C++
 	xul.dll!nsXBLPrototypeHandler::ExecuteHandler(mozilla::dom::EventTarget * aTarget, nsIDOMEvent * aEvent) Line 225	C++
 	xul.dll!nsXBLWindowKeyHandler::WalkHandlersAndExecute(nsIDOMKeyEvent * aKeyEvent, nsIAtom * aEventType, nsXBLPrototypeHandler * aFirstHandler, unsigned int aCharCode, const mozilla::dom::IgnoreModifierState & aIgnoreModifierState, bool aExecute, bool * aOutReservedForChrome) Line 757	C++
 	xul.dll!nsXBLWindowKeyHandler::WalkHandlersInternal(nsIDOMKeyEvent * aKeyEvent, nsIAtom * aEventType, nsXBLPrototypeHandler * aHandler, bool aExecute, bool * aOutReservedForChrome) Line 625	C++
 	xul.dll!nsXBLWindowKeyHandler::WalkHandlers(nsIDOMKeyEvent * aKeyEvent, nsIAtom * aEventType) Line 298	C++
 	xul.dll!nsXBLWindowKeyHandler::HandleEvent(nsIDOMEvent * aEvent) Line 475	C++
 	xul.dll!mozilla::EventListenerManager::HandleEventSubType(mozilla::EventListenerManager::Listener * aListener, nsIDOMEvent * aDOMEvent, mozilla::dom::EventTarget * aCurrentTarget) Line 1133	C++
 	xul.dll!mozilla::EventListenerManager::HandleEventInternal(nsPresContext * aPresContext, mozilla::WidgetEvent * aEvent, nsIDOMEvent * * aDOMEvent, mozilla::dom::EventTarget * aCurrentTarget, nsEventStatus * aEventStatus) Line 1286	C++
 	xul.dll!mozilla::EventListenerManager::HandleEvent(nsPresContext * aPresContext, mozilla::WidgetEvent * aEvent, nsIDOMEvent * * aDOMEvent, mozilla::dom::EventTarget * aCurrentTarget, nsEventStatus * aEventStatus) Line 374	C++
 	xul.dll!mozilla::EventTargetChainItem::HandleEvent(mozilla::EventChainPostVisitor & aVisitor, mozilla::ELMCreationDetector & aCd) Line 275	C++
 	xul.dll!mozilla::EventTargetChainItem::HandleEventTargetChain(nsTArray<mozilla::EventTargetChainItem> & aChain, mozilla::EventChainPostVisitor & aVisitor, mozilla::EventDispatchingCallback * aCallback, mozilla::ELMCreationDetector & aCd) Line 403	C++
 	xul.dll!mozilla::EventTargetChainItem::HandleEventTargetChain(nsTArray<mozilla::EventTargetChainItem> & aChain, mozilla::EventChainPostVisitor & aVisitor, mozilla::EventDispatchingCallback * aCallback, mozilla::ELMCreationDetector & aCd) Line 433	C++
 	xul.dll!mozilla::EventDispatcher::Dispatch(nsISupports * aTarget, nsPresContext * aPresContext, mozilla::WidgetEvent * aEvent, nsIDOMEvent * aDOMEvent, nsEventStatus * aEventStatus, mozilla::EventDispatchingCallback * aCallback, nsTArray<mozilla::dom::EventTarget *> * aTargets) Line 716	C++
 	xul.dll!PresShell::ForwardKeyToInputMethodAppOrDispatch(bool aIsTargetRemote, nsINode * aTarget, mozilla::WidgetKeyboardEvent & aEvent, nsEventStatus * aStatus, mozilla::EventDispatchingCallback * aEventCB) Line 7222	C++
 	xul.dll!PresShell::HandleKeyboardEvent(nsINode * aTarget, mozilla::WidgetKeyboardEvent & aEvent, bool aEmbeddedCancelled, nsEventStatus * aStatus, mozilla::EventDispatchingCallback * aEventCB) Line 7122	C++
 	xul.dll!PresShell::DispatchEventToDOM(mozilla::WidgetEvent * aEvent, nsEventStatus * aStatus, nsPresShellEventCB * aEventCB) Line 8313	C++
 	xul.dll!PresShell::HandleEventInternal(mozilla::WidgetEvent * aEvent, nsEventStatus * aStatus, bool aIsHandlingNativeEvent) Line 8190	C++
 	xul.dll!PresShell::HandleEvent(nsIFrame * aFrame, mozilla::WidgetGUIEvent * aEvent, bool aDontRetargetEvents, nsEventStatus * aEventStatus, nsIContent * * aTargetContent) Line 7897	C++
 	xul.dll!PresShell::HandleEvent(nsIFrame * aFrame, mozilla::WidgetGUIEvent * aEvent, bool aDontRetargetEvents, nsEventStatus * aEventStatus, nsIContent * * aTargetContent) Line 7410	C++
 	xul.dll!nsViewManager::DispatchEvent(mozilla::WidgetGUIEvent * aEvent, nsView * aView, nsEventStatus * aStatus) Line 816	C++
 	xul.dll!nsView::HandleEvent(mozilla::WidgetGUIEvent * aEvent, bool aUseAttachedEvents) Line 1118	C++
 	xul.dll!nsWindow::DispatchEvent(mozilla::WidgetGUIEvent * event, nsEventStatus & aStatus) Line 3880	C++
 	xul.dll!nsBaseWidget::DispatchInputEvent(mozilla::WidgetInputEvent * aEvent) Line 1240	C++
 	xul.dll!mozilla::widget::TextEventDispatcher::DispatchInputEvent(nsIWidget * aWidget, mozilla::WidgetInputEvent & aEvent, nsEventStatus & aStatus) Line 206	C++
 	xul.dll!mozilla::widget::TextEventDispatcher::DispatchKeyboardEventInternal(mozilla::EventMessage aMessage, const mozilla::WidgetKeyboardEvent & aKeyboardEvent, nsEventStatus & aStatus, void * aData, unsigned int aIndexOfKeypress) Line 534	C++
 	xul.dll!mozilla::widget::TextEventDispatcher::MaybeDispatchKeypressEvents(const mozilla::WidgetKeyboardEvent & aKeyboardEvent, nsEventStatus & aStatus, void * aData) Line 565	C++
 	xul.dll!mozilla::widget::NativeKey::HandleCharMessage(const tagMSG & aCharMsg, bool * aEventDispatched) Line 1948	C++
 	xul.dll!mozilla::widget::NativeKey::DispatchKeyPressEventForFollowingCharMessage(const tagMSG & aCharMsg) Line 2695	C++
 	xul.dll!mozilla::widget::NativeKey::HandleKeyDownMessage(bool * aEventDispatched) Line 1864	C++
 	xul.dll!nsWindow::ProcessKeyDownMessage(const tagMSG & aMsg, bool * aEventDispatched) Line 6070	C++
 	xul.dll!nsWindow::ProcessMessage(unsigned int msg, unsigned int & wParam, long & lParam, long * aRetValue) Line 5114	C++
 	xul.dll!nsWindow::WindowProcInternal(HWND__ * hWnd, unsigned int msg, unsigned int wParam, long lParam) Line 4608	C++
 	xul.dll!CallWindowProcCrashProtected(long (HWND__ *, unsigned int, unsigned int, long) * aWndProc, HWND__ * aHWnd, unsigned int aMsg, unsigned int aWParam, long aLParam) Line 35	C++
 	xul.dll!nsWindow::WindowProc(HWND__ * hWnd, unsigned int msg, unsigned int wParam, long lParam) Line 4560	C++
 	[External Code]	
 	xul.dll!nsAppShell::ProcessNextNativeEvent(bool mayWait) Line 376	C++
 	xul.dll!nsBaseAppShell::DoProcessNextNativeEvent(bool mayWait) Line 139	C++
 	xul.dll!nsBaseAppShell::OnProcessNextEvent(nsIThreadInternal * thr, bool mayWait) Line 289	C++
 	xul.dll!nsThread::ProcessNextEvent(bool aMayWait, bool * aResult) Line 1033	C++
 	xul.dll!NS_ProcessNextEvent(nsIThread * aThread, bool aMayWait) Line 290	C++
 	xul.dll!mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate * aDelegate) Line 124	C++
 	xul.dll!MessageLoop::RunHandler() Line 226	C++
 	xul.dll!MessageLoop::Run() Line 206	C++
 	xul.dll!nsBaseAppShell::Run() Line 158	C++
 	xul.dll!nsAppShell::Run() Line 264	C++
 	xul.dll!nsAppStartup::Run() Line 284	C++
 	xul.dll!XREMain::XRE_mainRun() Line 4419	C++
 	xul.dll!XREMain::XRE_main(int argc, char * * argv, const nsXREAppData * aAppData) Line 4552	C++
 	xul.dll!XRE_main(int argc, char * * argv, const nsXREAppData * aAppData, unsigned int aFlags) Line 4643	C++
 	daily.exe!do_main(int argc, char * * argv, char * * envp, nsIFile * xreDirectory) Line 245	C++
 	daily.exe!NS_internal_main(int argc, char * * argv, char * * envp) Line 378	C++
 	daily.exe!wmain(int argc, wchar_t * * argv) Line 118	C++

Locals

		this	Variable is optimized away and not available.	
+		aSourceFile	0x15805400 {mRefCnt={mValue={...} } mDirty=false mResolveDirty=false ...}	nsIFile * {nsLocalFile}
+		aDestParent	0x15805460 {mRefCnt={mValue={...} } mDirty=false mResolveDirty=false ...}	nsIFile * {nsLocalFile}
+		aNewName	{mData=0x035784a0 u"" mLength=0 mFlags=1 }	const nsAString_internal &
		aOptions	2	unsigned int
		copyOK	0	int
		dwCopyFlags	8	unsigned long
+		destPath	{mStorage=0x009e9b70 u"" }	nsAutoString
		rv	NS_OK (0)	nsresult
-		filePath	{mStorage=0x009e9adc u"" }	nsAutoString
+		nsFixedString	{mFixedCapacity=63 mFixedBuf=0x009e9adc u"" }	nsFixedString
-		mStorage	0x009e9adc u""	char16_t[64]
[removed this part because bugzilla threw up on the unicode]
-		aFileName	{mStorage=0x009e9a48 u"" }	nsAutoString
+		nsFixedString	{mFixedCapacity=63 mFixedBuf=0x009e9a48 u"" }	nsFixedString
+		mStorage	0x009e9a48 u""	char16_t[64]
		path1Remote	false	bool
		path2Remote	false	bool
		pSD	Variable is optimized away and not available.	
		pOldDACL	Variable is optimized away and not available.

Comment 10

[Adam Kessel]

Eset reports:

Time;Scanner;Object type;Object;Threat;Action;User;Information;Hash;First seen here
9/21/2016 7:18:18 PM;Real-time file system protection;file;[...]AppData\Local\Thunderbird\Profiles\co85e6jg.default\test_file.txt;Eicar test file;cleaned by deleting;[...];Event occurred on a new file created by the application: C:\Program Files (x86)\Vim\vim74\gvim.exe (70578F641F9272D65354BC0DB883298DC6F94DC7).;A6C80CE949469CC86F6C22355F4D3BB8773FC634;9/21/2016 7:18:13 PM

Does that answer the question re scanning?

Comment 11

[Wayne Mery (:wsmwk)]

Yes I think so

Comment 12

[Wayne Mery (:wsmwk)]

(In reply to Adam Kessel from comment #8)
> By the way, was my Visual Studio breakpoint information above useful at all?

no

Comment 13

[Adam Kessel]

> > By the way, was my Visual Studio breakpoint information above useful at all?
> no

Is there any kind of debugging or logging I can do on my end to better isolate the cause? All of the discussions of these sorts of problems seem to be a huge process of elimination that sometimes never resolves--can't we figure where exactly in the code it's getting stuck as a more direct clue to the cause? Mozilla codebase has always been very intimidating to me, but I'd be happy to run a debug build if such a thing exists to try to track down the bug. It's been so many years, I'd really like to help squash it.

Comment 14

[Wayne Mery (:wsmwk)]

> Is there any kind of debugging or logging I can do on my end to better isolate the cause?
No. Code analysis is a non-starter with out a performance profile. And (oddly enough) I think the only diagnostic worth doing is a performance profile - but you can't predict when it is going to happen, so getting the profile is not feasible.

Besides, given the information you have presented I give better than 50-50 odds that it's AV related. So I don't think it's worthy of further your time or mine in analysis until we can conclusively eliminate AV as a potential cause.  Ir a inspiration light bulb goes of that, "ah, it might be xxxx".

Comment 15

[Adam Kessel]

I'll spend some time on Linux again and see if I can reproduce semi-reliably. That would rule out AV.

Comment 16

[Wayne Mery (:wsmwk)]

good idea. the profile instructions are at https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Reporting_a_Thunderbird_Performance_Problem_with_G   n.b. the version requirements

Comment 17

[Adam Kessel]

Tried the profiler on Windows with Thunderbird 38 and Gecko Profiler 1.12.15 as suggested. When I enable and then "dump profile" I just get a new tab prompting to "Upload your profile here:". Nothing is loaded automatically. I also searched my disk for profile_captured.sym and found nothing. What am I doing wrong?

Comment 18

[Wayne Mery (:wsmwk)]

hmm, I'm seeing the same thing but it had worked for me recently.  I don't know what's up.
So you are not doing something wrong.

Comment 19

[Wayne Mery (:wsmwk)]

Does the problem go away if you set it to start in windows 7 compatibility?

Comment 20

[Adam Kessel]

Win7 Compatibility mode does not appear to make a difference.

Comment 21

[Wayne Mery (:wsmwk)]

> - global-messages-db.sqlite file is about 1.2GB

this seems on the high side, or suggests that you may have messages with large server logfiles, or high volume mailing lists for example

I'd put my money on eset having some impact. And do you have Thunderbird filtering enabled to move incoming messages to folders, or have server side filters?

Comment 22

[Adam Kessel]

All filtering is server-side with .procmailrc.

I do have a lot of mail. ~/Maildir folder serverside is about 15GB or 370K messages.

Would it make sense to install a portable version and only subscribe to a small set of folders to see if that makes a difference?

Comment 23

[Wayne Mery (:wsmwk)]

(In reply to Adam Kessel from comment #22)
> All filtering is server-side with .procmailrc.
> 
> I do have a lot of mail. ~/Maildir folder serverside is about 15GB or 370K
> messages.
> 
> Would it make sense to install a portable version and only subscribe to a
> small set of folders to see if that makes a difference?

Are you now on version 52?

In comment 1 I cited the wiki article.  Did you try disabling indexing on windows and in Thunderbird? (A future step is to delete folders on disk and have sync disabled)o 

I would not try portable - that will likely be worse. You wrote memory and CPU usage are both low.  So I'm doubtful most of the normal tweaks would help.  But in configuration editor try setting mail.db.max_open to 60.  

Also, what is the size of your three largest "active" folders in MB and #messages? You can get the size by right+click folder and pick properties (active meaning they receive new mail or you move mail to them, or are part of a virtual folder)

In comment 2 you wrote "Task manager shows RAM usage about 160MB". That sounds too low. Which OS? Which measurement tool?

In general this case is problematic because on your windows system eset is scanning your profile and surely must be affecting performance. On Mac, time machine can have the same effect. Linux, I'm not familiar with. But a question, what is size of /tmp ?

Comment 24

[Adam Kessel]

On version 52.3.0--this problem has been unchanged for years.

I did find, however, that disabling Windows Search -- unchecking "Allow Windows Search to search messages" -- has at least tentatively eliminated the problem entirely. I tried lots of other things before, including rebuilding all folders from scratch, and none of them made any difference. So it does seem to be a Windows Search integration issue. It's nice to have system-wide search include Thunderbird email, of course, but probably not worth the regular freezes. Is there anything more I can do on my end to help Thunderbird and Microsoft work this out between them so this functionality can be better used?

On your questions:

(1) my biggest folders are in the 15K-20K messages, 1.5GB-2GB size

(2) I'm monitoring RAM usage by Windows 10 Task Manager (the built-in tool), which shows 178.5MB memory usage at this exact moment.

(3) On Linux, /tmp is tiny (never more than a couple of megabytes). Obviously the Windows Search solution I identified above is irrelevant to Linux and OSX freezes, so there must be something else/additional going on there. But I'm not on those platforms day-to-day now; I'll check to see if the problem is still occurring with latest Linux releases and report back here.

Comment 25

[Wayne Mery (:wsmwk)]

Thanks for all the details. The Windows issue is bug 1249056. We do not know why it is so bad in recent years