Closed Bug 487965 Opened 15 years ago Closed 15 years ago

hang on shutdown with open imap connections and zero cpu

Categories

(MailNews Core :: Backend, defect)

x86
Windows Vista
defect
Not set
critical

Tracking

(Not tracked)

RESOLVED FIXED
Thunderbird 3.0b3

People

(Reporter: wsmwk, Assigned: Bienvenu)

References

Details

(Keywords: dogfood, hang, regression)

Attachments

(5 files, 1 obsolete file)

+++ This bug was initially created as a clone of Bug #485995 +++

hang on shutdown with open imap connections. no cpu.
happened on two systems (XP and vista) both using 20090407. 
never seen before, so regression. but don't know from what.
previous version running on XP was 20090331.

8 imaps connections in CLOSE_WAIT for 2 imap accounts
now seen on third machine (home laptop vista) using 20090408
Blast.  (on my vista desktop) I would have an imap log for this issue if not for the fact that when I attempt to start a new thunderbird, it blows away the imap log of the instance that hasn't completely shut down. (my imap logging is always on)  

The new startup dies silently, even before it gets to profile manager.  So some part of the startup/profilemanager gets run that blows away nspr log, before some later part of startup dies.
Stack from hung shutdown (derived using VS2005 from process dump)

>ntdll.dll!_KiFastSystemCallRet@0() 	
 ntdll.dll!_ZwWaitForSingleObject@12()  + 0xc bytes	
 ntdll.dll!_RtlpWaitOnCriticalSection@8()  + 0xc8 bytes	
 ntdll.dll!_RtlEnterCriticalSection@4()  - 0x4b84 bytes	
 nspr4.dll!PR_Lock(PRLock * lock=0x061a5470)  Line 240	C
 nspr4.dll!PR_EnterMonitor(PRMonitor * mon=0x05d407c0)  Line 99 + 0x6 bytes	C
 thunderbird.exe!nsImapProtocol::TellThreadToDie()  Line 1239	C++
 thunderbird.exe!nsImapMockChannel::Cancel(unsigned int status=0)  Line 8838	C++
 thunderbird.exe!nsLoadGroup::Cancel(unsigned int status=2152398850)  Line 347	C++
 thunderbird.exe!nsDocLoader::Stop()  Line 313 + 0xb bytes	C++
 thunderbird.exe!nsDocShell::Stop(unsigned int aStopFlags=1)  Line 3532	C++
 thunderbird.exe!nsMsgWindow::StopUrls()  Line 370 + 0xe bytes	C++
 thunderbird.exe!nsMsgWindow::CloseWindow()  Line 143	C++
 xpcom_core.dll!NS_InvokeByIndex_P(nsISupports * that=0x0000001c, unsigned int methodIndex=0, unsigned int paramCount=1699684, nsXPTCVariant * params=0x04006000)  Line 102	C++
 thunderbird.exe!XPCWrappedNative::CallMethod(XPCCallContext & ccx={...}, XPCWrappedNative::CallMode mode=109113952)  Line 2434 + 0x16 bytes C++ 0680f260()	
 js3250.dll!JS_malloc(JSContext * cx=, unsigned int nbytes=)  Line 1871 + 0x7 bytes	C++
 js3250.dll!PurgeProtoChain(JSContext * cx=0x02762460, JSObject * obj=0x00000000, int id=43298328)  Line 3583 + 0x9 bytes	C++
 js3250.dll!js_DefineNativeProperty(JSContext * cx=0x04006000, JSObject * obj=0x00000000, int id=69534052, int value=104127552, int (JSContext *, JSObject *, int, int *)* getter=0x724474e0, int (JSContext *, JSObject *, int, int *)* setter=0x00fffb07, unsigned int attrs=7, unsigned int flags=0, int shortid=0, JSProperty * * propp=0x00000000, JSPropCacheEntry * * entryp=0x00000000)  Line 3783 + 0x2b bytes	C++
 js3250.dll!js_DefineProperty(JSContext * cx=0x04006000, JSObject * obj=0x0680f260, int id=1700144, int value=1917350144, int (JSContext *, JSObject *, int, int *)* getter=0x04006000, int (JSContext *, JSObject *, int, int *)* setter=0x0634dc40, unsigned int attrs=1916626288, JSProperty * * propp=0x04006000)  Line 3669 + 0x32 bytes	C++
 thunderbird.exe!DefinePropertyIfFound(XPCCallContext & ccx={...}, JSObject * obj=0x0019f11c)  Line 466 + 0x34 bytes	C++
 thunderbird.exe!XPC_WN_CallMethod(JSContext * cx=0x04006000, JSObject * obj=0x0680f260, unsigned int argc=0, int * argv=0x0498e14c, int * vp=0x0019f238)  Line 1587 + 0xa bytes	C++
 js3250.dll!js_Invoke(JSContext * cx=0x04006000, unsigned int argc=0, int * vp=0x0498e144, unsigned int flags=2)  Line 1341 + 0x1d bytes	C++
 js3250.dll!js_Interpret(JSContext * cx=0x04006000)  Line 5045	C++
 js3250.dll!js_Invoke(JSContext * cx=0x04006000, unsigned int argc=1, int * vp=0x0498e024, unsigned int flags=0)  Line 1359 + 0xa bytes	C++
 js3250.dll!js_InternalInvoke(JSContext * cx=0x04006000, JSObject * obj=0x040916e0, int fval=69466560, unsigned int flags=0, unsigned int argc=1, int * argv=0x0498e020, int * rval=0x0019f4e0)  Line 1417 + 0xf bytes	C++
 js3250.dll!JS_CallFunction(JSContext * cx=0x04006000, JSObject * obj=0x040916e0, JSFunction * fun=0x0423f9c0, unsigned int argc=1, int * argv=0x0498e020, int * rval=0x0019f4e0)  Line 5152	C++
 thunderbird.exe!nsJSContext::CallEventHandler(nsISupports * aTarget=0x00000001, void * aScope=0x0498e020, void * aHandler=0x0423f9c0, nsIArray * aargv=0x0498e018, nsIVariant * * arv=0x0019f540)  Line 2007 + 0x19 bytes	C++
 thunderbird.exe!nsJSEventListener::HandleEvent(nsIDOMEvent * aEvent=0x0692a9a0)  Line 249	C++
 	thunderbird.exe!nsEventListenerManager::HandleEventSubType(nsListenerStruct * aListenerStruct=0x054aec20, nsIDOMEventListener * aListener=0x04292f00, nsIDOMEvent * aDOMEvent=0x0692a9a0, nsPIDOMEventTarget * aCurrentTarget=0x028ce0f0, unsigned int aPhaseFlags=6)  Line 1101 + 0x9 bytes	C++
 thunderbird.exe!nsEventListenerManager::HandleEvent(nsPresContext * aPresContext=0x04226000, nsEvent * aEvent=0x0019f864, nsIDOMEvent * * aDOMEvent=0x0019f804, nsPIDOMEventTarget * aCurrentTarget=0x028ce0f0, unsigned int aFlags=6, nsEventStatus * aEventStatus=0x0019f808)  Line 1206 + 0x15 bytes	C++
 	thunderbird.exe!nsEventTargetChainItem::HandleEvent(nsEventChainPostVisitor & aVisitor={...}, unsigned int aFlags=6, int aMayHaveNewListenerManagers=1)  Line 237	C++
 	thunderbird.exe!nsEventTargetChainItem::HandleEventTargetChain(nsEventChainPostVisitor & aVisitor={...}, unsigned int aFlags=6, nsDispatchingCallback * aCallback=0x00000000, int aMayHaveNewListenerManagers=1)  Line 302	C++
 thunderbird.exe!nsEventDispatcher::Dispatch(nsISupports * aTarget=0x028ce0b0, nsPresContext * aPresContext=0x04226000, nsEvent * aEvent=0x00000000, nsIDOMEvent * aDOMEvent=0x00000000, nsEventStatus * aEventStatus=0x0019f898, nsDispatchingCallback * aCallback=0x00000000)  Line 518	C++
 thunderbird.exe!DocumentViewerImpl::PageHide(int aIsUnload=0)  Line 1210	C++
 thunderbird.exe!nsDocShell::FirePageHideNotification(int aIsUnload=1)  Line 1053	C++
 thunderbird.exe!nsDocShell::Destroy()  Line 3794	C++
 thunderbird.exe!nsXULWindow::Destroy()  Line 523	C++
 thunderbird.exe!nsWebShellWindow::Destroy()  Line 874 + 0x6 bytes	C++
 thunderbird.exe!nsChromeTreeOwner::Destroy()  Line 384	C++
 thunderbird.exe!nsGlobalWindow::ReallyCloseWindow()  Line 5604	C++
 thunderbird.exe!nsCloseEvent::Run()  Line 5412	C++
 xpcom_core.dll!nsThread::ProcessNextEvent(int mayWait=1, int * result=0x0019f9f4)  Line 511	C++
 xpcom_core.dll!NS_ProcessNextEvent_P(nsIThread * thread=0x00000001, int mayWait=1)  Line 227 + 0xd bytes	C++
 thunderbird.exe!nsBaseAppShell::Run()  Line 170 + 0x9 bytes	C++
 thunderbird.exe!nsAppStartup::Run()  Line 193	C++
 thunderbird.exe!XRE_main(int argc=1, char * * argv=0x0271e090, const nsXREAppData * aAppData=0x02715380)  Line 3283	C++
 thunderbird.exe!NS_internal_main(int argc=1, char * * argv=0x0271e090)  Line 104	C++
 thunderbird.exe!wmain(int argc=41017488, wchar_t * * argv=0x0271c700)  Line 89	C++
 thunderbird.exe!__tmainCRTStartup()  Line 591 + 0x19 bytes	C
 kernel32.dll!@BaseThreadInitThunk@12()  + 0x12 bytes	
 ntdll.dll!___RtlUserThreadStart@8()  + 0x27 bytes	


xref bug 482094 for completeness, but should not be related.
nsImapMockChannel::Cancel is called from the UI thread, so I believe it should be calling the proxied version of TellThreadToDie, through the nsIImapProtocol interface. This is probably generating warnings in debug builds. Possible fix upcoming...
Oh good, we can remove that nasty static cast ;-)

Which thread does ProcessCurrentURL call nsImapMockChannel::Cancel on?
(In reply to comment #5)
> 
> Which thread does ProcessCurrentURL call nsImapMockChannel::Cancel on?

good question - the imap thread, but I don't think that's a good thing. We can't control which thread the load group will cancel the imap mock channel on - I believe that's always the UI thread, and I believe in general channels get canceled on the ui thread. My initial thought is that the code in ProcessCurrentUrl should doom the cache entry some other way than canceling the mock channel.
this is not a gospel reproduction case - but in at least two cases I had an unsent compose window open when I started shutdown.

for now, assuming this is regression of bug 485995
Blocks: 485995
or, perhaps bug 420744 (which I had forgotten about, thinking bug 382446 fixed it) never really went away and bug 485995 resurfaced the issue for me. 

not to confuse things, but bug 392235 hang when server being down may dovetail with comments in bug 420744 about shutdown hang when network was down.
Blocks: 420744
Summary: hang on shutdown with open imap connections → hang on shutdown with open imap connections and zero cpu
Attached patch proposed fixSplinter Review
I haven't recreated this, but we shouldn't be calling nsImapMockChannel::Cancel from non ui threads, and it should be calling TellThreadToDie via the xpcom proxy, since it should be called only from the ui thread.

So I added a helper routine to doom cache entries, and made ProcessCurrentUrl call that. There's also a little whitespace cleanup.
Assignee: nobody → bienvenu
Attachment #372966 - Flags: superreview?(bugzilla)
Attachment #372966 - Flags: review?(neil)
Attachment #372966 - Flags: review?(neil) → review+
Attachment #372966 - Flags: superreview?(bugzilla) → superreview+
I've checked in a potential fix - Wayne, let me know if you see this with builds from after today. If you do, I suspect the stack trace will be different so that would be useful information.
Status: NEW → RESOLVED
Closed: 15 years ago
Flags: blocking-thunderbird3+
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 3.0b3
I'm back from vac. Like stephend in bug 420744, still see this using 20090421033807. Sounds crazy but I don't find what looks like a main thread. 

9 imap threads are like:

>	ntdll.dll!_KiFastSystemCallRet@0() 	
 	ntdll.dll!_ZwWaitForSingleObject@12()  + 0xc bytes	
 	kernel32.dll!_WaitForSingleObjectEx@12()  + 0x8b bytes	
 	kernel32.dll!_WaitForSingleObject@8()  + 0x12 bytes	
 	nspr4.dll!00299c8b() 	
 	[Frames below may be incorrect and/or missing, no symbols loaded for nspr4.dll]	
 	nspr4.dll!00295bf9() 	
 	nspr4.dll!002959e3() 	
 	xpcom_core.dll!002da1a2() 	
 	xpcom_core.dll!002dca9f() 	
 	xpcom_core.dll!002b6245() 	
 	xpcom_core.dll!002dea5a() 	
 	xpcom_core.dll!002e3d6c() 	
 	mozcrt19.dll!78138f8b() 	
 	mozcrt19.dll!78138f95() 	
 	mozcrt19.dll!78139cdf() 	
 	mozcrt19.dll!78138ef8() 	
 	mozcrt19.dll!78138f8b() 	
 	mozcrt19.dll!78138f95() 	
 	mozcrt19.dll!78139cdf() 	
 	xpcom_core.dll!002eb1ef() 	
 	xpcom_core.dll!002ebaa5() 	
 	xpcom_core.dll!002e3dd3() 	
 	thunderbird.exe!nsImapProtocol::TellThreadToDie()  Line 1240	C++
 	thunderbird.exe!nsImapProtocol::ImapThreadMainLoop()  Line 1328	C++
 	thunderbird.exe!nsImapProtocol::Run()  Line 1051	C++
 	xpcom_core.dll!002dcabd() 	
 	xpcom_core.dll!002b6245() 	
 	xpcom_core.dll!002dc958() 	
 	nspr4.dll!00299dbf() 	
 	nspr4.dll!002964bb() 	
 	nspr4.dll!002985dd() 	
 	mozcrt19.dll!78132c28() 	
 	mozcrt19.dll!78132cb6() 	
 	kernel32.dll!_BaseThreadStart@8()  + 0x37 bytes
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
hung instance is still running, so I hooked it up to windbg with !analayze -v hang.  Not sure it's accurate, but this looks a little different from comment 3.


STACK_TEXT:  
1272f914 77609254 769ec244 000006a4 00000000 ntdll!KiFastSystemCallRet
1272f918 769ec244 000006a4 00000000 00000000 ntdll!ZwWaitForSingleObject+0xc
1272f988 769ec1b2 000006a4 ffffffff 00000000 kernel32!WaitForSingleObjectEx+0xbe
1272f99c 10019c5b 000006a4 ffffffff 0bb53d30 kernel32!WaitForSingleObject+0x12
1272f9b8 10015bc9 0bb53e34 0bb53d4c ffffffff nspr4!_PR_MD_WAIT_CV+0x8b [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\nsprpub\pr\src\md\windows\w95cv.c @ 282]
1272f9d0 100159b3 0df2eac0 0bb53dc0 0bb53d30 nspr4!_PR_WaitCondVar+0x59 [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\nsprpub\pr\src\threads\combined\prucv.c @ 205]
1272f9ec 727fa1d2 029c2330 ffffffff 00000001 nspr4!PR_Wait+0x33 [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\nsprpub\pr\src\threads\prmon.c @ 175]
1272fa10 727fcb18 00000001 1272fa30 80000000 xpcom_core!nsEventQueue::GetEvent+0x30 [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\xpcom\threads\nseventqueue.cpp @ 85]
1272fa34 727d6232 00000001 00000001 1272fa50 xpcom_core!nsThread::ProcessNextEvent+0xa5 [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\xpcom\threads\nsthread.cpp @ 503]
1272fa48 727fead9 06360ab0 00000001 00000000 xpcom_core!NS_ProcessNextEvent_P+0x20 [e:\builds\buildbot\comm-central-win32-nightly\build\objdir-tb\mozilla\xpcom\build\nsthreadutils.cpp @ 227]
1272fa6c 72803b84 06360ab0 04093340 06360ab0 xpcom_core!nsProxyEventObject::CallMethod+0x18d [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\xpcom\proxy\src\nsproxyeventobject.cpp @ 260]
1272fb20 72803beb 05e0ba00 00000003 1272fb48 xpcom_core!PrepareAndDispatch+0xe7 [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\xpcom\reflect\xptcall\src\md\win32\xptcstubs.cpp @ 114]
1272fb3c 0157c45a 05e0ba00 00000000 043d8800 xpcom_core!SharedStub+0x16 [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\xpcom\reflect\xptcall\src\md\win32\xptcstubs.cpp @ 142]
1272fb70 0157c640 00000000 1272fbac 043d8800 thunderbird!nsImapProtocol::TellThreadToDie+0x15a [e:\builds\buildbot\comm-central-win32-nightly\build\mailnews\imap\src\nsimapprotocol.cpp @ 1226]
1272fb90 0157e325 00000000 06360ab0 00000000 thunderbird!nsImapProtocol::ImapThreadMainLoop+0x16c [e:\builds\buildbot\comm-central-win32-nightly\build\mailnews\imap\src\nsimapprotocol.cpp @ 1314]
1272fbac 727fcb36 043d8804 00000000 7282c2f0 thunderbird!nsImapProtocol::Run+0x43 [e:\builds\buildbot\comm-central-win32-nightly\build\mailnews\imap\src\nsimapprotocol.cpp @ 1037]
1272fbcc 727d6232 00000001 00000001 1272fbe8 xpcom_core!nsThread::ProcessNextEvent+0xc3 [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\xpcom\threads\nsthread.cpp @ 511]
1272fbe0 727fc9d1 00000001 00000001 0df2eb6c xpcom_core!NS_ProcessNextEvent_P+0x20 [e:\builds\buildbot\comm-central-win32-nightly\build\objdir-tb\mozilla\xpcom\build\nsthreadutils.cpp @ 227]
1272fc08 1001648b 06360ab0 027113e0 027113e0 xpcom_core!nsThread::ThreadFunc+0x62 [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\xpcom\threads\nsthread.cpp @ 254]
1272fc24 100185ad 0498c400 72492c28 0df2eac0 nspr4!_PR_NativeRunThread+0xdb [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\nsprpub\pr\src\threads\combined\pruthr.c @ 458]
1272fc2c 72492c28 0df2eac0 918af6a5 00000000 nspr4!pr_root+0xd [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\nsprpub\pr\src\md\windows\w95thred.c @ 122]
1272fc64 72492cb6 00000000 769e4911 053f3000 MOZCRT19!_callthreadstartex+0x48 [e:\builds\buildbot\comm-central-win32-nightly\build\objdir-tb\mozilla\memory\jemalloc\src\threadex.c @ 348]
1272fc6c 769e4911 053f3000 1272fcb8 775ee4b6 MOZCRT19!_threadstartex+0x66 [e:\builds\buildbot\comm-central-win32-nightly\build\objdir-tb\mozilla\memory\jemalloc\src\threadex.c @ 326]
1272fc78 775ee4b6 053f3000 7d86127c 00000000 kernel32!BaseThreadInitThunk+0xe
1272fcb8 775ee489 72492c50 053f3000 00000000 ntdll!__RtlUserThreadStart+0x23
1272fcd0 00000000 72492c50 053f3000 00000000 ntdll!_RtlUserThreadStart+0x1b

FOLLOWUP_IP: 
nspr4!_PR_MD_WAIT_CV+8b [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\nsprpub\pr\src\md\windows\w95cv.c @ 282]
10019c5b 55              push    ebp

SYMBOL_STACK_INDEX:  4
SYMBOL_NAME:  nspr4!_PR_MD_WAIT_CV+8b
FOLLOWUP_NAME:  MachineOwner
MODULE_NAME: nspr4
IMAGE_NAME:  nspr4.dll
DEBUG_FLR_IMAGE_TIMESTAMP:  49db2b71
STACK_COMMAND:  ~26s ; kb
BUCKET_ID:  HANG_nspr4!_PR_MD_WAIT_CV+8b
FAILURE_BUCKET_ID:  APPLICATION_HANG_cfffffff_nspr4.dll!_PR_MD_WAIT_CV
vista @ home  buildid 20090425   6 imaps ports in CLOSE_WAIT open per netstat

0b40f89c 77749254 760dc244 000004b4 00000000 ntdll!KiFastSystemCallRet
0b40f8a0 760dc244 000004b4 00000000 00000000 ntdll!ZwWaitForSingleObject+0xc
0b40f910 760dc1b2 000004b4 ffffffff 00000000 kernel32!WaitForSingleObjectEx+0xbe
0b40f924 10019c8b 000004b4 ffffffff 03c91790 kernel32!WaitForSingleObject+0x12
0b40f940 10015bf9 03c91fe4 03c917ac ffffffff nspr4!_PR_MD_WAIT_CV+0x8b [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\nsprpub\pr\src\md\windows\w95cv.c @ 282]
0b40f958 100159e3 066c8280 03c91f70 03c91790 nspr4!_PR_WaitCondVar+0x59 [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\nsprpub\pr\src\threads\combined\prucv.c @ 205]
0b40f974 72a6a177 03623c80 ffffffff 00000001 nspr4!PR_Wait+0x33 [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\nsprpub\pr\src\threads\prmon.c @ 175]
0b40f998 72a6ca9f 00000001 0b40f9b8 80000000 xpcom_core!nsEventQueue::GetEvent+0x30 [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\xpcom\threads\nseventqueue.cpp @ 85]
0b40f9bc 72a46238 00000001 00000001 0b40f9d8 xpcom_core!nsThread::ProcessNextEvent+0xa5 [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\xpcom\threads\nsthread.cpp @ 503]
0b40f9d0 72a6ea5a 0473ea60 00000001 00000000 xpcom_core!NS_ProcessNextEvent_P+0x20 [e:\builds\buildbot\comm-central-win32-nightly\build\objdir-tb\mozilla\xpcom\build\nsthreadutils.cpp @ 227]
0b40f9f4 72a73d6c 0473ea60 0477d7c0 0473ea60 xpcom_core!nsProxyEventObject::CallMethod+0x18d [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\xpcom\proxy\src\nsproxyeventobject.cpp @ 260]
0b40faa8 72a73dd3 03c6cfb0 00000003 0b40fad0 xpcom_core!PrepareAndDispatch+0xe7 [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\xpcom\reflect\xptcall\src\md\win32\xptcstubs.cpp @ 114]
0b40fac4 007fee05 03c6cfb0 00000000 04951c00 xpcom_core!SharedStub+0x16 [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\xpcom\reflect\xptcall\src\md\win32\xptcstubs.cpp @ 142]
0b40faf8 008031d0 00000000 0b40fb34 04951c00 thunderbird!nsImapProtocol::TellThreadToDie+0x15a [e:\builds\buildbot\comm-central-win32-nightly\build\mailnews\imap\src\nsimapprotocol.cpp @ 1240]
0b40fb18 00804f91 00000000 0473ea60 00000000 thunderbird!nsImapProtocol::ImapThreadMainLoop+0x16c [e:\builds\buildbot\comm-central-win32-nightly\build\mailnews\imap\src\nsimapprotocol.cpp @ 1328]
0b40fb34 72a6cabd 04951c04 00000000 72a9c2f0 thunderbird!nsImapProtocol::Run+0x43 [e:\builds\buildbot\comm-central-win32-nightly\build\mailnews\imap\src\nsimapprotocol.cpp @ 1051]
0b40fb54 72a46238 00000001 00000001 0b40fb70 xpcom_core!nsThread::ProcessNextEvent+0xc3 [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\xpcom\threads\nsthread.cpp @ 511]
0b40fb68 72a6c958 00000001 00000001 066c832c xpcom_core!NS_ProcessNextEvent_P+0x20 [e:\builds\buildbot\comm-central-win32-nightly\build\objdir-tb\mozilla\xpcom\build\nsthreadutils.cpp @ 227]
0b40fb90 100164bb 0473ea60 02521350 02521350 xpcom_core!nsThread::ThreadFunc+0x62 [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\xpcom\threads\nsthread.cpp @ 254]
0b40fbac 100185dd 047479a0 73ac2c28 066c8280 nspr4!_PR_NativeRunThread+0xdb [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\nsprpub\pr\src\threads\combined\pruthr.c @ 458]
0b40fbb4 73ac2c28 066c8280 0aac8025 00000000 nspr4!pr_root+0xd [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\nsprpub\pr\src\md\windows\w95thred.c @ 122]
0b40fbec 73ac2cb6 00000000 760d4911 04952c00 MOZCRT19!_callthreadstartex+0x48 [e:\builds\buildbot\comm-central-win32-nightly\build\objdir-tb\mozilla\memory\jemalloc\src\threadex.c @ 348]
0b40fbf4 760d4911 04952c00 0b40fc40 7772e4b6 MOZCRT19!_threadstartex+0x66 [e:\builds\buildbot\comm-central-win32-nightly\build\objdir-tb\mozilla\memory\jemalloc\src\threadex.c @ 326]
0b40fc00 7772e4b6 04952c00 5d591b9d 00000000 kernel32!BaseThreadInitThunk+0xe
0b40fc40 7772e489 73ac2c50 04952c00 00000000 ntdll!__RtlUserThreadStart+0x23
0b40fc58 00000000 73ac2c50 04952c00 00000000 ntdll!_RtlUserThreadStart+0x1b


FOLLOWUP_IP: 
nspr4!_PR_MD_WAIT_CV+8b [e:\builds\buildbot\comm-central-win32-nightly\build\mozilla\nsprpub\pr\src\md\windows\w95cv.c @ 282]
10019c8b 55              push    ebp
SYMBOL_STACK_INDEX:  4
SYMBOL_NAME:  nspr4!_PR_MD_WAIT_CV+8b
FOLLOWUP_NAME:  MachineOwner
MODULE_NAME: nspr4
IMAGE_NAME:  nspr4.dll
DEBUG_FLR_IMAGE_TIMESTAMP:  49f2e72e
STACK_COMMAND:  ~19s ; kb
BUCKET_ID:  HANG_nspr4!_PR_MD_WAIT_CV+8b
while this is isn't preventing testing, this is becoming dogfoodish in the sense that it's quite annoying and ...

it's sort bad have this lingering because if another shutdown regression comes along (we've got big landings coming up in gloda, etc) then we may not catch it as quickly with this bug active.

It would also be good to determine if this and bug 420744 are the same, or different.
Keywords: dogfood
shutdown today on D531 Vista - I closed 3pane (which went away), but I had message search open.  Search window didn't close and windows shows it as "Not Responding". netstat shows the imap connections open
seen with Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b5pre) Gecko/20090515 Shredder/3.0b3pre
which would have the fix for bug 492318
If I could reproduce this, I'd have a much better chance of fixing it...Wayne, breaking into the debugger and getting stack traces might be useful again.
Whiteboard: [need info on reproducing]
8 mail connections open per netstat

-------------------- main thread ------------------------------
>	ntdll.dll!_KiFastSystemCallRet@0() 	
 	ntdll.dll!_ZwWaitForSingleObject@12()  + 0xc bytes	
 	ntdll.dll!_RtlpWaitForCriticalSection@4()  + 0x8c bytes	
 	ntdll.dll!_RtlEnterCriticalSection@4()  + 0x46 bytes	
 	nspr4.dll!PR_Lock(PRLock * lock=0x03979110)  Line 234	C
 	nspr4.dll!PR_EnterMonitor(PRMonitor * mon=0x0390e640)  Line 99 + 0x6 bytes	C
 	thunderbird.exe!nsImapProtocol::TellThreadToDie(int aIsSafeToClose=0)  Line 1188	C++
 	thunderbird.exe!nsImapMockChannel::Cancel(unsigned int status=61736960)  Line 8876	C++
 	thunderbird.exe!nsLoadGroup::Cancel(unsigned int status=2152398850)  Line 347	C++
 	thunderbird.exe!nsDocLoader::Stop()  Line 313 + 0xb bytes	C++
 	thunderbird.exe!nsDocShell::Stop(unsigned int aStopFlags=1)  Line 3532	C++
 	thunderbird.exe!nsMsgWindow::StopUrls()  Line 370 + 0xe bytes	C++
 	thunderbird.exe!nsMsgWindow::CloseWindow()  Line 143	C++
 	xpcom_core.dll!NS_InvokeByIndex_P(nsISupports * that=0x0000001c, unsigned int methodIndex=0, unsigned int paramCount=1241572, nsXPTCVariant * params=0x0182be00)  Line 102	C++
 	thunderbird.exe!XPCWrappedNative::CallMethod(XPCCallContext & ccx={...}, XPCWrappedNative::CallMode mode=CALL_GETTER)  Line 2450 + 0x16 bytes	C++
 	03750340()	
 	mozcrt19.dll!arena_bin_nonfull_run_get(arena_s * arena=0x024b0280, arena_bin_s * bin=0x00000000)  Line 3796	C
 	js3250.dll!PurgeProtoChain(JSContext * cx=0x0182be00, JSObject * obj=0x00000000, int id=0)  Line 3599 + 0x9 bytes	C++
 	js3250.dll!js_DefineNativeProperty(JSContext * cx=, JSObject * obj=, int id=, int value=, int (JSContext *, JSObject *, int, int *)* getter=, int (JSContext *, JSObject *, int, int *)* setter=, unsigned int attrs=, unsigned int flags=, int shortid=, JSProperty * * propp=, int cacheResult=)  Line 3806 + 0x24 bytes	C++
 	js3250.dll!JS_DefinePropertyById(JSContext * cx=, JSObject * obj=, int id=, int value=, int (JSContext *, JSObject *, int, int *)* getter=, int (JSContext *, JSObject *, int, int *)* setter=, unsigned int attrs=)  Line 3113 + 0x28 bytes	C++
 	thunderbird.exe!DefinePropertyIfFound()  Line 464 + 0x30 bytes	C++
 	0012f39c()	
 	js3250.dll!ComputeThis(JSContext * cx=0x00000000, int lazy=25345536, int * argv=0x00000000)  Line 907 + 0x10 bytes	C++
 	js3250.dll!js_Invoke(JSContext * cx=, unsigned int argc=, int * vp=, unsigned int flags=)  Line 1365 + 0x1c bytes	C++
 	js3250.dll!js_Interpret(JSContext * cx=0x0182be00)  Line 5144	C++
 	js3250.dll!js_Invoke(JSContext * cx=0x0182be00, unsigned int argc=1, int * vp=0x035e1024, unsigned int flags=0)  Line 1373 + 0x6 bytes	C++
 	js3250.dll!js_InternalInvoke(JSContext * cx=0x0182be00, JSObject * obj=0x0249fc60, int fval=45213120, unsigned int flags=0, unsigned int argc=1, int * argv=0x035e1020, int * rval=0x0012f76c)  Line 1426 + 0xf bytes	C++
 	js3250.dll!JS_CallFunction(JSContext * cx=0x0182be00, JSObject * obj=0x0249fc60, JSFunction * fun=0x02b1e5c0, unsigned int argc=1, int * argv=0x035e1020, int * rval=0x0012f76c)  Line 5188	C++
 	thunderbird.exe!nsJSContext::CallEventHandler(nsISupports * aTarget=0x00000001, void * aScope=0x035e1020, void * aHandler=0x02b1e5c0, nsIArray * aargv=0x035e1018, nsIVariant * * arv=0x0012f7cc)  Line 2011 + 0x19 bytes	C++
 	thunderbird.exe!nsJSEventListener::HandleEvent(nsIDOMEvent * aEvent=0x07137130)  Line 249	C++
 	thunderbird.exe!nsEventListenerManager::HandleEventSubType(nsListenerStruct * aListenerStruct=0x02f4f820, nsIDOMEventListener * aListener=0x02b46420, nsIDOMEvent * aDOMEvent=0x07137130, nsPIDOMEventTarget * aCurrentTarget=0x01a652a0, unsigned int aPhaseFlags=6)  Line 1101 + 0x9 bytes	C++
 	thunderbird.exe!nsEventListenerManager::HandleEvent(nsPresContext * aPresContext=0x02593000, nsEvent * aEvent=0x0012faf0, nsIDOMEvent * * aDOMEvent=0x0012fa90, nsPIDOMEventTarget * aCurrentTarget=0x01a652a0, unsigned int aFlags=6, nsEventStatus * aEventStatus=0x0012fa94)  Line 1206 + 0x15 bytes	C++
 	thunderbird.exe!nsEventTargetChainItem::HandleEvent(nsEventChainPostVisitor & aVisitor={...}, unsigned int aFlags=6, int aMayHaveNewListenerManagers=1)  Line 237	C++
 	thunderbird.exe!nsEventTargetChainItem::HandleEventTargetChain(nsEventChainPostVisitor & aVisitor={...}, unsigned int aFlags=6, nsDispatchingCallback * aCallback=0x00000000, int aMayHaveNewListenerManagers=1)  Line 302	C++
 	thunderbird.exe!nsEventDispatcher::Dispatch(nsISupports * aTarget=0x01a65260, nsPresContext * aPresContext=0x02593000, nsEvent * aEvent=0x00000000, nsIDOMEvent * aDOMEvent=0x00000000, nsEventStatus * aEventStatus=0x0012fb24, nsDispatchingCallback * aCallback=0x00000000)  Line 518	C++
 	thunderbird.exe!DocumentViewerImpl::PageHide(int aIsUnload=0)  Line 1210	C++
 	thunderbird.exe!nsDocShell::FirePageHideNotification(int aIsUnload=1)  Line 1053	C++
 	thunderbird.exe!nsDocShell::Destroy()  Line 3794	C++
 	thunderbird.exe!nsXULWindow::Destroy()  Line 525	C++
 	thunderbird.exe!nsWebShellWindow::Destroy()  Line 874 + 0x6 bytes	C++
 	thunderbird.exe!nsChromeTreeOwner::Destroy()  Line 384	C++
 	thunderbird.exe!nsGlobalWindow::ReallyCloseWindow()  Line 5609	C++
 	thunderbird.exe!nsCloseEvent::Run()  Line 5417	C++
 	xpcom_core.dll!nsThread::ProcessNextEvent(int mayWait=1, int * result=0x0012fc80)  Line 511	C++
 	xpcom_core.dll!NS_ProcessNextEvent_P(nsIThread * thread=0x00000001, int mayWait=1)  Line 227 + 0xd bytes	C++
 	thunderbird.exe!nsBaseAppShell::Run()  Line 170 + 0x9 bytes	C++
 	thunderbird.exe!nsAppStartup::Run()  Line 194	C++
 	thunderbird.exe!XRE_main(int argc=1, char * * argv=0x01811090, const nsXREAppData * aAppData=0x01814340)  Line 3300	C++
 	thunderbird.exe!NS_internal_main(int argc=1, char * * argv=0x01811090)  Line 104	C++
 	thunderbird.exe!wmain(int argc=25235600, wchar_t * * argv=0x01823780)  Line 112	C++
 	thunderbird.exe!__tmainCRTStartup()  Line 591 + 0x19 bytes	C

imap in no particular order
------------------ imap connection 1 ----------------------
id 2284
 	ntdll.dll!_KiFastSystemCallRet@0() 	
 	ntdll.dll!_ZwWaitForSingleObject@12()  + 0xc bytes	
 	kernel32.dll!_WaitForSingleObjectEx@12()  + 0x8b bytes	
 	kernel32.dll!_WaitForSingleObject@8()  + 0x12 bytes	
 	nspr4.dll!_PR_MD_WAIT_CV(_MDCVar * cv=0x0c97e7b4, _MDLock * lock=0x0c97e6cc, unsigned int timeout=4294967295)  Line 282	C
 	nspr4.dll!_PR_WaitCondVar(PRThread * thread=0x02fe88c0, PRCondVar * cvar=0x0c97e740, PRLock * lock=0x0c97e6b0, unsigned int timeout=4294967295)  Line 205	C
 	nspr4.dll!PR_Wait(PRMonitor * mon=0x02fd8620, unsigned int ticks=4294967295)  Line 184 + 0x1a bytes	C
 	xpcom_core.dll!nsEventQueue::GetEvent(int mayWait=1, nsIRunnable * * result=0x04cffd78)  Line 85 + 0x9 bytes	C++
 	xpcom_core.dll!nsThread::ProcessNextEvent(int mayWait=1, int * result=0x04cffd98)  Line 503	C++
 	xpcom_core.dll!NS_ProcessNextEvent_P(nsIThread * thread=0x03a8a880, int mayWait=1)  Line 227 + 0xd bytes	C++
 	xpcom_core.dll!nsProxyEventObject::CallMethod(unsigned short methodIndex=448, const XPTMethodDescriptor * methodInfo=0x03a8a880, nsXPTCMiniVariant * params=0x01811ab8)  Line 260 + 0xa bytes	C++
 	xpcom_core.dll!PrepareAndDispatch(nsXPTCStubBase * self=0x03122e00, unsigned int methodIndex=3, unsigned int * args=0x04cffe90, unsigned int * stackBytesToPop=0x04cffe80)  Line 114 + 0x15 bytes	C++
 	xpcom_core.dll!SharedStub()  Line 142	C++
 	thunderbird.exe!nsImapProtocol::TellThreadToDie()  Line 1245	C++
 	thunderbird.exe!nsImapProtocol::TellThreadToDie()  Line 1245	C++
 	thunderbird.exe!nsImapProtocol::ImapThreadMainLoop()  Line 1333	C++
 	thunderbird.exe!nsImapProtocol::Run()  Line 1056	C++
 	xpcom_core.dll!nsThread::ProcessNextEvent(int mayWait=1, int * result=0x04cfff30)  Line 511	C++
 	xpcom_core.dll!NS_ProcessNextEvent_P(nsIThread * thread=0x00000001, int mayWait=1)  Line 227 + 0xd bytes	C++
 	xpcom_core.dll!nsThread::ThreadFunc(void * arg=0x03a8a880)  Line 254 + 0x8 bytes	C++
 	nspr4.dll!_PR_NativeRunThread(void * arg=0x03014280)  Line 448	C
 	nspr4.dll!pr_root(void * arg=0x02fe88c0)  Line 122 + 0xd bytes	C
>	mozcrt19.dll!_callthreadstartex()  Line 348 + 0x9 bytes	C

------------------ imap connection 2 ----------------------
id 1212
 	ntdll.dll!_KiFastSystemCallRet@0() 	
 	ntdll.dll!_ZwWaitForSingleObject@12()  + 0xc bytes	
 	kernel32.dll!_WaitForSingleObjectEx@12()  + 0x8b bytes	
 	kernel32.dll!_WaitForSingleObject@8()  + 0x12 bytes	
 	nspr4.dll!_PR_MD_WAIT_CV(_MDCVar * cv=0x0c97ede4, _MDLock * lock=0x0c97ecfc, unsigned int timeout=4294967295)  Line 282	C
 	nspr4.dll!_PR_WaitCondVar(PRThread * thread=0x02fecec0, PRCondVar * cvar=0x0c97ed70, PRLock * lock=0x0c97ece0, unsigned int timeout=4294967295)  Line 205	C
 	nspr4.dll!PR_Wait(PRMonitor * mon=0x0390e2b0, unsigned int ticks=4294967295)  Line 184 + 0x1a bytes	C
 	xpcom_core.dll!nsEventQueue::GetEvent(int mayWait=1, nsIRunnable * * result=0x069ffd78)  Line 85 + 0x9 bytes	C++
 	xpcom_core.dll!nsThread::ProcessNextEvent(int mayWait=1, int * result=0x069ffd98)  Line 503	C++
 	xpcom_core.dll!NS_ProcessNextEvent_P(nsIThread * thread=0x044733d0, int mayWait=1)  Line 227 + 0xd bytes	C++
 	xpcom_core.dll!nsProxyEventObject::CallMethod(unsigned short methodIndex=37504, const XPTMethodDescriptor * methodInfo=0x044733d0, nsXPTCMiniVariant * params=0x02f57290)  Line 260 + 0xa bytes	C++
 	xpcom_core.dll!PrepareAndDispatch(nsXPTCStubBase * self=0x0555cf00, unsigned int methodIndex=3, unsigned int * args=0x069ffe90, unsigned int * stackBytesToPop=0x069ffe80)  Line 114 + 0x15 bytes	C++
 	xpcom_core.dll!SharedStub()  Line 142	C++
 	thunderbird.exe!nsImapProtocol::TellThreadToDie()  Line 1245	C++
 	thunderbird.exe!nsImapProtocol::TellThreadToDie()  Line 1245	C++
 	thunderbird.exe!nsImapProtocol::ImapThreadMainLoop()  Line 1333	C++
 	thunderbird.exe!nsImapProtocol::Run()  Line 1056	C++
 	xpcom_core.dll!nsThread::ProcessNextEvent(int mayWait=1, int * result=0x069fff30)  Line 511	C++
 	xpcom_core.dll!NS_ProcessNextEvent_P(nsIThread * thread=0x00000001, int mayWait=1)  Line 227 + 0xd bytes	C++
 	xpcom_core.dll!nsThread::ThreadFunc(void * arg=0x044733d0)  Line 254 + 0x8 bytes	C++
 	nspr4.dll!_PR_NativeRunThread(void * arg=0x04464ca0)  Line 448	C
 	nspr4.dll!pr_root(void * arg=0x02fecec0)  Line 122 + 0xd bytes	C
>	mozcrt19.dll!_callthreadstartex()  Line 348 + 0x9 bytes	C
 	mozcrt19.dll!_threadstartex(void * ptd=0x03e76c00)  Line 326 + 0x5 bytes	C
 	kernel32.dll!_BaseThreadStart@8()  + 0x37 bytes	

------------------ imap connection 3 ----------------------
id 2316
 	ntdll.dll!_KiFastSystemCallRet@0() 	
 	ntdll.dll!_ZwWaitForSingleObject@12()  + 0xc bytes	
 	kernel32.dll!_WaitForSingleObjectEx@12()  + 0x8b bytes	
 	kernel32.dll!_WaitForSingleObject@8()  + 0x12 bytes	
 	nspr4.dll!_PR_MD_WAIT_CV(_MDCVar * cv=0x0c97e9f4, _MDLock * lock=0x0c97e90c, unsigned int timeout=4294967295)  Line 282	C
 	nspr4.dll!_PR_WaitCondVar(PRThread * thread=0x060a9c00, PRCondVar * cvar=0x0c97e980, PRLock * lock=0x0c97e8f0, unsigned int timeout=4294967295)  Line 205	C
 	nspr4.dll!PR_Wait(PRMonitor * mon=0x02469ae0, unsigned int ticks=4294967295)  Line 184 + 0x1a bytes	C
 	xpcom_core.dll!nsEventQueue::GetEvent(int mayWait=1, nsIRunnable * * result=0x07fffd78)  Line 85 + 0x9 bytes	C++
 	xpcom_core.dll!nsThread::ProcessNextEvent(int mayWait=1, int * result=0x07fffd98)  Line 503	C++
 	xpcom_core.dll!NS_ProcessNextEvent_P(nsIThread * thread=0x04a15970, int mayWait=1)  Line 227 + 0xd bytes	C++
 	xpcom_core.dll!nsProxyEventObject::CallMethod(unsigned short methodIndex=48000, const XPTMethodDescriptor * methodInfo=0x04a15970, nsXPTCMiniVariant * params=0x03066b18)  Line 260 + 0xa bytes	C++
 	xpcom_core.dll!PrepareAndDispatch(nsXPTCStubBase * self=0x064be940, unsigned int methodIndex=3, unsigned int * args=0x07fffe90, unsigned int * stackBytesToPop=0x07fffe80)  Line 114 + 0x15 bytes	C++
 	xpcom_core.dll!SharedStub()  Line 142	C++
 	thunderbird.exe!nsImapProtocol::TellThreadToDie()  Line 1245	C++
 	thunderbird.exe!nsImapProtocol::TellThreadToDie()  Line 1245	C++
 	thunderbird.exe!nsImapProtocol::ImapThreadMainLoop()  Line 1333	C++
 	thunderbird.exe!nsImapProtocol::Run()  Line 1056	C++
 	xpcom_core.dll!nsThread::ProcessNextEvent(int mayWait=1, int * result=0x07ffff30)  Line 511	C++
 	xpcom_core.dll!NS_ProcessNextEvent_P(nsIThread * thread=0x00000001, int mayWait=1)  Line 227 + 0xd bytes	C++
 	xpcom_core.dll!nsThread::ThreadFunc(void * arg=0x04a15970)  Line 254 + 0x8 bytes	C++
 	nspr4.dll!_PR_NativeRunThread(void * arg=0x035700a0)  Line 448	C
 	nspr4.dll!pr_root(void * arg=0x060a9c00)  Line 122 + 0xd bytes	C
>	mozcrt19.dll!_callthreadstartex()  Line 348 + 0x9 bytes	C
 	mozcrt19.dll!_threadstartex(void * ptd=0x07120000)  Line 326 + 0x5 bytes	C
 	kernel32.dll!_BaseThreadStart@8()  + 0x37 bytes	

------------------ imap connection 4 ----------------------
id 572
 	ntdll.dll!_KiFastSystemCallRet@0() 	
 	ntdll.dll!_ZwWaitForSingleObject@12()  + 0xc bytes	
 	kernel32.dll!_WaitForSingleObjectEx@12()  + 0x8b bytes	
 	kernel32.dll!_WaitForSingleObject@8()  + 0x12 bytes	
 	nspr4.dll!_PR_MD_WAIT_CV(_MDCVar * cv=0x0c97ecc4, _MDLock * lock=0x0c97ebdc, unsigned int timeout=4294967295)  Line 282	C
 	nspr4.dll!_PR_WaitCondVar(PRThread * thread=0x02fe8500, PRCondVar * cvar=0x0c97ec50, PRLock * lock=0x0c97ebc0, unsigned int timeout=4294967295)  Line 205	C
 	nspr4.dll!PR_Wait(PRMonitor * mon=0x03067080, unsigned int ticks=4294967295)  Line 184 + 0x1a bytes	C
 	xpcom_core.dll!nsEventQueue::GetEvent(int mayWait=1, nsIRunnable * * result=0x094ffd78)  Line 85 + 0x9 bytes	C++
 	xpcom_core.dll!nsThread::ProcessNextEvent(int mayWait=1, int * result=0x094ffd98)  Line 503	C++
 	xpcom_core.dll!NS_ProcessNextEvent_P(nsIThread * thread=0x03e47740, int mayWait=1)  Line 227 + 0xd bytes	C++
 	xpcom_core.dll!nsProxyEventObject::CallMethod(unsigned short methodIndex=37456, const XPTMethodDescriptor * methodInfo=0x03e47740, nsXPTCMiniVariant * params=0x02f57288)  Line 260 + 0xa bytes	C++
 	xpcom_core.dll!PrepareAndDispatch(nsXPTCStubBase * self=0x03dec800, unsigned int methodIndex=3, unsigned int * args=0x094ffe90, unsigned int * stackBytesToPop=0x094ffe80)  Line 114 + 0x15 bytes	C++
 	xpcom_core.dll!SharedStub()  Line 142	C++
 	thunderbird.exe!nsImapProtocol::TellThreadToDie()  Line 1245	C++
 	thunderbird.exe!nsImapProtocol::TellThreadToDie()  Line 1245	C++
 	thunderbird.exe!nsImapProtocol::ImapThreadMainLoop()  Line 1333	C++
 	thunderbird.exe!nsImapProtocol::Run()  Line 1056	C++
 	xpcom_core.dll!nsThread::ProcessNextEvent(int mayWait=1, int * result=0x094fff30)  Line 511	C++
 	xpcom_core.dll!NS_ProcessNextEvent_P(nsIThread * thread=0x00000001, int mayWait=1)  Line 227 + 0xd bytes	C++
 	xpcom_core.dll!nsThread::ThreadFunc(void * arg=0x03e47740)  Line 254 + 0x8 bytes	C++
 	nspr4.dll!_PR_NativeRunThread(void * arg=0x036e5fa0)  Line 448	C
 	nspr4.dll!pr_root(void * arg=0x02fe8500)  Line 122 + 0xd bytes	C
>	mozcrt19.dll!_callthreadstartex()  Line 348 + 0x9 bytes	C
 	mozcrt19.dll!_threadstartex(void * ptd=0x03ade800)  Line 326 + 0x5 bytes	C
 	kernel32.dll!_BaseThreadStart@8()  + 0x37 bytes	

------------------ imap connection 5 ----------------------
id 1092
 	ntdll.dll!_KiFastSystemCallRet@0() 	
 	ntdll.dll!_ZwWaitForSingleObject@12()  + 0xc bytes	
 	kernel32.dll!_WaitForSingleObjectEx@12()  + 0x8b bytes	
 	kernel32.dll!_WaitForSingleObject@8()  + 0x12 bytes	
 	nspr4.dll!_PR_MD_WAIT_CV(_MDCVar * cv=0x0c97eba4, _MDLock * lock=0x0c97eabc, unsigned int timeout=4294967295)  Line 282	C
 	nspr4.dll!_PR_WaitCondVar(PRThread * thread=0x035301c0, PRCondVar * cvar=0x0c97eb30, PRLock * lock=0x0c97eaa0, unsigned int timeout=4294967295)  Line 205	C
 	nspr4.dll!PR_Wait(PRMonitor * mon=0x03956340, unsigned int ticks=4294967295)  Line 184 + 0x1a bytes	C
 	xpcom_core.dll!nsEventQueue::GetEvent(int mayWait=1, nsIRunnable * * result=0x062ffd78)  Line 85 + 0x9 bytes	C++
 	xpcom_core.dll!nsThread::ProcessNextEvent(int mayWait=1, int * result=0x062ffd98)  Line 503	C++
 	xpcom_core.dll!NS_ProcessNextEvent_P(nsIThread * thread=0x0397bfb0, int mayWait=1)  Line 227 + 0xd bytes	C++
 	xpcom_core.dll!nsProxyEventObject::CallMethod(unsigned short methodIndex=55952, const XPTMethodDescriptor * methodInfo=0x0397bfb0, nsXPTCMiniVariant * params=0x02f57280)  Line 260 + 0xa bytes	C++
 	xpcom_core.dll!PrepareAndDispatch(nsXPTCStubBase * self=0x04d512e0, unsigned int methodIndex=3, unsigned int * args=0x062ffe90, unsigned int * stackBytesToPop=0x062ffe80)  Line 114 + 0x15 bytes	C++
 	xpcom_core.dll!SharedStub()  Line 142	C++
 	thunderbird.exe!nsImapProtocol::TellThreadToDie()  Line 1245	C++
 	thunderbird.exe!nsImapProtocol::TellThreadToDie()  Line 1245	C++
 	thunderbird.exe!nsImapProtocol::ImapThreadMainLoop()  Line 1333	C++
 	thunderbird.exe!nsImapProtocol::Run()  Line 1056	C++
 	xpcom_core.dll!nsThread::ProcessNextEvent(int mayWait=1, int * result=0x062fff30)  Line 511	C++
 	xpcom_core.dll!NS_ProcessNextEvent_P(nsIThread * thread=0x00000001, int mayWait=1)  Line 227 + 0xd bytes	C++
 	xpcom_core.dll!nsThread::ThreadFunc(void * arg=0x0397bfb0)  Line 254 + 0x8 bytes	C++
 	nspr4.dll!_PR_NativeRunThread(void * arg=0x030cd820)  Line 448	C
 	nspr4.dll!pr_root(void * arg=0x035301c0)  Line 122 + 0xd bytes	C
>	mozcrt19.dll!_callthreadstartex()  Line 348 + 0x9 bytes	C
 	mozcrt19.dll!_threadstartex(void * ptd=0x06cd2800)  Line 326 + 0x5 bytes	C
 	kernel32.dll!_BaseThreadStart@8()  + 0x37 bytes	

------------------ imap connection 6 ----------------------
id 3008
 	ntdll.dll!_KiFastSystemCallRet@0() 	
 	ntdll.dll!_ZwWaitForSingleObject@12()  + 0xc bytes	
 	kernel32.dll!_WaitForSingleObjectEx@12()  + 0x8b bytes	
 	kernel32.dll!_WaitForSingleObject@8()  + 0x12 bytes	
 	nspr4.dll!_PR_MD_WAIT_CV(_MDCVar * cv=0x0c97e8d4, _MDLock * lock=0x0c97e7ec, unsigned int timeout=4294967295)  Line 282	C
 	nspr4.dll!_PR_WaitCondVar(PRThread * thread=0x02fe8f00, PRCondVar * cvar=0x0c97e860, PRLock * lock=0x0c97e7d0, unsigned int timeout=4294967295)  Line 205	C
 	nspr4.dll!PR_Wait(PRMonitor * mon=0x030672d0, unsigned int ticks=4294967295)  Line 184 + 0x1a bytes	C
 	xpcom_core.dll!nsEventQueue::GetEvent(int mayWait=1, nsIRunnable * * result=0x05cffd78)  Line 85 + 0x9 bytes	C++
 	xpcom_core.dll!nsThread::ProcessNextEvent(int mayWait=1, int * result=0x05cffd98)  Line 503	C++
 	xpcom_core.dll!NS_ProcessNextEvent_P(nsIThread * thread=0x04525970, int mayWait=1)  Line 227 + 0xd bytes	C++
 	xpcom_core.dll!nsProxyEventObject::CallMethod(unsigned short methodIndex=47424, const XPTMethodDescriptor * methodInfo=0x04525970, nsXPTCMiniVariant * params=0x03066b10)  Line 260 + 0xa bytes	C++
 	xpcom_core.dll!PrepareAndDispatch(nsXPTCStubBase * self=0x04428270, unsigned int methodIndex=3, unsigned int * args=0x05cffe90, unsigned int * stackBytesToPop=0x05cffe80)  Line 114 + 0x15 bytes	C++
 	xpcom_core.dll!SharedStub()  Line 142	C++
 	thunderbird.exe!nsImapProtocol::TellThreadToDie()  Line 1245	C++
 	thunderbird.exe!nsImapProtocol::TellThreadToDie()  Line 1245	C++
 	thunderbird.exe!nsImapProtocol::ImapThreadMainLoop()  Line 1333	C++
 	thunderbird.exe!nsImapProtocol::Run()  Line 1056	C++
 	xpcom_core.dll!nsThread::ProcessNextEvent(int mayWait=1, int * result=0x05cfff30)  Line 511	C++
 	xpcom_core.dll!NS_ProcessNextEvent_P(nsIThread * thread=0x00000001, int mayWait=1)  Line 227 + 0xd bytes	C++
 	xpcom_core.dll!nsThread::ThreadFunc(void * arg=0x04525970)  Line 254 + 0x8 bytes	C++
 	nspr4.dll!_PR_NativeRunThread(void * arg=0x02f544f0)  Line 448	C
 	nspr4.dll!pr_root(void * arg=0x02fe8f00)  Line 122 + 0xd bytes	C
>	mozcrt19.dll!_callthreadstartex()  Line 348 + 0x9 bytes	C
 	mozcrt19.dll!_threadstartex(void * ptd=0x03e78000)  Line 326 + 0x5 bytes	C
 	kernel32.dll!_BaseThreadStart@8()  + 0x37 bytes	

------------------ imap connection 7 ----------------------
id 1932
 	ntdll.dll!_KiFastSystemCallRet@0() 	
 	ntdll.dll!_ZwWaitForSingleObject@12()  + 0xc bytes	
 	kernel32.dll!_WaitForSingleObjectEx@12()  + 0x8b bytes	
 	kernel32.dll!_WaitForSingleObject@8()  + 0x12 bytes	
 	nspr4.dll!_PR_MD_WAIT_CV(_MDCVar * cv=0x03a6c694, _MDLock * lock=0x03a6c5ac, unsigned int timeout=4294967295)  Line 282	C
 	nspr4.dll!_PR_WaitCondVar(PRThread * thread=0x02fe92c0, PRCondVar * cvar=0x03a6c620, PRLock * lock=0x03a6c590, unsigned int timeout=4294967295)  Line 205	C
 	nspr4.dll!PR_Wait(PRMonitor * mon=0x03933860, unsigned int ticks=4294967295)  Line 184 + 0x1a bytes	C
 	xpcom_core.dll!nsEventQueue::GetEvent(int mayWait=1, nsIRunnable * * result=0x0a4ffd78)  Line 85 + 0x9 bytes	C++
 	xpcom_core.dll!nsThread::ProcessNextEvent(int mayWait=1, int * result=0x0a4ffd98)  Line 503	C++
 	xpcom_core.dll!NS_ProcessNextEvent_P(nsIThread * thread=0x05e86fb0, int mayWait=1)  Line 227 + 0xd bytes	C++
 	xpcom_core.dll!nsProxyEventObject::CallMethod(unsigned short methodIndex=37408, const XPTMethodDescriptor * methodInfo=0x05e86fb0, nsXPTCMiniVariant * params=0x05598920)  Line 260 + 0xa bytes	C++
 	xpcom_core.dll!PrepareAndDispatch(nsXPTCStubBase * self=0x07c944a0, unsigned int methodIndex=3, unsigned int * args=0x0a4ffe90, unsigned int * stackBytesToPop=0x0a4ffe80)  Line 114 + 0x15 bytes	C++
 	xpcom_core.dll!SharedStub()  Line 142	C++
 	thunderbird.exe!nsImapProtocol::TellThreadToDie()  Line 1245	C++
 	thunderbird.exe!nsImapProtocol::TellThreadToDie()  Line 1245	C++
 	thunderbird.exe!nsImapProtocol::ImapThreadMainLoop()  Line 1333	C++
 	thunderbird.exe!nsImapProtocol::Run()  Line 1056	C++
 	xpcom_core.dll!nsThread::ProcessNextEvent(int mayWait=1, int * result=0x0a4fff30)  Line 511	C++
 	xpcom_core.dll!NS_ProcessNextEvent_P(nsIThread * thread=0x00000001, int mayWait=1)  Line 227 + 0xd bytes	C++
 	xpcom_core.dll!nsThread::ThreadFunc(void * arg=0x05e86fb0)  Line 254 + 0x8 bytes	C++
 	nspr4.dll!_PR_NativeRunThread(void * arg=0x06c15ac0)  Line 448	C
 	nspr4.dll!pr_root(void * arg=0x02fe92c0)  Line 122 + 0xd bytes	C
>	mozcrt19.dll!_callthreadstartex()  Line 348 + 0x9 bytes	C
 	mozcrt19.dll!_threadstartex(void * ptd=0x04ff8800)  Line 326 + 0x5 bytes	C
 	kernel32.dll!_BaseThreadStart@8()  + 0x37 bytes	

------------------ imap connection 8 ----------------------
id 1104
 	ntdll.dll!_KiFastSystemCallRet@0() 	
 	ntdll.dll!_ZwWaitForSingleObject@12()  + 0xc bytes	
 	kernel32.dll!_WaitForSingleObjectEx@12()  + 0x8b bytes	
 	kernel32.dll!_WaitForSingleObject@8()  + 0x12 bytes	
 	nspr4.dll!_PR_MD_WAIT_CV(_MDCVar * cv=0x0c97ea84, _MDLock * lock=0x06c1005c, unsigned int timeout=4294967295)  Line 282	C
 	nspr4.dll!_PR_WaitCondVar(PRThread * thread=0x02fe9b80, PRCondVar * cvar=0x0c97ea10, PRLock * lock=0x06c10040, unsigned int timeout=4294967295)  Line 205	C
 	nspr4.dll!PR_Wait(PRMonitor * mon=0x03067500, unsigned int ticks=4294967295)  Line 184 + 0x1a bytes	C
 	xpcom_core.dll!nsEventQueue::GetEvent(int mayWait=1, nsIRunnable * * result=0x0a3ffd78)  Line 85 + 0x9 bytes	C++
 	xpcom_core.dll!nsThread::ProcessNextEvent(int mayWait=1, int * result=0x0a3ffd98)  Line 503	C++
 	xpcom_core.dll!NS_ProcessNextEvent_P(nsIThread * thread=0x06d76290, int mayWait=1)  Line 227 + 0xd bytes	C++
 	xpcom_core.dll!nsProxyEventObject::CallMethod(unsigned short methodIndex=55904, const XPTMethodDescriptor * methodInfo=0x06d76290, nsXPTCMiniVariant * params=0x02f57120)  Line 260 + 0xa bytes	C++
 	xpcom_core.dll!PrepareAndDispatch(nsXPTCStubBase * self=0x07c94060, unsigned int methodIndex=3, unsigned int * args=0x0a3ffe90, unsigned int * stackBytesToPop=0x0a3ffe80)  Line 114 + 0x15 bytes	C++
 	xpcom_core.dll!SharedStub()  Line 142	C++
 	thunderbird.exe!nsImapProtocol::TellThreadToDie()  Line 1245	C++
 	thunderbird.exe!nsImapProtocol::TellThreadToDie()  Line 1245	C++
 	thunderbird.exe!nsImapProtocol::ImapThreadMainLoop()  Line 1333	C++
 	thunderbird.exe!nsImapProtocol::Run()  Line 1056	C++
 	xpcom_core.dll!nsThread::ProcessNextEvent(int mayWait=1, int * result=0x0a3fff30)  Line 511	C++
 	xpcom_core.dll!NS_ProcessNextEvent_P(nsIThread * thread=0x00000001, int mayWait=1)  Line 227 + 0xd bytes	C++
 	xpcom_core.dll!nsThread::ThreadFunc(void * arg=0x06d76290)  Line 254 + 0x8 bytes	C++
 	nspr4.dll!_PR_NativeRunThread(void * arg=0x07032160)  Line 448	C
 	nspr4.dll!pr_root(void * arg=0x02fe9b80)  Line 122 + 0xd bytes	C
>	mozcrt19.dll!_callthreadstartex()  Line 348 + 0x9 bytes	C
 	mozcrt19.dll!_threadstartex(void * ptd=0x04fd6c00)  Line 326 + 0x5 bytes	C
 	kernel32.dll!_BaseThreadStart@8()  + 0x37 bytes	

-------------------------------------------------------------------------
and FWIW
id 1072	nsFixedCString::nsFixedCString	_PR_MD_WAIT_CV	Normal	0

 	ntdll.dll!_KiFastSystemCallRet@0() 	
 	ntdll.dll!_ZwWaitForSingleObject@12()  + 0xc bytes	
 	kernel32.dll!_WaitForSingleObjectEx@12()  + 0x8b bytes	
 	kernel32.dll!_WaitForSingleObject@8()  + 0x12 bytes	
 	nspr4.dll!_PR_MD_WAIT_CV(_MDCVar * cv=0x01daf654, _MDLock * lock=0x01daf4dc, unsigned int timeout=60000)  Line 282	C
 	nspr4.dll!_PR_WaitCondVar(PRThread * thread=0x02fec4c0, PRCondVar * cvar=0x01daf5e0, PRLock * lock=0x01daf4c0, unsigned int timeout=60000)  Line 205	C
 	nspr4.dll!PR_Wait(PRMonitor * mon=0x024169e0, unsigned int ticks=60000)  Line 184 + 0x1a bytes	C
 	xpcom_core.dll!nsAutoMonitor::Wait(unsigned int interval=60000)  Line 340 + 0xc bytes	C++
 	xpcom_core.dll!nsThreadPool::Run()  Line 211	C++
 	xpcom_core.dll!nsThread::ProcessNextEvent(int mayWait=1, int * result=0x061fff30)  Line 511	C++
 	xpcom_core.dll!NS_ProcessNextEvent_P(nsIThread * thread=0x00000001, int mayWait=1)  Line 227 + 0xd bytes	C++
 	xpcom_core.dll!nsThread::ThreadFunc(void * arg=0x06d76f60)  Line 254 + 0x8 bytes	C++
 	nspr4.dll!_PR_NativeRunThread(void * arg=0x0012d404)  Line 448	C
 	xpcom_core.dll!nsFixedCString::nsFixedCString(char * data=0x0012d404, unsigned int storageSize=81541120, unsigned int length=2252946848)  Line 82 + 0x19 bytes	C++
 	xpcom_core.dll!nsFixedCString::nsFixedCString(char * data=0x0012d404, unsigned int storageSize=2088810281, unsigned int length=81541120)  Line 82 + 0x19 bytes	C++
>	mozcrt19.dll!_threadstartex(void * ptd=)  Line 326 + 0x5 bytes	C
those lines imply we're trying to enter m_threadDeathMonitor, but I don't see how we could contend over that, since we never do anything substantial while holding that monitor. But if the lines were slightly off, and we were trying to enter m_urlReadyToRunMonitor, then I could see how we'd get blocked. nsImapProtocol::ImapThreadMainLoop() holds onto the m_urlReadyToRunMonitor even if it calls TellThreadToDie(). TellThreadToDie can cause us to proxy back to the UI thread to close streams, but if the UI thread is waiting for the monitor, all the threads that want to proxy over to the UI thread will block. So if we're killing the thread, I think we need to release that monitor first. Patch coming up.
Attached patch possible fix (obsolete) — Splinter Review
maybe I'll try the shiny new try server stuff to get a build to Wayne.
Attachment #378913 - Flags: superreview?(neil)
Attachment #378913 - Flags: review?(bugzilla)
(In reply to comment #21)
> Created an attachment (id=378913) [details]
> possible fix
> 
> maybe I'll try the shiny new try server stuff to get a build to Wayne.

I'm interested in trying this out too, as I can reproduce my own hang on shutdown 100%.
Comment on attachment 378913 [details] [diff] [review]
possible fix

>       // This will happen if the UI thread signals us to die
>       if (m_threadShouldDie)
Why can't we move this check out of the monitor's scope?
(In reply to comment #23)
> (From update of attachment 378913 [details] [diff] [review])
> >       // This will happen if the UI thread signals us to die
> >       if (m_threadShouldDie)
> Why can't we move this check out of the monitor's scope?

That's the other option I considered - perhaps that's cleaner since we can keep using the auto monitor. I'll do that patch as well.
Neil, is this what you meant?
Attachment #378913 - Attachment is obsolete: true
Attachment #378972 - Flags: superreview?(bugzilla)
Attachment #378972 - Flags: review?(neil)
Attachment #378913 - Flags: superreview?(neil)
Attachment #378913 - Flags: review?(bugzilla)
Comment on attachment 378972 [details] [diff] [review]
alternate fix - checked in.

Yes, but I thought DeathSignalReceived() already checks m_threadShouldDie (and TellThreadToDie() also sets m_nextUrlReadyToRun).
Attachment #378972 - Flags: review?(neil) → review+
I get this hang on shutdown on Mac - Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1pre) Gecko/20090523 Lightning/1.0pre Shredder/3.0b3pre

To reproduce, start up debug TB then when the mail3pane UI displays (with open IMAP connections and a working net connection), shutdown TB immediately.

Here's a sample of the stack trace.
you can get bienvenu's 20090521 tryserver build at http://s3.mozillamessaging.com/build/try-server/list.html

running with it now. initial results are good, but need a couple days testing.
(In reply to comment #28)
> you can get bienvenu's 20090521 tryserver build at
> http://s3.mozillamessaging.com/build/try-server/list.html
> 
> running with it now. initial results are good, but need a couple days testing.

No mac build there though. I see only Linux and Windows.
try server mac build failed for some reason 

Re Neil's comment about DeathSignalReceived already returning True if m_threadShouldDie is set, it does, except in the case of Cancel, though canceling does call TellThreadToDie...so I could probably remove the extra check for m_threadShouldDie from the while loop, though it doesn't hurt and is a bit more robust in the face of other changes...
hung with my only shutdown today using the tryserver build
Attachment #378972 - Flags: superreview?(bugzilla) → superreview+
(In reply to comment #31)
> hung with my only shutdown today using the tryserver build

The patch in tryserver build did not help my bug 494014 - I just hung with high cpu (vista laptop home)
does this bug, the zero cpu hang, happen with the tryserver build? Is it possible to get a stack trace with the try server build?
Target Milestone: Thunderbird 3.0b3 → Thunderbird 3.0b4
The latest try server build doesn't solve the problem for me either (win xp). I'm a little worried about the fact that this isn't blocking beta 3.
Comment on attachment 378972 [details] [diff] [review]
alternate fix - checked in.

I've landed this, since it really seems like the right thing to do. Stack traces from hangs in tomorrow's build would be helpful, to see if they're any different from the previous stack traces.
Attachment #378972 - Attachment description: alternate fix → alternate fix - checked in.
no cpu, 2 imap connections on port 993 on XP.
Will try tomorrow's nightly
hang in XP
1 imap connection port 993
16 threads
20090527 nightly build
Comment on attachment 379960 [details]
stacks from visual studio

this is a completely different stack, and different issue, so the prev patch may have actually helped make this hang less common.

It looks to me like in the process of shutting down, someone tried to run an imap url - it failed trying to read from the socket, and tried to put up an alert about it, which requires proxying over to the ui thread. Meanwhile, the ui thread is trying to shutdown the psm thread, which seems to be blocked for some reason. 

If the IMAP thread didn't try to put up a modal alert, maybe we wouldn't have this hang. But, it might be the case that simply trying to do a proxied call to the ui thread is causing the hang, and we might still want to do something like that in the case of a network error, just to mark the server as having had a connection issue.

We're trying to get rid of the modal alerts. Until then, maybe we can be smarter about not starting urls, or putting up the alerts if we've started the shutdown process.
Thomas has determined "regression definitely started with the 20090325 build".  
Gary, stephend, nikolay, can you confirm the regression range 20090324-20090325?
Mark/bienvenu, does it make sense that this is a regression from bug 484377 - IMAP connections hanging - which landed on 2009-03-24?  (484377 is a regression of bug 466318)

My hangs are indeed less frequent with david's patch, so reclosing fixed. 

For comment 36 and 37 I filed bug 495551 hang on shutdown, zero cpu, few (1-2) imap connections - from imap url initiated during shutdown (modal alert from imap thread / proxy call to UI thread)

And I still see bug 494014 shutdown hang, high cpu, no open imap connections

(I wonder if either of these are regressions of bug 466318)  

Unsure where bug 420744 - thunderbird hangs on shutdown - is headed. It spans a rather large time period.  I'm guessing it should have been marked a regression.  Perhaps caused by bug 405440 trunk fix 2008-01-20?
Status: REOPENED → RESOLVED
Closed: 15 years ago15 years ago
Resolution: --- → FIXED
bug 405440 is not involved, I'm fairly sure. I don't think  bug 484377 is the cause of the hangs whose stack traces I've seen.

If the stack traces for the hangs don't have RetryUrl in them, then it's unlikely bug 466318 is involved.

I think the thing to do going forward is to fix the remaining issues. Some of the fixes may have exposed existing issues but I think the fixes are going in the right direction. Part of the problem is that with imap autosync we're much much more likely to be running imap urls at shutdown than we ever were before.
Sorry I haven't listed steps - I thought I had.  Steps I have used here and other hang bugs (except ldap related) was built off of bug 420744 comment 33. Not necessarily fully reduced, but fairly reliable:
1. delete several messages
then in quick succession
2. alt-f, y (empty trash)
3. alt-f, f (compact folders)
4. alt-f, x (to exit)


Attempting to understand the range of the problem, I tested pre and post autosync (autosync bug 436615 landed on 10-14).  Not definitive because I didn't check stacks and testing was limited:
* post autosync [1] - both high and zero cpu hang occur 
* pre  autosync [2] - only the high cpu hang

[1] 3 tests: 1 with 2008-10-31 and 1 with 2008-11-01 (1 was high and 1 was zero)
    5 tests: with 2008-10-15  3 high cpu
[2] 6 tests: 2008-10-14 build  4 high cpu out of 6 shutdowns 

Bug 494014 high cpu no connections definitely predates autosync. Can't say 100% certain whether this bug, no cpu, is a direct regression of autosync. But that's not what bienvenu is saying.
Whiteboard: [need info on reproducing]
Target Milestone: Thunderbird 3.0b4 → Thunderbird 3.0b3
See Also: → 495551
You need to log in before you can comment on or make changes to this bug.