1407281 - Crash in shutdownhang | NtWaitForAlertByThreadId | mozilla::SpinEventLoopUntil<T>

Status: RESOLVED WONTFIX

(Core :: Networking, defect, P2)

Description

[Marcia Knous [:marcia]]

This bug was filed from the Socorro interface and is 
report bp-f169434b-5e99-49de-be01-a8b6c0171009.
=============================================================

Seen while looking at 57 beta crash stats: http://bit.ly/2kCKIVr. Seems there are two browser signatures at #11 and #31 that seem similar and don't have bugs.

Not very many comments - one about facebook and the other about streaming on Amazon. There are crashes in shutdownhang | NtWaitForAlertByThreadId | mozilla::SpinEventLoopUntil<T> all the way back to 55.0.3

ni on :nfroyd since I think he worked in this area. Perhaps he could shed some light on these hangs.

Comment 1

[Marcia Knous [:marcia]]

[@ shutdownhang | __psynch_cvwait | <name omitted> | nsThreadManager::SpinEventLoopUntil ] is another Mac signature I have seen, affects 57 and 58: http://bit.ly/2yeh11U

Comment 2

[Nathan Froyd [:froydnj]]

Note that we changed some strings on the prefix list and whatnot in bug 1402037, so we probably have a bug(s) on file for some of these already, just under different signatures.

The particular crash report from comment 0 looks like a network hang:

0 	ntdll.dll 	NtWaitForAlertByThreadId 	
1 	ntdll.dll 	RtlSleepConditionVariableCS 	
2 	kernelbase.dll 	SleepConditionVariableCS 	
3 	mozglue.dll 	mozilla::detail::ConditionVariableImpl::wait(mozilla::detail::MutexImpl&) 	mozglue/misc/ConditionVariable_windows.cpp:58
4 	xul.dll 	mozilla::CondVar::Wait(unsigned int) 	xpcom/threads/CondVar.h:68
5 	xul.dll 	mozilla::ThreadEventQueue<mozilla::PrioritizedEventQueue<mozilla::EventQueue> >::GetEvent(bool, mozilla::EventPriority*) 	xpcom/threads/ThreadEventQueue.cpp:138
6 	xul.dll 	nsThread::ProcessNextEvent(bool, bool*) 	xpcom/threads/nsThread.cpp:967
7 	xul.dll 	NS_ProcessNextEvent(nsIThread*, bool) 	xpcom/threads/nsThreadUtils.cpp:521
8 	xul.dll 	mozilla::SpinEventLoopUntil<1, <lambda_ba23c3a8cdf222e68bd8d404642209b6> >(<lambda_ba23c3a8cdf222e68bd8d404642209b6>&&, nsIThread*) 	xpcom/threads/nsThreadUtils.h:323
9 	xul.dll 	mozilla::net::nsHttpConnectionMgr::Shutdown() 	netwerk/protocol/http/nsHttpConnectionMgr.cpp:247
10 	xul.dll 	mozilla::net::nsHttpHandler::ShutdownConnectionManager() 	netwerk/protocol/http/nsHttpHandler.cpp:2751
11 	xul.dll 	mozilla::net::nsHttpHandler::Observe(nsISupports*, char const*, char16_t const*) 	netwerk/protocol/http/nsHttpHandler.cpp:2316

and another thread is also waiting on network-y things:

0 	ntdll.dll 	NtClose 	
1 	mswsock.dll 	SockCloseSocket 	
2 	mswsock.dll 	WSPCloseSocket 	
3 	ws2_32.dll 	closesocket 	
4 	nss3.dll 	SocketClose 	nsprpub/pr/src/io/prsocket.c:740
5 	nss3.dll 	ssl_DefClose 	security/nss/lib/ssl/ssldef.c:219
6 	xul.dll 	nsNSSSocketInfo::CloseSocketAndDestroy(nsNSSShutDownPreventionLock const&) 	security/manager/ssl/nsNSSIOLayer.cpp:1002
7 	xul.dll 	nsSSLIOLayerClose 	security/manager/ssl/nsNSSIOLayer.cpp:981
8 	xul.dll 	mozilla::net::nsSocketTransport::CloseSocket(PRFileDesc*, mozilla::net::nsSocketTransportService*) 	netwerk/base/nsSocketTransport2.cpp:3551
9 	xul.dll 	mozilla::net::nsSocketTransport::ReleaseFD_Locked(PRFileDesc*) 	netwerk/base/nsSocketTransport2.cpp:2039
10 	xul.dll 	mozilla::net::nsSocketTransport::OnSocketDetached(PRFileDesc*) 	netwerk/base/nsSocketTransport2.cpp:2408
11 	xul.dll 	mozilla::net::nsSocketTransportService::DetachSocket(mozilla::net::nsSocketTransportService::SocketContext*, mozilla::net::nsSocketTransportService::SocketContext*) 	netwerk/base/nsSocketTransportService2.cpp:271
12 	xul.dll 	mozilla::net::nsSocketTransportService::DoPollIteration(mozilla::BaseTimeDuration<mozilla::TimeDurationValueCalculator>*) 	netwerk/base/nsSocketTransportService2.cpp:1232

Looking at some of the other crashes in https://crash-stats.mozilla.com/signature/?signature=shutdownhang%20%7C%20NtWaitForAlertByThreadId%20%7C%20mozilla%3A%3ASpinEventLoopUntil%3CT%3E&date=%3E%3D2017-10-03T17%3A29%3A00.000Z&date=%3C2017-10-10T17%3A29%3A00.000Z&_columns=date&_columns=product&_columns=version&_columns=build_id&_columns=platform&_columns=reason&_columns=address&_columns=install_time&_sort=-date&page=1#reports

https://crash-stats.mozilla.com/report/index/aa450670-9d38-4327-93b6-fee230171010 looks like the same thing.

https://crash-stats.mozilla.com/report/index/f6c0e1a9-d1f2-4517-bc04-139ca0171010 looks like something totally different; there are a bunch of what look like non-Firefox threads running around.

https://crash-stats.mozilla.com/report/index/21ef6061-9f3b-4a88-aa20-e153c0171010 is waiting for the stream transport service, one of whose threads is still servicing requests.

https://crash-stats.mozilla.com/report/index/27251ab4-646b-4b39-b127-5668d0171010 is waiting for some kind of media shutdown in thread 38:

5 	xul.dll 	mozilla::MediaRawData::~MediaRawData() 	dom/media/MediaData.cpp:492
6 	xul.dll 	mozilla::MediaRawData::`scalar deleting destructor'(unsigned int) 	
7 	xul.dll 	mozilla::layers::CompositableClient::Release() 	obj-firefox/dist/include/mozilla/layers/CompositableClient.h:75
8 	xul.dll 	RefPtr<mozilla::AudioData>::`scalar deleting destructor'(unsigned int) 	
9 	xul.dll 	nsTArray_Impl<RefPtr<mozilla::MediaRawData>, nsTArrayInfallibleAllocator>::DestructRange(unsigned int, unsigned int) 	obj-firefox/dist/include/nsTArray.h:2012
10 	xul.dll 	nsTArray_Impl<RefPtr<mozilla::MediaRawData>, nsTArrayInfallibleAllocator>::RemoveElementsAt(unsigned int, unsigned int) 	obj-firefox/dist/include/nsTArray.h:2060
11 	xul.dll 	nsTArray_Impl<nsTArray<RefPtr<mozilla::MediaRawData> >, nsTArrayInfallibleAllocator>::RemoveElementsAt(unsigned int, unsigned int) 	obj-firefox/dist/include/nsTArray.h:2060
12 	xul.dll 	mozilla::TrackBuffersManager::TrackData::~TrackData() 	
13 	xul.dll 	mozilla::TrackBuffersManager::~TrackBuffersManager() 	dom/media/mediasource/TrackBuffersManager.cpp:118
14 	xul.dll 	RefPtr<mozilla::TrackBuffersManager>::assign_assuming_AddRef(mozilla::TrackBuffersManager*) 	obj-firefox/dist/include/mozilla/RefPtr.h:65
15 	xul.dll 	mozilla::detail::RunnableFunction<<lambda_af500ccff88ec96ce717458b890cb591> >::Run() 	obj-firefox/dist/include/nsThreadUtils.h:507

https://crash-stats.mozilla.com/report/index/128f01ec-22f9-4f50-9683-520290171010 looks like the same thing as comment 0.

https://crash-stats.mozilla.com/report/index/f64b9ba2-2937-4748-9c85-1efb20171010 looks like the same thing as comment 0.

https://crash-stats.mozilla.com/report/index/7b50ec4e-e3d0-4a63-9bc6-2c4740171010 looks like the same thing as comment 0.

https://crash-stats.mozilla.com/report/index/f78d60f3-a6b2-4a82-bab2-82a1a0171010 is another quota manager hang while waiting for IO to complete.

So out of ten reports with the same signature, that's...four distinct problems? :(

Forwarding request to jduell to ask about the networking ones; I suspect there might be a bug on file for them already.  Maybe if we're in shutdown, we could more forcibly terminate network connections?

Comment 3

[Jason Duell]

Dragana--does this look similar to any of our previous/existing bugs about shutting down HTTP connections?

Comment 4

[Dragana Damjanovic [:dragana]]

Yes, this are our regular shutdown crashes. Only the signature has changed.

They are not all network hangs, but it is hard to separate them.

Comment 5

[Andrew McCreight [:mccr8]]

Bug 1410149 will split up this signature.

Comment 6

[Sylvestre Ledru [:Sylvestre]]

Closing because no crash reported since 12 weeks.

Comment 7

[Sylvestre Ledru [:Sylvestre]]

Closing because no crash reported since 12 weeks.