904474 - crash in nsDocLoader::DoFireOnStateChange @ GetContextFromObjectOrDefault firing nsHttpChannel::OnStopRequest on a non-main thread

Status: RESOLVED INACTIVE

(Core :: XPConnect, defect)

Description

[Scoobidiver (away)]

It first showed up in 25.0a1/20130710. The regression range is:
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=c0830a5933e8&tochange=04d8c309fe72

Signature 	GetContextFromObjectOrDefault More Reports Search
UUID 	d7522ad0-b091-4246-890a-44f212130811
Date Processed	2013-08-11 16:47:03.505780
Uptime	887
Install Age 	887 since version was first installed.
Install Time 	2013-08-10 05:28:10
Product 	Firefox
Version 	26.0a1
Build ID 	20130811030225
Release Channel 	nightly
OS 	Windows NT
OS Version 	6.1.7601 Service Pack 1
Build Architecture 	x86
Build Architecture Info 	GenuineIntel family 15 model 6 stepping 5 | 2
Crash Reason 	EXCEPTION_BREAKPOINT
Crash Address 	0x6053c8da
App Notes 	
AdapterVendorID: 0x8086, AdapterDeviceID: 0x2772, AdapterSubsysID: 00000000, AdapterDriverVersion: 8.15.10.1930
D3D10 Layers? D3D10 Layers- D3D9 Layers? D3D9 Layers- 

Frame 	Module 	Signature 	Source
0 	xul.dll 	GetContextFromObjectOrDefault 	js/xpconnect/src/XPCWrappedJSClass.cpp
1 	xul.dll 	nsXPCWrappedJSClass::DelegatedQueryInterface(nsXPCWrappedJS *,nsID const &,void * *) 	js/xpconnect/src/XPCWrappedJSClass.cpp
2 	xul.dll 	nsXPCWrappedJS::QueryInterface(nsID const &,void * *) 	js/xpconnect/src/XPCWrappedJS.cpp
3 	xul.dll 	nsCOMPtr_base::assign_from_helper(nsCOMPtr_helper const &,nsID const &) 	obj-firefox/xpcom/build/nsCOMPtr.cpp
4 	xul.dll 	nsDocLoader::DoFireOnStateChange(nsIWebProgress * const,nsIRequest * const,int &,tag_nsresult) 	uriloader/base/nsDocLoader.cpp
5 	xul.dll 	nsDocLoader::FireOnStateChange(nsIWebProgress *,nsIRequest *,int,tag_nsresult) 	uriloader/base/nsDocLoader.cpp
6 	xul.dll 	nsDocLoader::OnStopRequest(nsIRequest *,nsISupports *,tag_nsresult) 	uriloader/base/nsDocLoader.cpp
7 	xul.dll 	nsLoadGroup::RemoveRequest(nsIRequest *,nsISupports *,tag_nsresult) 	netwerk/base/src/nsLoadGroup.cpp
8 	xul.dll 	mozilla::net::nsHttpChannel::OnStopRequest(nsIRequest *,nsISupports *,tag_nsresult) 	netwerk/protocol/http/nsHttpChannel.cpp
9 	xul.dll 	nsInputStreamPump::OnStateStop() 	netwerk/base/src/nsInputStreamPump.cpp
10 	xul.dll 	nsInputStreamPump::OnInputStreamReady(nsIAsyncInputStream *) 	netwerk/base/src/nsInputStreamPump.cpp
11 	xul.dll 	nsInputStreamReadyEvent::Run() 	xpcom/io/nsStreamUtils.cpp
12 	xul.dll 	nsThread::ProcessNextEvent(bool,bool *) 	xpcom/threads/nsThread.cpp
13 	xul.dll 	nsThread::ThreadFunc(void *) 	xpcom/threads/nsThread.cpp
14 	nss3.dll 	_PR_NativeRunThread 	nsprpub/pr/src/threads/combined/pruthr.c
15 	nss3.dll 	pr_root 	nsprpub/pr/src/md/windows/w95thred.c
16 	msvcr100.dll 	_callthreadstartex 	f:\\dd\\vctools\\crt_bld\\self_x86\\crt\\src\\threadex.c
17 	msvcr100.dll 	_threadstartex 	f:\\dd\\vctools\\crt_bld\\self_x86\\crt\\src\\threadex.c
18 	kernel32.dll 	BaseThreadInitThunk 	
19 	ntdll.dll 	__RtlUserThreadStart 	
20 	ntdll.dll 	_RtlUserThreadStart

More reports at:
https://crash-stats.mozilla.com/report/list?product=Firefox&signature=GetContextFromObjectOrDefault

Comment 1

[Josh Matthews [:jdm]]

This stack shows us firing nsHttpChannel::OnStopRequest on a non-main thread. Is that expected as part of the off-main-thread-ODA changes? Can't we end up triggering arbitrary listeners in this case? That's bad, because we can't use JS components off the main thread, which is presumably what is triggering the problem shown in this stack.

Comment 2

[Jason Duell]

It could be a result of the off-main-thread OnDataAvaiable changes, but IIRC even when that's used we should be delivering OnStop to the main thread.  Steve, can you take a look?

Comment 3

[Steve Workman [:sworkman] (INACTIVE)]

I'll take a look. And Jason is right - it *shouldn't* be delivering OnStopRequest off main thread.

Comment 4

[Steve Workman [:sworkman] (INACTIVE)]

Took a look: I don't see anything from the report to help determine a set of steps to reproduce. I started setting up my machine to look at the minidump from one of the crashes, but socorro won't let me login to get access to it - probably a reasonable response from socorro for privacy and security. Nonetheless, how do I go about getting access to the minidump to examine what was going on in more detail?

Scoobidiver - any ideas?

Comment 5

[Josh Matthews [:jdm]]

File a bug under Account Requests. See bug 857222 for an example.

Comment 6

[Steve Workman [:sworkman] (INACTIVE)]

Thanks jdm! Bug 909517.

Comment 8

[Steve Workman [:sworkman] (INACTIVE)]

Got a minidump (from this report - https://crash-stats.mozilla.com/report/index/3fe0ea39-ae08-4744-b25b-e13672130826) and tried looking at more details in WinDbg. Alas, any variables that would provide more useful data are on the heap (nsInputStreamPump's member vars), and there is no heap memory in minidumps (https://developer.mozilla.org/en-US/docs/Debugging_a_minidump). So, no further forward.

For now, Bug 913151 "Always call nsInputStreamPump::OnStateTransfer on the main thread" is the best I can offer. A specific root cause is unknown, but this bandaid should handle the error case well enough to stop crashes and keep FF running.

There is another fix in the pipeline, however, that makes each iteration of the for loop in nsInputStreamPump::OnInputStreamReady more atomic. This was added to stop similar error cases for imagelib code, but the scenario seemed to require more than 2 threads, specifically, nsInputStreamPump::mTargetThread needed to be a threadpool. In this bug, mTargetThread is a single thread. So, this patch would be a speculative fix at best for the specific crash documented in this bug.

Comment 9

[BMO Automation]

Per policy at https://wiki.mozilla.org/Bug_Triage/Projects/Bug_Handling/Bug_Husbandry#Inactive_Bugs. If this bug is not an enhancement request or a bug not present in a supported release of Firefox, then it may be reopened.