770805 - crash in mozilla::ipc::RPCChannel::EnteredCxxStack

Status: RESOLVED FIXED

(Core Graveyard :: Plug-ins, defect)

Description

[Scoobidiver (away)]

It's #2 top browser crasher in today's build.
It first appeared in 16.0a1/20120703110846. The regression range is:
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=e61399f31505&tochange=1d370ca5bb8d
The culprit is bug 769048.

Signature 	mozilla::ipc::RPCChannel::EnteredCxxStack() | mozilla::ipc::RPCChannel::CxxStackFrame::CxxStackFrame(mozilla::ipc::RPCChannel&, mozilla::ipc::RPCChannel::Direction, IPC::Message const*) | mozilla::plugins::PPluginInstanceParent::CallUpdateWindow() More Reports Search
UUID	61e301d6-8e29-49c3-a257-9f6792120704
Date Processed	2012-07-04 07:19:46
Uptime	1267
Last Crash	1.5 days before submission
Install Age	6.7 hours since version was first installed.
Install Time	2012-07-04 00:39:21
Product	Firefox
Version	16.0a1
Build ID	20120703110846
Release Channel	nightly
OS	Windows NT
OS Version	6.1.7601 Service Pack 1
Build Architecture	x86
Build Architecture Info	GenuineIntel family 6 model 37 stepping 5
Crash Reason	EXCEPTION_ACCESS_VIOLATION_READ
Crash Address	0x0
App Notes 	
AdapterVendorID: 0x8086, AdapterDeviceID: 0x0046, AdapterSubsysID: 04881025, AdapterDriverVersion: 8.15.10.2622
Has dual GPUs. GPU #2: AdapterVendorID2: 0x10de, AdapterDeviceID2: 0x0df4, AdapterSubsysID2: 04881025, AdapterDriverVersion2: 8.17.13.142D2D? D2D+ DWrite? DWrite+ D3D10 Layers? D3D10 Layers+ 
EMCheckCompatibility	True
Adapter Vendor ID	0x8086
Adapter Device ID	0x0046
Total Virtual Memory	4294836224
Available Virtual Memory	3400708096
System Memory Use Percentage	42
Available Page File	5826461696
Available Physical Memory	2269347840

Frame 	Module 	Signature 	Source
0 	xul.dll 	mozilla::ipc::RPCChannel::EnteredCxxStack 	obj-firefox/dist/include/mozilla/ipc/RPCChannel.h:202
1 	xul.dll 	mozilla::ipc::RPCChannel::CxxStackFrame::CxxStackFrame 	obj-firefox/dist/include/mozilla/ipc/RPCChannel.h:255
2 	xul.dll 	mozilla::ipc::RPCChannel::Call 	ipc/glue/RPCChannel.cpp:136
3 	xul.dll 	mozilla::plugins::PPluginInstanceParent::CallUpdateWindow 	obj-firefox/ipc/ipdl/PPluginInstanceParent.cpp:1050
4 	xul.dll 	nsWindow::OnPaint 	widget/windows/nsWindowGfx.cpp:225
5 	xul.dll 	nsWindow::ProcessMessage 	widget/windows/nsWindow.cpp:4727
6 	xul.dll 	nsCOMPtr_base::assign_from_qi 	obj-firefox/xpcom/build/nsCOMPtr.cpp:65
7 	xul.dll 	nsWindow::WindowProcInternal 	widget/windows/nsWindow.cpp:4314
8 	xul.dll 	CallWindowProcCrashProtected 	xpcom/base/nsCrashOnException.cpp:32
9 	xul.dll 	xul.dll@0x1d6d4f 	
10 	user32.dll 	InternalCallWinProc 	
11 	xul.dll 	CallWindowProcCrashProtected 	xpcom/base/nsCrashOnException.cpp:38
12 	xul.dll 	CallWindowProcCrashProtected 	xpcom/base/nsCrashOnException.cpp:38
13 	user32.dll 	CallWindowProcAorW 	
14 	user32.dll 	CallWindowProcW 	
15 	xul.dll 	mozilla::plugins::PluginInstanceParent::PluginWindowHookProc 	dom/plugins/ipc/PluginInstanceParent.cpp:1857
16 	user32.dll 	InternalCallWinProc 	
17 	user32.dll 	UserCallWinProcCheckWow 	
18 	user32.dll 	CallWindowProcAorW 	
19 	user32.dll 	CallWindowProcW 	
20 	xul.dll 	PluginWndProcInternal 	dom/plugins/base/nsPluginNativeWindowWin.cpp:325
21 	xul.dll 	CallWindowProcCrashProtected 	xpcom/base/nsCrashOnException.cpp:32
22 	xul.dll 	PluginWndProc 	dom/plugins/base/nsPluginNativeWindowWin.cpp:354
23 	user32.dll 	InternalCallWinProc 	
24 	user32.dll 	UserCallWinProcCheckWow 	
25 	user32.dll 	DispatchClientMessage 	
26 	user32.dll 	__fnDWORD 	
27 	ntdll.dll 	KiUserCallbackDispatcher 	
28 	ntdll.dll 	KiUserApcDispatcher 	
29 	user32.dll 	DispatchMessageW 	
30 	xul.dll 	nsAppShell::ProcessNextNativeEvent 	widget/windows/nsAppShell.cpp:322
31 	xul.dll 	nsBaseAppShell::OnProcessNextEvent 	widget/xpwidgets/nsBaseAppShell.cpp:280
...

More reports at:
https://crash-stats.mozilla.com/report/list?signature=mozilla%3A%3Aipc%3A%3ARPCChannel%3A%3AEnteredCxxStack%28%29+|+mozilla%3A%3Aipc%3A%3ARPCChannel%3A%3ACxxStackFrame%3A%3ACxxStackFrame%28mozilla%3A%3Aipc%3A%3ARPCChannel%26%2C+mozilla%3A%3Aipc%3A%3ARPCChannel%3A%3ADirection%2C+IPC%3A%3AMessage+const*%29+|+mozilla%3A%3Aplugins%3A%3APPluginInstanceParent%3A%3ACallUpdateWindow%28%29

Comment 1

[Martijn Wargers (dead)]

I get this crash with the testcase from bug 763237 using bsmedberg's special build from bug 769048.

Comment 2

[Scoobidiver (away)]

I add another signature with a stack that slightly differs and appeared at the same time:
Frame 	Module 	Signature 	Source
0 	xul.dll 	mozilla::ipc::RPCChannel::EnteredCxxStack 	obj-firefox/dist/include/mozilla/ipc/RPCChannel.h:202
1 	xul.dll 	mozilla::ipc::RPCChannel::CxxStackFrame::CxxStackFrame 	obj-firefox/dist/include/mozilla/ipc/RPCChannel.h:255
2 	xul.dll 	mozilla::ipc::RPCChannel::Call 	ipc/glue/RPCChannel.cpp:136
3 	xul.dll 	mozilla::plugins::PPluginModuleParent::CallNPP_ClearSiteData 	obj-firefox/ipc/ipdl/PPluginModuleParent.cpp:513
4 	xul.dll 	mozilla::plugins::PluginModuleParent::NPP_ClearSiteData 	dom/plugins/ipc/PluginModuleParent.cpp:919
5 	xul.dll 	nsPluginHost::ClearSiteData 	dom/plugins/base/nsPluginHost.cpp:1787
6 	xul.dll 	NS_InvokeByIndex_P 	xpcom/reflect/xptcall/src/md/win32/xptcinvoke.cpp:70
7 	xul.dll 	XPCWrappedNative::CallMethod 	js/xpconnect/src/XPCWrappedNative.cpp:2371
8 	xul.dll 	XPC_WN_CallMethod 	js/xpconnect/src/XPCWrappedNativeJSOps.cpp:1474
...

More reports at:
https://crash-stats.mozilla.com/report/list?signature=mozilla%3A%3Aipc%3A%3ARPCChannel%3A%3AEnteredCxxStack%28%29+|+mozilla%3A%3Aipc%3A%3ARPCChannel%3A%3ACxxStackFrame%3A%3ACxxStackFrame%28mozilla%3A%3Aipc%3A%3ARPCChannel%26%2C+mozilla%3A%3Aipc%3A%3ARPCChannel%3A%3ADirection%2C+IPC%3A%3AMessage+const*%29+|+mozilla%3A%3Aplugins%3A%3APPluginModuleParent%3A%3ACallNPP_ClearSiteData%28nsC...

Comment 3

[Benjamin Smedberg]

Attachment: Close the IPC channel correctly so that it reports an error, rev. 1

Although I can verify in local testing that this seems to fix the crash issues, I don't have a good way of automated testing this because the crash injector is limited to Flash subprocesses.

Comment 4

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

Comment on attachment 639419 [details] [diff] [review]
Close the IPC channel correctly so that it reports an error, rev. 1

I don't like the fact that normal shutdown vs. abnormal shutdown
causes such an (apparent) behavioral difference in
PluginInstanceParent, but this code is delicate and untested enough a
workaround is probably a good practical option.

>diff --git a/ipc/glue/AsyncChannel.h b/ipc/glue/AsyncChannel.h

>+    // Force the channel to behave as if a channel error occurred. Valid
>+    // for process links only, not thread links.
>+    void ForceError();
>+

Call this |CloseWithError()|.

r=me with that.

Comment 5

[(no longer active)]

https://hg.mozilla.org/mozilla-central/rev/27d4a1f95ac5

Comment 6

[Benjamin Smedberg]

https://hg.mozilla.org/releases/mozilla-aurora/rev/e807b189b32a (a=akeybl in bug 769048)

Comment 8

[Ioana (away)]

This crash doesn't reproduce anymore on any of the builds with the fix:
https://crash-stats.mozilla.com/report/list?query_search=signature&query_type=contains&reason_type=contains&range_value=4&range_unit=weeks&hang_type=any&process_type=any&signature=mozilla%3A%3Aipc%3A%3ARPCChannel%3A%3AEnteredCxxStack%28%29%20%7C%20mozilla%3A%3Aipc%3A%3ARPCChannel%3A%3ACxxStackFrame%3A%3ACxxStackFrame%28mozilla%3A%3Aipc%3A%3ARPCChannel%26%2C%20mozilla%3A%3Aipc%3A%3ARPCChannel%3A%3ADirection%2C%20IPC%3A%3AMessage%20const%2A%29%20%7C%20mozilla%3A%3Aplugins%3A%3APPluginInstanceParent%3A%3ACallUpdateWindow%28%29