677580 - Crash in google_breakpad::ExceptionHandler::WriteMinidumpOnHandlerThread at kernelbase.dll@0x10db

Status: RESOLVED WONTFIX

(Core :: General, defect)

Description

[Scoobidiver (away)]

It is #11 top browser crasher in 8.0a1.

Stack traces are various:
0 	ntdll.dll 	NtWaitForSingleObject 	
1 	KERNELBASE.dll 	KERNELBASE.dll@0x10db 	
2 	xul.dll 	std::_Vector_const_iterator<google_breakpad::ExceptionHandler*,std::allocator<google_breakpad::ExceptionHandler*> >::operator* 	vector:103
3 	ntdll.dll 	RtlEncodePointer 	
4 	xul.dll 	google_breakpad::ExceptionHandler::WriteMinidumpOnHandlerThread 	toolkit/crashreporter/google-breakpad/src/client/windows/handler/exception_handler.cc:753
5 	xul.dll 	google_breakpad::ExceptionHandler::HandleException 	toolkit/crashreporter/google-breakpad/src/client/windows/handler/exception_handler.cc:521
6 	kernel32.dll 	kernel32.dll@0x9944f 	
7 	mozcrt19.dll 	getptd_noexit 	obj-firefox/memory/jemalloc/crtsrc/tidtable.c:614
8 	ntdll.dll 	RtlDecodePointer 	
9 	ntdll.dll 	RtlAllocateMemoryBlockLookaside 	
10 	ntdll.dll 	RtlAllocateMemoryBlockLookaside 	
11 	ntdll.dll 	?? ::FNODOBFM::`string' 	
12 	ntdll.dll 	_C_specific_handler 	
13 	ntdll.dll 	RtlUserThreadStart 	
14 	ntdll.dll 	RtlpExecuteHandlerForException 	
15 	ntdll.dll 	RtlAllocateMemoryBlockLookaside
...

Frame 	Module 	Signature [Expand] 	Source
0 	ntdll.dll 	NtWaitForSingleObject 	
1 	KERNELBASE.dll 	KERNELBASE.dll@0x10db 	
2 	xul.dll 	nsComponentManagerImpl::GetServiceByContractID 	xpcom/components/nsComponentManager.cpp:1643
3 	xul.dll 	xul.dll@0x2a911f 	
4 	xul.dll 	google_breakpad::ExceptionHandler::WriteMinidumpOnHandlerThread 	toolkit/crashreporter/google-breakpad/src/client/windows/handler/exception_handler.cc:753
5 	xul.dll 	google_breakpad::ExceptionHandler::WriteMinidumpForException 	toolkit/crashreporter/google-breakpad/src/client/windows/handler/exception_handler.cc:779
6 	xul.dll 	CrashReporter::WriteMinidumpForException 	toolkit/crashreporter/nsExceptionHandler.cpp:1266
7 	ntdll.dll 	RtlAllocateHeap 	
8 	xul.dll 	mozilla::ReportException 	xpcom/base/nsCrashOnException.cpp:55
9 	msvcrt.dll 	_InternalCxxFrameHandler 	
10 	xul.dll 	CallWindowProcCrashProtected$filt$0 	xpcom/base/nsCrashOnException.cpp:67
11 	xul.dll 	CallWindowProcCrashProtected 	xpcom/base/nsCrashOnException.cpp:65
12 	mozcrt19.dll 	_C_specific_handler 	
13 	dxgi.dll 	CD3D10Resource::GetDC 	
14 	msvcrt.dll 	_CxxFrameHandler 	
15 	xul.dll 	CallWindowProcCrashProtected 	xpcom/base/nsCrashOnException.cpp:65
16 	ntdll.dll 	RtlpExecuteHandlerForException
...

Frame 	Module 	Signature [Expand] 	Source
0 	ntdll.dll 	NtWaitForSingleObject 	
1 	KERNELBASE.dll 	KERNELBASE.dll@0x10db 	
2 	xul.dll 	std::_Vector_const_iterator<google_breakpad::ExceptionHandler*,std::allocator<google_breakpad::ExceptionHandler*> >::operator* 	vector:103
3 	ntdll.dll 	RtlEncodePointer 	
4 	xul.dll 	google_breakpad::ExceptionHandler::WriteMinidumpOnHandlerThread 	toolkit/crashreporter/google-breakpad/src/client/windows/handler/exception_handler.cc:753
5 	xul.dll 	google_breakpad::ExceptionHandler::HandleException 	toolkit/crashreporter/google-breakpad/src/client/windows/handler/exception_handler.cc:521
6 	kernel32.dll 	kernel32.dll@0x9944f 	
7 	xul.dll 	GetContextFromObject 	js/src/xpconnect/src/xpcwrappedjsclass.cpp:585
8 	xul.dll 	PRMJ_Now 	js/src/prmjtime.cpp:467
9 	mozcrt19.dll 	getptd_noexit 	obj-firefox/memory/jemalloc/crtsrc/tidtable.c:614
10 	ntdll.dll 	RtlDecodePointer 	
11 	ntdll.dll 	RtlAllocateMemoryBlockLookaside 	
...

Frame 	Module 	Signature [Expand] 	Source
0 	ntdll.dll 	NtWaitForSingleObject 	
1 	KERNELBASE.dll 	KERNELBASE.dll@0x10db 	
2 	xul.dll 	nsComponentManagerImpl::GetServiceByContractID 	xpcom/components/nsComponentManager.cpp:1643
3 	xul.dll 	xul.dll@0x2a911f 	
4 	xul.dll 	google_breakpad::ExceptionHandler::WriteMinidumpOnHandlerThread 	toolkit/crashreporter/google-breakpad/src/client/windows/handler/exception_handler.cc:753
5 	xul.dll 	google_breakpad::ExceptionHandler::WriteMinidumpForException 	toolkit/crashreporter/google-breakpad/src/client/windows/handler/exception_handler.cc:779
6 	xul.dll 	CrashReporter::WriteMinidumpForException 	toolkit/crashreporter/nsExceptionHandler.cpp:1266
7 	xul.dll 	mozilla::ReportException 	xpcom/base/nsCrashOnException.cpp:55
8 	xul.dll 	CallWindowProcCrashProtected$filt$0 	xpcom/base/nsCrashOnException.cpp:67
9 	xul.dll 	CallWindowProcCrashProtected 	xpcom/base/nsCrashOnException.cpp:65
10 	mozcrt19.dll 	_C_specific_handler 	
11 	xul.dll 	CallWindowProcCrashProtected 	xpcom/base/nsCrashOnException.cpp:65
12 	ntdll.dll 	RtlpExecuteHandlerForException 	
13 	ntdll.dll 	RtlDispatchException 	
14 	igd10umd64.dll 	igd10umd64.dll@0x290093 	
15 	xul.dll 	CallWindowProcCrashProtected 	xpcom/base/nsCrashOnException.cpp:65

More reports at:
https://crash-stats.mozilla.com/report/list?signature=kernelbase.dll%400x10db

Comment 1

[(not currently active) Ted Mielczarek]

These are all Win64 crashes, and the stacks are all kind of nonsensical. We don't have real stack unwind info on Win64 currently, so these are all bogus.

Comment 2

[(not currently active) Ted Mielczarek]

I pulled one of the crashes into WinDBG and the stack looks totally different. I'm not sure why Breakpad is choking on this so badly, even the top of the stack looks different.

Comment 3

[Scoobidiver (away)]

(In reply to Ted Mielczarek [:ted, :luser] from comment #1)
> These are all Win64 crashes, and the stacks are all kind of nonsensical.
Maybe it's the 64-bit version of bug 677579 which is only 32 bits.

Comment 4

[(not currently active) Ted Mielczarek]

Interesting, there may be a Breakpad bug here as well. I can get a useful stack in WinDBG. Poking at the dump with Breakpad tools shows that Breakpad can't read the exception context for some reason, so it's just trying to read the crashing thread from the very top, which includes Breakpad writing the dump.

Comment 5

[(not currently active) Ted Mielczarek]

I would agree with that assessment. Neither of these are actual Breakpad crashes. If we were crashing in WriteMinidumpOnHandlerThread, we wouldn't get a minidump!

Comment 6

[(not currently active) Ted Mielczarek]

I found the underlying Breakpad problem with processing these dumps:
http://groups.google.com/group/google-breakpad-dev/browse_thread/thread/ead298a5f8869a94

(It's pretty technical, just thought I'd mention it.)

Comment 7

[Robert Kaiser]

Is this something we need to track for 8 or even 7?

Comment 8

[(not currently active) Ted Mielczarek]

I'm a bit worried that a Windows OS update changed some minidump writing behavior and it's causing us to lose data. I can patch Breakpad to handle these dumps server-side, which should fix the problem. I'd guess that this is just a mix of existing crashes that wind up lumped under the same signature(s) because of the Breakpad failure.

It would be interesting to see if this correlates with a specific version of DbgHelp.dll being loaded.

Comment 9

[(not currently active) Ted Mielczarek]

When Socorro 2.2.2 is rolled out to production (probably tomorrow) this should go away. (These crashes will wind up with new signatures, and they're not all the same.) We can verify this by reprocessing some of the crashes and checking the signatures/stacks.

Comment 10

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

kernelbase.dll@0x10db is the #1 crash on mozilla-central nightlies, so they're still around.

Comment 11

[Makoto Kato [:m_kato]]

(In reply to David Baron [:dbaron] from comment #10)
> kernelbase.dll@0x10db is the #1 crash on mozilla-central nightlies, so
> they're still around.

This address is generic crash when breakpad cannot walk stacks correctly.

Since breakpad symbol has no unwind information, stack walker of breakpad cannot walk crash stack correctly.

Comment 12

[(not currently active) Ted Mielczarek]

I don't think bug 683162 actually made it into production yet. These signatures should all morph into something else once that happens.

Comment 13

[Sylvestre Ledru [:Sylvestre]]

Closing because no crash reported since 12 weeks.