Crash in shutdownhang | GenImageNtHeader generating minidumps

NEW
Unassigned

Status

()

P3
critical
2 years ago
5 months ago

People

(Reporter: alex_mayorga, Unassigned)

Tracking

({crash, nightly-community})

38 Branch
All
Windows
crash, nightly-community
Points:
---

Firefox Tracking Flags

(firefox-esr45 wontfix, firefox50 wontfix, firefox51 wontfix, firefox52 wontfix, firefox-esr52 wontfix, firefox53 affected, firefox54 affected, firefox55 affected)

Details

(crash signature)

(Reporter)

Description

2 years ago
This bug was filed from the Socorro interface and is 
report bp-ec240b55-4ef6-415f-9dc7-e702f2161128.
=============================================================

¡Hola!

Got this crash and found 1381 others in the past week at https://crash-stats.mozilla.com/signature/?product=Firefox&signature=shutdownhang%20%7C%20GenImageNtHeader

Filing this FWIW.

¡Gracias!
Alex


Crashing Thread (0)
Frame 	Module 	Signature 	Source
0 	dbghelp.dll 	GenImageNtHeader(void*, _IMAGE_NT_HEADERS64*) 	
1 	dbghelp.dll 	GenAllocateModuleObject(_MINIDUMP_STATE*, _INTERNAL_PROCESS*, unsigned short*, unsigned __int64, unsigned long, _INTERNAL_MODULE**)
(Reporter)

Updated

2 years ago
Keywords: nightly-community
This is hanging in Breakpad if I'm reading this correctly?
Component: General → Breakpad Integration
Product: Core → Toolkit
Version: 53 Branch → unspecified
Crash volume for signature 'shutdownhang | GenImageNtHeader':
 - nightly (version 53): 67 crashes from 2016-11-14.
 - aurora  (version 52): 161 crashes from 2016-11-14.
 - beta    (version 51): 1151 crashes from 2016-11-14.
 - release (version 50): 6621 crashes from 2016-11-01.
 - esr     (version 45): 331 crashes from 2016-07-06.

Crash volume on the last weeks (Week N is from 01-02 to 01-08):
            W. N-1  W. N-2  W. N-3  W. N-4  W. N-5  W. N-6  W. N-7
 - nightly       8       9      10       9       9      14       5
 - aurora       29      26      32      23      22      23       0
 - beta        158     173     206     236     147     135      55
 - release     877     963    1168    1081    1073     859     321
 - esr           8       5      13      18      20      11      14

Affected platform: Windows

Crash rank on the last 7 days:
           Browser   Content   Plugin
 - nightly #62
 - aurora  #34
 - beta    #69
 - release #52
 - esr     #631
status-firefox-esr45: --- → affected
(In reply to Ryan VanderMeulen [:RyanVM] from comment #1)
> This is hanging in Breakpad if I'm reading this correctly?

Seems pretty unlikely given that we got a crash report out the other end, but I'll try opening it in a debugger to see if we can get a better stack.
Here's the full stack for thread 0 from Visual C++:
 	dbghelp.dll!GenImageNtHeader(void *,struct _IMAGE_NT_HEADERS64 *)	Unknown
 	dbghelp.dll!GenAllocateModuleObject(struct _MINIDUMP_STATE *,struct _INTERNAL_PROCESS *,unsigned short *,unsigned __int64,unsigned long,struct _INTERNAL_MODULE * *)	Unknown
 	dbghelp.dll!GenGetProcessInfo(struct _MINIDUMP_STATE *,struct _INTERNAL_PROCESS * *)	Unknown
 	dbghelp.dll!MiniDumpProvideDump()	Unknown
 	dbghelp.dll!MiniDumpWriteDump()	Unknown
>	xul.dll!google_breakpad::ExceptionHandler::WriteMinidumpWithExceptionForProcess(unsigned long requesting_thread_id, _EXCEPTION_POINTERS * exinfo, MDRawAssertionInfo * assertion, void * process, bool write_requester_stream) Line 1016	C++
 	xul.dll!google_breakpad::ExceptionHandler::WriteMinidumpForChild(void * child, unsigned long child_blamed_thread, const std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > & dump_path, bool(*)(const wchar_t *, const wchar_t *, void *, _EXCEPTION_POINTERS *, MDRawAssertionInfo *, bool) callback, void * callback_context, _MINIDUMP_TYPE dump_type) Line 810	C++
 	xul.dll!CrashReporter::CreateMinidumpsAndPair(void * aTargetPid, unsigned long aTargetBlamedThread, const nsACString_internal & aIncomingPairName, nsIFile * aIncomingDumpToPair, nsIFile * * aMainDumpOut) Line 4005	C++
 	xul.dll!mozilla::dom::CrashReporterParent::GenerateCompleteMinidump<mozilla::dom::ContentParent>(mozilla::dom::ContentParent * t) Line 273	C++
 	xul.dll!mozilla::dom::ContentParent::KillHard(const char * aReason) Line 2732	C++
 	xul.dll!mozilla::dom::ContentParent::ForceKillTimerCallback(nsITimer * aTimer, void * aClosure) Line 2690	C++
 	xul.dll!nsTimerImpl::Fire() Line 478	C++
 	xul.dll!nsTimerEvent::Run() Line 293	C++
 	xul.dll!nsThread::ProcessNextEvent(bool aMayWait, bool * aResult) Line 1213	C++
 	xul.dll!nsThread::Shutdown() Line 982	C++
 	xul.dll!mozilla::DataStorage::Observe(nsISupports * aSubject, const char * aTopic, const char16_t * aData) Line 886	C++
 	xul.dll!nsObserverList::NotifyObservers(nsISupports * aSubject, const char * aTopic, const char16_t * someData) Line 112	C++
 	xul.dll!nsObserverService::NotifyObservers(nsISupports * aSubject, const char * aTopic, const char16_t * aSomeData) Line 285	C++
 	xul.dll!nsXREDirProvider::DoShutdown() Line 1251	C++
 	xul.dll!ScopedXPCOMStartup::~ScopedXPCOMStartup() Line 1410	C++
 	xul.dll!mozilla::UniquePtr<ScopedXPCOMStartup,mozilla::DefaultDelete<ScopedXPCOMStartup> >::reset(ScopedXPCOMStartup * aPtr) Line 343	C++
 	xul.dll!XREMain::XRE_main(int argc, char * * argv, const nsXREAppData * aAppData) Line 4644	C++
 	xul.dll!XRE_main(int argc, char * * argv, const nsXREAppData * aAppData, unsigned int aFlags) Line 4705	C++
 	firefox.exe!do_main(int argc, char * * argv, char * * envp, nsIFile * xreDirectory) Line 328	C++
 	firefox.exe!wmain(int argc, wchar_t * * argv) Line 115	C++
 	firefox.exe!__scrt_common_main_seh() Line 253	C++
 	kernel32.dll!BaseThreadInitThunk()	Unknown
 	ntdll.dll!RtlUserThreadStart()	Unknown

It looks like DataStorage is waiting for its worker thread to shut down, spinning the event loop while it does so. Then things got a little weird, in that we hit `ContentParent::ForceKillTimerCallback`, so we decided to kill the content process for not responding, and then while that was happening we also hit the shutdown timeout and killed the chrome process, so we wound up writing a minidump of the chrome process while it was writing a minidump of the content process.

I think the specific shutdownhang here belongs to storage, but I'm also going to file another bug about the weird interaction of the shutdown timer and the content process hang timer.
Component: Breakpad Integration → Storage
Crash volume for signature 'shutdownhang | GenImageNtHeader':
 - nightly (version 54): 17 crashes from 2017-01-23.
 - aurora  (version 53): 33 crashes from 2017-01-23.
 - beta    (version 52): 367 crashes from 2017-01-23.
 - release (version 51): 2101 crashes from 2017-01-16.
 - esr     (version 45): 320 crashes from 2016-08-10.

Crash volume on the last weeks (Week N is from 02-06 to 02-12):
            W. N-1  W. N-2  W. N-3  W. N-4  W. N-5  W. N-6  W. N-7
 - nightly      10       3
 - aurora       21       3
 - beta        201      83
 - release    1342     321       0
 - esr          13      17      13      10      12       8       5

Affected platform: Windows

Crash rank on the last 7 days:
           Browser   Content   Plugin
 - nightly #80
 - aurora  #30
 - beta    #51
 - release #22
 - esr     #703
status-firefox54: --- → affected
Too late for firefox 52, mass-wontfix.
status-firefox52: affected → wontfix
Win7, 64bit, FF55.0a1
https://crash-stats.mozilla.com/report/index/5f6c9b55-c1e9-40f6-aecd-955220170501

Data for bug edit from:
https://crash-stats.mozilla.com/signature/?product=Firefox&signature=shutdownhang%20%7C%20GenImageNtHeader

Architecture last 7 days:
x86 	2190 	96.7%
amd64 	75 	3.3%
status-firefox50: affected → wontfix
status-firefox51: affected → wontfix
status-firefox55: --- → affected
status-firefox-esr45: affected → wontfix
status-firefox-esr52: --- → wontfix
OS: Windows 7 → Windows
Hardware: x86 → All
Version: unspecified → 38 Branch
FF55.0a1, Build-ID 20170611030208, 64-bit, E10s, Win7:
https://crash-stats.mozilla.com/report/index/0d086a2e-1ee0-4330-8049-48eed0170611

Crashing Thread
Frame 	Module 	Signature 	Source
0 	dbghelp.dll 	GenImageNtHeader(void*, _IMAGE_NT_HEADERS64*) 	
1 	dbghelp.dll 	GenAllocateModuleObject(_MINIDUMP_STATE*, _INTERNAL_PROCESS*, unsigned short*, unsigned __int64, unsigned long, _INTERNAL_MODULE**)
DataStorage is unrelated to Storage. all the latest bugs touching it seem to point to Core / Security PSM
Component: Storage → Security: PSM
Product: Toolkit → Core
Recent crash reports don't seem to be specific to DataStorage (or involve it at all). Maybe general IPC issue?
Component: Security: PSM → IPC

Comment 12

5 months ago
This is a shutdown hang while generating crash dumps.

"Shutdown too long, probably frozen, causing a crash"
Priority: -- → P3
Summary: Crash in shutdownhang | GenImageNtHeader → Crash in shutdownhang | GenImageNtHeader generating minidumps

Comment 13

5 months ago
I received this crash signature while shutting down Firefox (52-ESR).

Report ID 	Date Submitted
bp-dd7a77e8-a087-47af-b301-98c210180327
	3/27/2018	5:24 PM
bp-08afa438-ab4f-4c86-b0a3-46c5d0180327
	3/27/2018	5:24 PM
bp-dd7a8d45-8e35-4f9c-bca6-3e87f0180327
	3/27/2018	5:22 PM

Firefox 52.6.0esr Crash Report [@ shutdownhang | GenImageNtHeader ]
ID: dd7a8d45-8e35-4f9c-bca6-3e87f0180327

MOZ_CRASH Reason 	MOZ_CRASH(Shutdown too long, probably frozen, causing a crash.)
Crash Reason 	EXCEPTION_BREAKPOINT
Crash Address 	0x5e819ee0
User Comments: Firefox crashed while shutting down. 

Total Virtual Memory 	4,294,836,224 bytes (4.29 GB)
Available Virtual Memory 	3,468,918,784 bytes (3.47 GB)
Available Page File 	957,706,240 bytes (957.71 MB)
Available Physical Memory 	465,629,184 bytes (465.63 MB)
System Memory Use Percentage 	88 

Crashing Thread (0)
Frame 	Module 	Signature 	Source
0 	dbghelp.dll 	GenImageNtHeader(void*, _IMAGE_NT_HEADERS64*) 	
1 	dbghelp.dll 	GenAllocateModuleObject(_MINIDUMP_STATE*, _INTERNAL_PROCESS*, unsigned short*, unsigned __int64, unsigned long, _INTERNAL_MODULE**) 	
2 	dbghelp.dll 	GenGetProcessInfo(_MINIDUMP_STATE*, _INTERNAL_PROCESS**) 	
3 	dbghelp.dll 	MiniDumpProvideDump 	
4 	dbghelp.dll 	MiniDumpWriteDump 	
5 	xul.dll 	google_breakpad::ExceptionHandler::WriteMinidumpWithExceptionForProcess(unsigned long, _EXCEPTION_POINTERS*, MDRawAssertionInfo*, void*, bool)


Firefox 52.6.0esr Crash Report [@ shutdownhang | GenImageNtHeader ]
ID: 08afa438-ab4f-4c86-b0a3-46c5d0180327

MOZ_CRASH Reason 	MOZ_CRASH(Shutdown too long, probably frozen, causing a crash.)
Crash Reason 	EXCEPTION_BREAKPOINT
Crash Address 	0x5e819ee0
User Comments 	

Crashing Thread (0)
Frame 	Module 	Signature 	Source
0 	dbghelp.dll 	GenImageNtHeader(void*, _IMAGE_NT_HEADERS64*) 	
1 	dbghelp.dll 	GenAllocateModuleObject(_MINIDUMP_STATE*, _INTERNAL_PROCESS*, unsigned short*, unsigned __int64, unsigned long, _INTERNAL_MODULE**) 	
2 	dbghelp.dll 	GenGetProcessInfo(_MINIDUMP_STATE*, _INTERNAL_PROCESS**) 	
3 	dbghelp.dll 	MiniDumpProvideDump 	
4 	dbghelp.dll 	MiniDumpWriteDump 	
5 	xul.dll 	google_breakpad::ExceptionHandler::WriteMinidumpWithExceptionForProcess(unsigned long, _EXCEPTION_POINTERS*, MDRawAssertionInfo*, void*, bool)


Firefox 52.6.0esr Crash Report [@ IPCError-browser | ShutDownKill ]
ID: dd7a77e8-a087-47af-b301-98c210180327

Process Type 	content (web)
Crash Reason 	EXCEPTION_BREAKPOINT
Crash Address 	0x71cedaa3

Processor Notes 	
processor_prod-processor-i-0b0c48fc9cc57b96e_1406; Processor2015; 
non-integer value of "SecondsSinceLastCrash"; 
Signature replaced with an IPC Channel Error, was: "memset | arena_dalloc_small | je_free | js::SharedScriptData::decRefCount"

Crashing Thread (0)
Frame 	Module 	Signature 	Source
0 	vcruntime140.dll 	memset 	f:\dd\vctools\crt\vcruntime\src\string\i386\memset.asm:113
1 	mozglue.dll 	arena_dalloc_small 	memory/mozjemalloc/jemalloc.c:4598
2 	mozglue.dll 	je_free 	memory/mozjemalloc/jemalloc.c:6468
3 	xul.dll 	js::SharedScriptData::decRefCount() 	js/src/jsscript.h:718
4 	xul.dll 	js::SweepScriptData(JSRuntime*, js::AutoLockForExclusiveAccess&) 	js/src/jsscript.cpp:2249
You need to log in before you can comment on or make changes to this bug.