Closed Bug 1267924 Opened 8 years ago Closed 4 years ago

crash in moz_abort | arena_run_split | arena_malloc_large | malloc_impl | PLDHashTable::Add | CCGraphBuilder::AddNode

Categories

(Core :: XPCOM, defect)

Product:
Core
Component:
XPCOM
Platform:
Unspecified
Windows NT
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

()

Status:
RESOLVED WORKSFORME
Tracking Flags:
Tracking Status
firefox49 --- affected
firefox52 --- wontfix
firefox-esr52 --- affected
firefox53 --- affected
firefox56 --- affected

People

(Reporter: n.nethercote, Unassigned)

References

Details

(Keywords: crash)

Crash Data

This bug was filed from the Socorro interface and is 
report bp-93f2f33b-f7b1-4be0-ab13-098892160425.
=============================================================

This signature has been around for quite some time but no bug has ever been filed, AFAICT. Recently, it has been striking on average 2 or 3 times in each Nightly build.

The stack trace is odd:

> 0 	mozglue.dll 	moz_abort 	memory/build/jemalloc_config.cpp
> 1 	mozglue.dll 	arena_run_split 	memory/mozjemalloc/jemalloc.c
> 2 	mozglue.dll 	arena_malloc_large 	memory/mozjemalloc/jemalloc.c
> 3 	mozglue.dll 	malloc_impl 	memory/build/replace_malloc.c
> 4 	xul.dll 	PLDHashTable::Add(void const*, mozilla::fallible_t const&) 	xpcom/glue/PLDHashTable.cpp
> 5 	xul.dll 	CCGraphBuilder::AddNode(void*, nsCycleCollectionParticipant*) 	xpcom/base/nsCycleCollector.cpp
> 6 	xul.dll 	CCGraphBuilder::NoteRoot(void*, nsCycleCollectionParticipant*) 	xpcom/base/nsCycleCollector.cpp

jemalloc aborts on what looks like a very ordinary malloc() call. It's not even a large malloc() request, because it's happening on the first Add() call to a fresh PLDHashTable.
mccr8, any ideas?
Flags: needinfo?(continuation)
I had a vague impression this signature was indicative of jemalloc overcommitting and running out of physical memory, but maybe that's not right. The 5 I looked at were all crashes with DOM worker CCs. In one I looked at there was another worker thread also GCing at the same time.

As you say, it is odd that we're hitting this right at the start, when the hash table wouldn't be too large.
Flags: needinfo?(continuation)
May or may not be related to bug 1229384.
I looked at a minidump in Visual Studio just in case that helped. The stack trace was identical to the one in the crash report, and I didn't learn anything new.
From the minidump of crash 960f180e-349a-493b-95b3-88dcf2160517 [1], the nbytes [2] passes to malloc is 0x080000 (512KB).

[1] https://crash-stats.mozilla.com/report/index/960f180e-349a-493b-95b3-88dcf2160517
[2] https://dxr.mozilla.org/mozilla-central/rev/f3f2fa1d7eed5a8262f6401ef18ff8117a3ce43e/xpcom/glue/PLDHashTable.cpp#539
Crash Signature: [@ moz_abort | arena_run_split | arena_malloc_large | malloc_impl | PLDHashTable::Add | CCGraphBuilder::AddNode] → [@ moz_abort | arena_run_split | arena_malloc_large | malloc_impl | PLDHashTable::Add | CCGraphBuilder::AddNode] [@ moz_abort | arena_run_split | arena_malloc_large | malloc_impl | xul.dll@0x1e25b2]
It seems for some reasons, the VirtualAlloc returns null: https://dxr.mozilla.org/mozilla-central/rev/c4449eab07d39e20ea315603f1b1863eeed7dcfe/memory/mozjemalloc/jemalloc.c#2071-2072, could it be about racing?

From the crash 960f180e-349a-493b-95b3-88dcf2160517, the main thread has stack:

  ntdll.dll!NtWaitForSingleObject ()    Unknown
  ntdll.dll!RtlpWaitOnCriticalSection()    Unknown
  ntdll.dll!RtlpEnterCriticalSectionContended ()    Unknown
  xul.dll!google_breakpad::AutoExceptionHandler::AutoExceptionHandler() Line 439    C++
  xul.dll!google_breakpad::ExceptionHandler::HandleException(_EXCEPTION_POINTERS * exinfo) Line 480    C++
  KERNELBASE.dll!UnhandledExceptionFilter ()    Unknown
  ntdll.dll!RtlUserThreadStart$filt$0()    Unknown
  ntdll.dll!__C_specific_handler ()    Unknown
  ntdll.dll!RtlpExecuteHandlerForException ()    Unknown
  ntdll.dll!RtlDispatchException()    Unknown
  ntdll.dll!KiUserExceptionDispatch ()    Unknown
  mozglue.dll!moz_abort() Line 163    C++
  mozglue.dll!pages_commit(void * addr, unsigned __int64 size) Line 2072    C
  mozglue.dll!chunk_recycle(extent_tree_t * size, extent_tree_t * alignment, unsigned __int64 base, unsigned __int64 zero, int) Line 2894    C
  mozglue.dll!chunk_alloc(unsigned __int64 size, unsigned __int64 alignment, int base, int zero) Line 2933    C
  mozglue.dll!huge_palloc(unsigned __int64 size, unsigned __int64 alignment, int zero) Line 5100    C
  mozglue.dll!malloc_impl(unsigned __int64 size) Line 151    C
  xul.dll!nsDeque::GrowCapacity() Line 155    C++
  xul.dll!GraphWalker<ScanBlackVisitor>::DoWalk(nsDeque & aQueue) Line 1472    C++
  xul.dll!GraphWalker<ScanBlackVisitor>::Walk(PtrInfo * aPi) Line 1444    C++
  xul.dll!FloodBlackNode(unsigned int & aWhiteNodeCount, bool & aFailed, PtrInfo * aPi) Line 2953    C++
  xul.dll!nsPurpleBuffer::Block::VisitEntries<PurpleScanBlackVisitor>(nsPurpleBuffer & aVisitor, PurpleScanBlackVisitor &) Line 1022    C++
  xul.dll!nsCycleCollector::ScanIncrementalRoots() Line 3061    C++
  xul.dll!nsCycleCollector::ScanRoots(bool aFullySynchGraphBuild) Line 3219    C++
  xul.dll!nsCycleCollector::Collect(ccType aCCType, js::SliceBudget & aBudget, nsICycleCollectorListener * aManualListener, bool aPreferShorterSlices) Line 3691    C++
  xul.dll!nsCycleCollector_collectSlice(js::SliceBudget & budget, bool aPreferShorterSlices) Line 4179    C++
  xul.dll!nsJSContext::RunCycleCollectorSlice() Line 1553    C++
  xul.dll!ICCTimerFired(nsITimer * aTimer, void * aClosure) Line 1609    C++
  xul.dll!nsTimerImpl::Fire() Line 525    C++
  xul.dll!nsTimerEvent::Run() Line 290    C++
  xul.dll!nsThread::ProcessNextEvent(bool aMayWait, bool * aResult) Line 1073    C++
  xul.dll!NS_ProcessNextEvent(nsIThread * aThread, bool aMayWait) Line 290    C++
  xul.dll!mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate * aDelegate) Line 131    C++
  xul.dll!MessageLoop::RunHandler() Line 227    C++
  xul.dll!MessageLoop::Run() Line 207    C++
  xul.dll!nsBaseAppShell::Run() Line 158    C++
  xul.dll!nsAppShell::Run() Line 264    C++
  xul.dll!nsAppStartup::Run() Line 285    C++
  xul.dll!XREMain::XRE_mainRun() Line 4368    C++
  xul.dll!XREMain::XRE_main(int argc, char * * argv, const nsXREAppData * aAppData) Line 4472    C++
  xul.dll!XRE_main(int argc, char * * argv, const nsXREAppData * aAppData, unsigned int aFlags) Line 4581    C++

and the worker thread has stack:

  mozglue.dll!moz_abort() Line 163    C++
  mozglue.dll!arena_run_split(arena_s * arena, arena_run_s * run, unsigned __int64 size, int large, int zero) Line 3500    C
  mozglue.dll!arena_malloc_large(arena_s * arena, unsigned __int64 size, int zero) Line 4273    C
  mozglue.dll!malloc_impl(unsigned __int64 size) Line 151    C
  xul.dll!PLDHashTable::Add(const void * aKey, const mozilla::fallible_t &) Line 539    C++
  xul.dll!CCGraphBuilder::AddNode(void * aPtr, nsCycleCollectionParticipant * aParticipant) Line 2218    C++
  xul.dll!CCGraphBuilder::NoteRoot(void * aRoot, nsCycleCollectionParticipant * aParticipant) Line 2149    C++
  xul.dll!mozilla::CycleCollectedJSRuntime::TraverseNativeRoots(nsCycleCollectionNoteRootCallback & aCb) Line 765    C++
  xul.dll!mozilla::CycleCollectedJSRuntime::TraverseRoots(nsCycleCollectionNoteRootCallback & aCb) Line 1172    C++
  xul.dll!nsCycleCollector::BeginCollection(ccType aCCType, nsICycleCollectorListener * aManualListener) Line 3875    C++
  xul.dll!nsCycleCollector::Collect(ccType aCCType, js::SliceBudget & aBudget, nsICycleCollectorListener * aManualListener, bool aPreferShorterSlices) Line 3668    C++
  xul.dll!nsCycleCollector_collect(nsICycleCollectorListener * aManualListener) Line 4163    C++
  xul.dll!`anonymous namespace'::WorkerJSRuntime::CustomGCCallback(JSGCStatus aStatus) Line 950    C++
  xul.dll!js::gc::GCRuntime::callGCCallback(JSGCStatus status) Line 1683    C++
  xul.dll!`anonymous namespace'::AutoNotifyGCActivity::~AutoNotifyGCActivity() Line 1714    C++
  xul.dll!js::gc::GCRuntime::gcCycle(bool nonincrementalByAPI, js::SliceBudget & budget, JS::gcreason::Reason reason) Line 6546    C++
  xul.dll!js::gc::GCRuntime::collect(bool nonincrementalByAPI, js::SliceBudget budget, JS::gcreason::Reason reason) Line 6635    C++
  xul.dll!js::gc::GCRuntime::gc(JSGCInvocationKind gckind, JS::gcreason::Reason reason) Line 6693    C++
  xul.dll!mozilla::dom::workers::WorkerPrivate::GarbageCollectInternal(JSContext * aCx, bool aShrinking, bool aCollectChildren) Line 6413    C++
  xul.dll!`anonymous namespace'::GarbageCollectRunnable::WorkerRun(JSContext * aCx, mozilla::dom::workers::WorkerPrivate * aWorkerPrivate) Line 1586    C++
  xul.dll!mozilla::dom::workers::WorkerRunnable::Run() Line 419    C++
The address/size passes to VirtualAlloc in the main thread (0x5ad9b00000, 0x100000), and the worker thread (0x5b20d02000, 0x80000) are not overlapped, also from minidump the target pages are in reserved state. Not sure why VirtualAlloc returns null.

BaseAddress	AllocationBase	AllocationProtect	RegionSize	State	Protect	Type
5ad9b00000	5ad9b00000	4	                100000	        2000	0	20000
5b20d02000	5b20d00000	4	                80000    	2000	0	20000
I just hit this crash when I unplugged a USB hard-drive enclosure. Three other programs crashed simultaneously, so perhaps the stack just isn't very helpful.

https://crash-stats.mozilla.com/report/index/afbbb678-0a66-4b7d-8326-a14012160606
The System Memory Use Percentage in that report is 95%. I think this is a crash when we run out of physical memory.
(In reply to Andrew McCreight [:mccr8] from comment #9)
> The System Memory Use Percentage in that report is 95%. I think this is a
> crash when we run out of physical memory.

Certainly the stack says it's an OOM, but 5% of 16GB is 800MB; that's a pretty large allocation to be doing during a GC.
¡Hola!

Just crashed like bp-28d12d08-fd41-4886-bc76-133db2170103

Windows 7 did have a warning regarding low memory when I opened the lid of this laptop.

¡Gracias!

Crashing Thread (37)
Frame 	Module 	Signature 	Source
0 	mozglue.dll 	moz_abort 	memory/build/jemalloc_config.cpp:163
1 	mozglue.dll 	arena_run_split 	memory/mozjemalloc/jemalloc.c:3507
2 	mozglue.dll 	arena_malloc_large 	memory/mozjemalloc/jemalloc.c:4279
3 	mozglue.dll 	malloc_impl 	memory/build/replace_malloc.c:151
4 	xul.dll 	PLDHashTable::Add(void const*, mozilla::fallible_t const&) 	xpcom/glue/PLDHashTable.cpp:540
5 	xul.dll 	CCGraphBuilder::AddNode(void*, nsCycleCollectionParticipant*) 	xpcom/base/nsCycleCollector.cpp:2214
6 	xul.dll 	CCGraphBuilder::NoteRoot(void*, nsCycleCollectionParticipant*) 	xpcom/base/nsCycleCollector.cpp:2143
7 	xul.dll 	mozilla::CycleCollectedJSContext::TraverseNativeRoots(nsCycleCollectionNoteRootCallback&) 	xpcom/base/CycleCollectedJSContext.cpp:779
8 	xul.dll 	mozilla::CycleCollectedJSContext::TraverseRoots(nsCycleCollectionNoteRootCallback&) 	xpcom/base/CycleCollectedJSContext.cpp:1213
9 	xul.dll 	nsCycleCollector::BeginCollection(ccType, nsICycleCollectorListener*) 	xpcom/base/nsCycleCollector.cpp:3853
10 	xul.dll 	nsCycleCollector::Collect(ccType, js::SliceBudget&, nsICycleCollectorListener*, bool) 	xpcom/base/nsCycleCollector.cpp:3651
11 	xul.dll 	nsCycleCollector_collect(nsICycleCollectorListener*) 	xpcom/base/nsCycleCollector.cpp:4144
12 	xul.dll 	`anonymous namespace'::WorkerJSContext::CustomGCCallback 	dom/workers/RuntimeService.cpp:1132
13 	xul.dll 	js::gc::GCRuntime::callGCCallback(JSGCStatus) 	js/src/jsgc.cpp:1362
14 	xul.dll 	`anonymous namespace'::AutoNotifyGCActivity::~AutoNotifyGCActivity 	js/src/jsgc.cpp:1393
15 	xul.dll 	js::gc::GCRuntime::gcCycle(bool, js::SliceBudget&, JS::gcreason::Reason) 	js/src/jsgc.cpp:6200
16 	xul.dll 	js::gc::GCRuntime::collect(bool, js::SliceBudget, JS::gcreason::Reason) 	js/src/jsgc.cpp:6337
17 	xul.dll 	js::gc::GCRuntime::gc(JSGCInvocationKind, JS::gcreason::Reason) 	js/src/jsgc.cpp:6398
18 	xul.dll 	mozilla::dom::workers::WorkerPrivate::GarbageCollectInternal(JSContext*, bool, bool) 	dom/workers/WorkerPrivate.cpp:6267
19 	xul.dll 	`anonymous namespace'::GarbageCollectRunnable::WorkerRun 	dom/workers/WorkerPrivate.cpp:1394
20 	xul.dll 	mozilla::dom::workers::WorkerRunnable::Run() 	dom/workers/WorkerRunnable.cpp:374
21 	xul.dll 	mozilla::dom::workers::WorkerPrivate::ProcessAllControlRunnablesLocked() 	dom/workers/WorkerPrivate.cpp:5055
22 	xul.dll 	mozilla::dom::workers::WorkerPrivate::DoRunLoop(JSContext*) 	dom/workers/WorkerPrivate.cpp:4576
23 	xul.dll 	`anonymous namespace'::WorkerThreadPrimaryRunnable::Run 	dom/workers/RuntimeService.cpp:2871
24 	xul.dll 	nsThread::ProcessNextEvent(bool, bool*) 	xpcom/threads/nsThread.cpp:1213
25 	xul.dll 	NS_ProcessNextEvent(nsIThread*, bool) 	xpcom/glue/nsThreadUtils.cpp:381
26 	xul.dll 	mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp:338
27 	xul.dll 	MessageLoop::RunHandler() 	ipc/chromium/src/base/message_loop.cc:225
28 	xul.dll 	MessageLoop::Run() 	ipc/chromium/src/base/message_loop.cc:205
29 	xul.dll 	nsThread::ThreadFunc(void*) 	xpcom/threads/nsThread.cpp:467
30 	nss3.dll 	PR_NativeRunThread 	nsprpub/pr/src/threads/combined/pruthr.c:397
31 	nss3.dll 	pr_root 	nsprpub/pr/src/md/windows/w95thred.c:95
32 	ucrtbase.dll 	o__realloc_base 	
33 	kernel32.dll 	BaseThreadInitThunk 	
34 	ntdll.dll 	RtlUserThreadStart
status-firefox52: --- → affected
status-firefox53: --- → affected
Crash Signature: [@ moz_abort | arena_run_split | arena_malloc_large | malloc_impl | PLDHashTable::Add | CCGraphBuilder::AddNode] [@ moz_abort | arena_run_split | arena_malloc_large | malloc_impl | xul.dll@0x1e25b2] → [@ moz_abort | arena_run_split | arena_malloc_large | malloc_impl | PLDHashTable::Add | CCGraphBuilder::AddNode] [@ moz_abort | arena_run_split | arena_malloc_large | malloc_impl | xul.dll@0x1e25b2] [@ moz_abort | arena_run_split | arena_malloc_large | …
Too late for firefox 52, mass-wontfix.
status-firefox52: affected → wontfix
234 crashes in the last week in v56, all on Windows.
me 311849ad-3ab9-4aaf-9275-8d0480171020 showed up while resuming laptop. uptime 3 days
See Also: → 1487616

Closing because no crashes reported for 12 weeks.

Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.