702812 - Nightly hangs on startup, jemalloc in backtrace

Status: RESOLVED FIXED

(Core :: Memory Allocator, defect)

Description

[Doug Sherk (:drs) (inactive)]

I see the browser chrome and nightly starts, but it hangs apparently as soon as it starts loading the default nightly page. This started sometime between 3 days and 1 day ago. The page loads but it takes forever (i.e. 5 minutes, ish) and it is still unresponsive afterwards.

With NO_MAC_JEMALLOC=1, it is as responsive as I'd expect (with slightly slower startup time it seems, although that may have just been random).

Here's a bt that I got by stopping it while it was stalled. I tried multiple times and it always looked either very similar or identical to this.

#0  0x00007fff839a97e0 in memset_pattern4 ()
#1  0x00007fff839a967b in memset ()
#2  0x000000010002e174 in huge_dalloc (ptr=0x129d00000) at /Users/dsherk/builds/mozilla-central8/memory/jemalloc/jemalloc.c:5218
#3  0x000000010002c3c9 in idalloc (ptr=0x129d00000) at /Users/dsherk/builds/mozilla-central8/memory/jemalloc/jemalloc.c:4554
#4  0x000000010002df7d in huge_ralloc (ptr=0x129d00000, size=134217728, oldsize=67108864) at /Users/dsherk/builds/mozilla-central8/memory/jemalloc/jemalloc.c:5190
#5  0x000000010002d132 in iralloc (ptr=0x129d00000, size=134217728) at /Users/dsherk/builds/mozilla-central8/memory/jemalloc/jemalloc.c:4747
#6  0x000000010003010b in je_realloc (ptr=0x129d00000, size=134217728) at /Users/dsherk/builds/mozilla-central8/memory/jemalloc/jemalloc.c:6439
#7  0x0000000100030e7a in ozone_realloc (zone=0x10027f000, ptr=0x129d00000, size=134217728) at /Users/dsherk/builds/mozilla-central8/memory/jemalloc/jemalloc.c:6916
#8  0x00007fff839a8243 in malloc_zone_realloc ()
#9  0x00007fff839a9032 in realloc ()
#10 0x0000000103106db7 in js_realloc (p=0x129d00000, bytes=134217728) at Utility.h:178
#11 0x0000000103117182 in JSRuntime::realloc_ (this=0x109923000, p=0x129d00000, oldBytes=67108864, newBytes=134217728, cx=0x107df6f20) at jscntxt.h:759
#12 0x00000001031171f8 in JSContext::realloc_ (this=0x107df6f20, p=0x129d00000, oldBytes=67108864, newBytes=134217728) at jscntxt.h:1266
#13 0x0000000102fc8c54 in js::ContextAllocPolicy::realloc_ (this=0x7fff5fbf8fd0, p=0x129d00000, oldBytes=67108864, bytes=134217728) at jscntxt.h:2464
#14 0x0000000103117264 in js::VectorImpl<unsigned short, 32ul, js::ContextAllocPolicy, true>::growTo (v=@0x7fff5fbf8fd0, newcap=67108864) at Vector.h:179
#15 0x0000000102fc8f46 in js::Vector<unsigned short, 32ul, js::ContextAllocPolicy>::growHeapStorageBy (this=0x7fff5fbf8fd0, lengthInc=1) at Vector.h:599
#16 0x00000001030b571f in js::Vector<unsigned short, 32ul, js::ContextAllocPolicy>::growStorageBy (this=0x7fff5fbf8fd0, incr=1) at Vector.h:639
#17 0x0000000102fc935a in js::Vector<unsigned short, 32ul, js::ContextAllocPolicy>::append<unsigned short> (this=0x7fff5fbf8fd0, t=92) at Vector.h:759
#18 0x0000000102fc94d3 in js::StringBuffer::append (this=0x7fff5fbf8fd0, c=92) at jsstrinlines.h:151
#19 0x00000001030b0de8 in Quote (cx=0x107df6f20, sb=@0x7fff5fbf8fd0, str=0x109fb5020) at /Users/dsherk/builds/mozilla-central8/js/src/json.cpp:182
#20 0x00000001030b176f in Str (cx=0x107df6f20, v=@0x7fff5fbf7d90, scx=0x7fff5fbf8cc0) at /Users/dsherk/builds/mozilla-central8/js/src/json.cpp:577
#21 0x00000001030b155c in JO (cx=0x107df6f20, obj=0x10a463310, scx=0x7fff5fbf8cc0) at /Users/dsherk/builds/mozilla-central8/js/src/json.cpp:468
#22 0x00000001030b19fc in Str (cx=0x107df6f20, v=@0x7fff5fbf7fc0, scx=0x7fff5fbf8cc0) at /Users/dsherk/builds/mozilla-central8/js/src/json.cpp:610
#23 0x00000001030b155c in JO (cx=0x107df6f20, obj=0x10a4632c8, scx=0x7fff5fbf8cc0) at /Users/dsherk/builds/mozilla-central8/js/src/json.cpp:468
#24 0x00000001030b19fc in Str (cx=0x107df6f20, v=@0x7fff5fbf81e0, scx=0x7fff5fbf8cc0) at /Users/dsherk/builds/mozilla-central8/js/src/json.cpp:610
#25 0x00000001030b2679 in JA (cx=0x107df6f20, obj=0x109f371a8, scx=0x7fff5fbf8cc0) at /Users/dsherk/builds/mozilla-central8/js/src/json.cpp:533
#26 0x00000001030b19da in Str (cx=0x107df6f20, v=@0x7fff5fbf8340, scx=0x7fff5fbf8cc0) at /Users/dsherk/builds/mozilla-central8/js/src/json.cpp:608
#27 0x00000001030b155c in JO (cx=0x107df6f20, obj=0x10a463280, scx=0x7fff5fbf8cc0) at /Users/dsherk/builds/mozilla-central8/js/src/json.cpp:468
#28 0x00000001030b19fc in Str (cx=0x107df6f20, v=@0x7fff5fbf8560, scx=0x7fff5fbf8cc0) at /Users/dsherk/builds/mozilla-central8/js/src/json.cpp:610
#29 0x00000001030b2679 in JA (cx=0x107df6f20, obj=0x109f37120, scx=0x7fff5fbf8cc0) at /Users/dsherk/builds/mozilla-central8/js/src/json.cpp:533
#30 0x00000001030b19da in Str (cx=0x107df6f20, v=@0x7fff5fbf86c0, scx=0x7fff5fbf8cc0) at /Users/dsherk/builds/mozilla-central8/js/src/json.cpp:608
#31 0x00000001030b155c in JO (cx=0x107df6f20, obj=0x10a4631f0, scx=0x7fff5fbf8cc0) at /Users/dsherk/builds/mozilla-central8/js/src/json.cpp:468
#32 0x00000001030b19fc in Str (cx=0x107df6f20, v=@0x7fff5fbf88e0, scx=0x7fff5fbf8cc0) at /Users/dsherk/builds/mozilla-central8/js/src/json.cpp:610
#33 0x00000001030b2679 in JA (cx=0x107df6f20, obj=0x109f37098, scx=0x7fff5fbf8cc0) at /Users/dsherk/builds/mozilla-central8/js/src/json.cpp:533
#34 0x00000001030b19da in Str (cx=0x107df6f20, v=@0x7fff5fbf8a40, scx=0x7fff5fbf8cc0) at /Users/dsherk/builds/mozilla-central8/js/src/json.cpp:608
#35 0x00000001030b155c in JO (cx=0x107df6f20, obj=0x10a463118, scx=0x7fff5fbf8cc0) at /Users/dsherk/builds/mozilla-central8/js/src/json.cpp:468
#36 0x00000001030b19fc in Str (cx=0x107df6f20, v=@0x109a83308, scx=0x7fff5fbf8cc0) at /Users/dsherk/builds/mozilla-central8/js/src/json.cpp:610
#37 0x00000001030b2446 in js_Stringify (cx=0x107df6f20, vp=0x109a83308, replacer=0x0, space={data = {asBits = 18444773748872577024, debugView = {payload47 = 0, tag = JSVAL_TAG_UNDEFINED}, s = {payload = {i32 = 0, u32 = 0, why = JS_ARRAY_HOLE}}, asDouble = -nan(0x9000000000000), asPtr = 0xfff9000000000000, asWord = 18444773748872577024}}, sb=@0x7fff5fbf8fd0) at /Users/dsherk/builds/mozilla-central8/js/src/json.cpp:773
#38 0x00000001030b2885 in js_json_stringify (cx=0x107df6f20, argc=1, vp=0x109a83308) at /Users/dsherk/builds/mozilla-central8/js/src/json.cpp:123
#39 0x0000000103078096 in js::CallJSNative (cx=0x107df6f20, native=0x1030b2764 <js_json_stringify(JSContext*, unsigned int, JS::Value*)>, args=@0x7fff5fbf91a0) at jscntxtinlines.h:297
#40 0x000000010307512b in js::InvokeKernel (cx=0x107df6f20, args={<js::CallReceiver> = {usedRval_ = false, argv_ = 0x109a83318}, argc_ = 1}, construct=js::NO_CONSTRUCT) at /Users/dsherk/builds/mozilla-central8/js/src/jsinterp.cpp:629
#41 0x0000000103067aef in js::Interpret (cx=0x107df6f20, entryFrame=0x109a831f8, interpMode=js::JSINTERP_NORMAL) at /Users/dsherk/builds/mozilla-central8/js/src/jsinterp.cpp:3950
#42 0x0000000103074adb in js::RunScript (cx=0x107df6f20, script=0x11d1e4cc0, fp=0x109a831f8) at /Users/dsherk/builds/mozilla-central8/js/src/jsinterp.cpp:584
#43 0x000000010307520a in js::InvokeKernel (cx=0x107df6f20, args={<js::CallReceiver> = {usedRval_ = false, argv_ = 0x109a831f0}, argc_ = 1}, construct=js::NO_CONSTRUCT) at /Users/dsherk/builds/mozilla-central8/js/src/jsinterp.cpp:647
#44 0x000000010307906f in js::Invoke (cx=0x107df6f20, args=@0x7fff5fbfa5d0, construct=js::NO_CONSTRUCT) at jsinterp.h:148
#45 0x000000010307562a in js::Invoke (cx=0x107df6f20, thisv=@0x7fff5fbfa6a0, fval=@0x7fff5fbfa668, argc=1, argv=0x7fff5fbfad80, rval=0x7fff5fbfaaa0) at /Users/dsherk/builds/mozilla-central8/js/src/jsinterp.cpp:679
#46 0x0000000102f927e0 in JS_CallFunctionValue (cx=0x107df6f20, obj=0x11d1d6bf0, fval={data = {asBits = 18445477441097826040, debugView = {payload47 = 4783472376, tag = JSVAL_TAG_OBJECT}, s = {payload = {i32 = 488505080, u32 = 488505080, why = 488505080}}, asDouble = -nan(0xb80011d1dfef8), asPtr = 0xfffb80011d1dfef8, asWord = 18445477441097826040}}, argc=1, argv=0x7fff5fbfad80, rval=0x7fff5fbfaaa0) at /Users/dsherk/builds/mozilla-central8/js/src/jsapi.cpp:5221
#47 0x0000000102357007 in nsXPCWrappedJSClass::CallMethod (this=0x120a3cfb0, wrapper=0x125a66e80, methodIndex=3, info=0x107cefc10, nativeParams=0x7fff5fbfaed0) at /Users/dsherk/builds/mozilla-central8/js/xpconnect/src/XPCWrappedJSClass.cpp:1530
#48 0x000000010234dbf6 in nsXPCWrappedJS::CallMethod (this=0x125a66e80, methodIndex=3, info=0x107cefc10, params=0x7fff5fbfaed0) at /Users/dsherk/builds/mozilla-central8/js/xpconnect/src/XPCWrappedJS.cpp:611
#49 0x0000000102aea3e4 in PrepareAndDispatch (self=0x120a57920, methodIndex=3, args=0x7fff5fbfb050, gpregs=0x7fff5fbfafd0, fpregs=0x7fff5fbfb000) at /Users/dsherk/builds/mozilla-central8/xpcom/reflect/xptcall/src/md/unix/xptcstubs_x86_64_darwin.cpp:153
#50 0x0000000102ae8e43 in SharedStub () at xpt_struct.h:334
#51 0x0000000102ae8d91 in NS_InvokeByIndex_P (that=0x120a57920, methodIndex=3, paramCount=1, params=0x7fff5fbfb300) at /Users/dsherk/builds/mozilla-central8/xpcom/reflect/xptcall/src/md/unix/xptcinvoke_x86_64_unix.cpp:195
#52 0x0000000102366e48 in CallMethodHelper::Invoke (this=0x7fff5fbfb2c0) at /Users/dsherk/builds/mozilla-central8/js/xpconnect/src/XPCWrappedNative.cpp:2927
#53 0x000000010236999f in CallMethodHelper::Call (this=0x7fff5fbfb2c0) at /Users/dsherk/builds/mozilla-central8/js/xpconnect/src/XPCWrappedNative.cpp:2234
#54 0x0000000102362fce in XPCWrappedNative::CallMethod (ccx=@0x7fff5fbfb460, mode=XPCWrappedNative::CALL_METHOD) at /Users/dsherk/builds/mozilla-central8/js/xpconnect/src/XPCWrappedNative.cpp:2200
#55 0x000000010236f1a1 in XPC_WN_CallMethod (cx=0x107df6f20, argc=1, vp=0x109a831c8) at /Users/dsherk/builds/mozilla-central8/js/xpconnect/src/XPCWrappedNativeJSOps.cpp:1554
#56 0x0000000103078096 in js::CallJSNative (cx=0x107df6f20, native=0x10236ef49 <XPC_WN_CallMethod(JSContext*, unsigned int, JS::Value*)>, args=@0x7fff5fbfb6b0) at jscntxtinlines.h:297
#57 0x000000010307512b in js::InvokeKernel (cx=0x107df6f20, args={<js::CallReceiver> = {usedRval_ = false, argv_ = 0x109a831d8}, argc_ = 1}, construct=js::NO_CONSTRUCT) at /Users/dsherk/builds/mozilla-central8/js/src/jsinterp.cpp:629
#58 0x0000000103067aef in js::Interpret (cx=0x107df6f20, entryFrame=0x109a830d0, interpMode=js::JSINTERP_NORMAL) at /Users/dsherk/builds/mozilla-central8/js/src/jsinterp.cpp:3950
#59 0x0000000103074adb in js::RunScript (cx=0x107df6f20, script=0x111b33a50, fp=0x109a830d0) at /Users/dsherk/builds/mozilla-central8/js/src/jsinterp.cpp:584
#60 0x000000010307520a in js::InvokeKernel (cx=0x107df6f20, args={<js::CallReceiver> = {usedRval_ = false, argv_ = 0x109a83098}, argc_ = 3}, construct=js::NO_CONSTRUCT) at /Users/dsherk/builds/mozilla-central8/js/src/jsinterp.cpp:647
#61 0x000000010307906f in js::Invoke (cx=0x107df6f20, args=@0x7fff5fbfcae0, construct=js::NO_CONSTRUCT) at jsinterp.h:148
#62 0x000000010307562a in js::Invoke (cx=0x107df6f20, thisv=@0x7fff5fbfcbb0, fval=@0x7fff5fbfcb78, argc=3, argv=0x121214800, rval=0x7fff5fbfcdb0) at /Users/dsherk/builds/mozilla-central8/js/src/jsinterp.cpp:679
#63 0x0000000102f927e0 in JS_CallFunctionValue (cx=0x107df6f20, obj=0x10a90a9c0, fval={data = {asBits = 18445477440787549064, debugView = {payload47 = 4473195400, tag = JSVAL_TAG_OBJECT}, s = {payload = {i32 = 178228104, u32 = 178228104, why = 178228104}}, asDouble = -nan(0xb80010a9f8b88), asPtr = 0xfffb80010a9f8b88, asWord = 18445477440787549064}}, argc=3, argv=0x121214800, rval=0x7fff5fbfcdb0) at /Users/dsherk/builds/mozilla-central8/js/src/jsapi.cpp:5221
#64 0x0000000101d992f9 in nsJSContext::CallEventHandler (this=0x10a569710, aTarget=0x10a3cfc00, aScope=0x10a90a9c0, aHandler=0x10a9f8b88, aargv=0x120eddf08, arv=0x7fff5fbfd030) at /Users/dsherk/builds/mozilla-central8/dom/base/nsJSEnvironment.cpp:1945
#65 0x0000000101dd401d in nsGlobalWindow::RunTimeout (this=0x10a3cfc00, aTimeout=0x12120ada0) at /Users/dsherk/builds/mozilla-central8/dom/base/nsGlobalWindow.cpp:9314
#66 0x0000000101dd4607 in nsGlobalWindow::TimerCallback (aTimer=0x120ecb8e0, aClosure=0x12120ada0) at /Users/dsherk/builds/mozilla-central8/dom/base/nsGlobalWindow.cpp:9754
#67 0x0000000102acc4bb in nsTimerImpl::Fire (this=0x120ecb8e0) at /Users/dsherk/builds/mozilla-central8/xpcom/threads/nsTimerImpl.cpp:425
#68 0x0000000102acc77a in nsTimerEvent::Run (this=0x12122a070) at /Users/dsherk/builds/mozilla-central8/xpcom/threads/nsTimerImpl.cpp:521
#69 0x0000000102ac4d0c in nsThread::ProcessNextEvent (this=0x100538520, mayWait=false, result=0x7fff5fbfd3ef) at /Users/dsherk/builds/mozilla-central8/xpcom/threads/nsThread.cpp:631
#70 0x0000000102a53084 in NS_ProcessPendingEvents_P (thread=0x100538520, timeout=20) at /Users/dsherk/builds/obj/xpcom/build/nsThreadUtils.cpp:195
#71 0x0000000102819ff1 in nsBaseAppShell::NativeEventCallback (this=0x107d0ea60) at /Users/dsherk/builds/mozilla-central8/widget/src/xpwidgets/nsBaseAppShell.cpp:130
#72 0x00000001027c6c3a in nsAppShell::ProcessGeckoEvents (aInfo=0x107d0ea60) at /Users/dsherk/builds/mozilla-central8/widget/src/cocoa/nsAppShell.mm:424
#73 0x00007fff85078b51 in __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ ()
#74 0x00007fff850783bd in __CFRunLoopDoSources0 ()
#75 0x00007fff8509f1a9 in __CFRunLoopRun ()
#76 0x00007fff8509eae6 in CFRunLoopRunSpecific ()
#77 0x00007fff8cb9b3d3 in RunCurrentEventLoopInMode ()
#78 0x00007fff8cba263d in ReceiveNextEventCommon ()
#79 0x00007fff8cba24ca in BlockUntilNextEventMatchingListInMode ()
#80 0x00007fff83ac53f1 in _DPSNextEvent ()
#81 0x00007fff83ac4cf5 in -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] ()
#82 0x00007fff83ac162d in -[NSApplication run] ()
#83 0x00000001027c655e in nsAppShell::Run (this=0x107d0ea60) at /Users/dsherk/builds/mozilla-central8/widget/src/cocoa/nsAppShell.mm:771
#84 0x0000000102522875 in nsAppStartup::Run (this=0x107d0c830) at /Users/dsherk/builds/mozilla-central8/toolkit/components/startup/nsAppStartup.cpp:228
#85 0x000000010136c848 in XRE_main (argc=5, argv=0x7fff5fbffb00, aAppData=0x10050de80) at /Users/dsherk/builds/mozilla-central8/toolkit/xre/nsAppRunner.cpp:3551
#86 0x0000000100001b4d in do_main (exePath=0x7fff5fbff6d0 "/Users/dsherk/builds/obj/dist/NightlyDebug.app/Contents/MacOS/libxpcom.dylib", argc=5, argv=0x7fff5fbffb00) at /Users/dsherk/builds/mozilla-central8/browser/app/nsBrowserApp.cpp:198
#87 0x0000000100001db4 in main (argc=5, argv=0x7fff5fbffb00) at /Users/dsherk/builds/mozilla-central8/browser/app/nsBrowserApp.cpp:281

Comment 1

[Doug Sherk (:drs) (inactive)]

Hmm, after I set NO_MAC_JEMALLOC=1, then re-enabled it again (by removing the env var), it seems to be fine. I'll leave this open for now in case there is actually a problem here. Just a note: I was working with stuff that was causing stack smashing, but I removed it all and went back to stock and rebuild/re-ran Fx several times yet this issue still happened.

Comment 2

[Justin Lebar (not reading bugmail)]

Was this a debug build?  I don't see why huge_dalloc() should be calling memset otherwise.

Comment 3

[Doug Sherk (:drs) (inactive)]

Yes.

Comment 4

[Justin Lebar (not reading bugmail)]

You don't have any other stacks sitting around, do you?

This doesn't necessarily look like a jemalloc issue.

Just to check, if you exported the NO_MAC_JEMALLOC env var, did you delete it via |unset| (or opening a new shell)?

Comment 5

[Doug Sherk (:drs) (inactive)]

Closing, haven't had this in forever, I think I just failed at something.