Closed Bug 999158 Opened 11 years ago Closed 10 years ago

crash in js::CrashAtUnhandlableOOM(char const*) | js::Nursery::MinorGCCallback(JSTracer*, void**, JSGCTraceKind)

Categories

(Core :: JavaScript: GC, defect)

Product:
Core
Component:
JavaScript: GC
Version:
31 Branch
Platform:
x86
Windows NT
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

()

Status:
VERIFIED FIXED
Milestone:
mozilla33
Tracking Flags:
Tracking Status
firefox31 + verified disabled
firefox32 + fixed
firefox33 + verified
firefox34 --- unaffected

People

(Reporter: lizzard, Assigned: jonco)

References

Details

(Keywords: crash, topcrash-win, Whiteboard: [GGC] )

Crash Data

Attachments

(2 files)

keep-spare-chunk-for-ggc
6.28 KB, patch
terrence
: review+
lmandel
: approval-mozilla-beta+
terrence
: checkin+
Details | Diff | Splinter Review
keep-spare-chunk-for-ggc-beta-v0.diff
5.89 KB, patch
terrence
: review+
Details | Diff | Splinter Review
This bug was filed from the Socorro interface and is report bp-13a18be7-b5f0-4b27-8ac0-4fbca2140421. ============================================================= This bug first started appearing in the 2014032903 build for Firefox 31.0a1. More reports: https://crash-stats.mozilla.com/report/list?range_unit=days&range_value=28&signature=js%3A%3ACrashAtUnhandlableOOM%28char+const*%29+|+js%3A%3ANursery%3A%3AMinorGCCallback%28JSTracer*%2C+void**%2C+JSGCTraceKind%29#tab-sigsummary URLs: http://worldofwarplanes.eu/ http://airmozilla-ops2.corpdmz.scl3.mozilla.com/ https://www.facebook.com/ stack: 0 mozjs.dll js::CrashAtUnhandlableOOM(char const *) js/src/jscntxt.cpp 1 mozjs.dll js::Nursery::MinorGCCallback(JSTracer *,void * *,JSGCTraceKind) js/src/gc/Nursery.cpp 2 mozjs.dll js::gc::StoreBuffer::MonoTypeBuffer<js::gc::StoreBuffer::CellPtrEdge>::mark(js::gc::StoreBuffer *,JSTracer *) js/src/gc/StoreBuffer.cpp 3 mozjs.dll js::Nursery::collect(JSRuntime *,JS::gcreason::Reason,js::Vector<js::types::TypeObject *,0,js::SystemAllocPolicy> *) js/src/gc/Nursery.cpp 4 mozjs.dll Collect js/src/jsgc.cpp 5 mozjs.dll js::GC(JSRuntime *,js::JSGCInvocationKind,JS::gcreason::Reason) js/src/jsgc.cpp 6 mozjs.dll RunLastDitchGC js/src/jsgc.cpp 7 mozjs.dll js::gc::ArenaLists::refillFreeList<1>(js::ThreadSafeContext *,js::gc::AllocKind) js/src/jsgc.cpp 8 mozjs.dll NewObject js/src/jsobj.cpp 9 mozjs.dll js::NewObjectWithClassProtoCommon(js::ExclusiveContext *,js::Class const *,JSObject *,JSObject *,js::gc::AllocKind,js::NewObjectKind) js/src/jsobj.cpp 10 mozjs.dll js::NewFunctionByIdWithReserved(JSContext *,bool (*)(JSContext *,unsigned int,JS::Value *),unsigned int,unsigned int,JSObject *,jsid) js/src/jsfriendapi.cpp 11 xul.dll XPCNativeMember::Resolve(XPCCallContext &,XPCNativeInterface *,JS::Handle<JSObject *>,JS::Value *) js/xpconnect/src/XPCWrappedNativeInfo.cpp 12 xul.dll DefinePropertyIfFound js/xpconnect/src/XPCWrappedNativeJSOps.cpp 13 xul.dll XPC_WN_ModsAllowed_Proto_Resolve js/xpconnect/src/XPCWrappedNativeJSOps.cpp 14 mozjs.dll js::DefineNativeProperty(js::ExclusiveContext *,JS::Handle<JSObject *>,JS::Handle<jsid>,JS::Handle<JS::Value>,bool (*)(JSContext *,JS::Handle<JSObject *>,JS::Handle<jsid>,JS::MutableHandle<JS::Value>),bool (*)(JSContext *,JS::Handle<JSObject *>,JS::Handle<jsid>,bool,JS::MutableHandle<JS::Value>),unsigned int,unsigned int,unsigned int) js/src/jsobj.cpp 15 mozjs.dll JSObject::defineGeneric(js::ExclusiveContext *,JS::Handle<JSObject *>,JS::Handle<jsid>,JS::Handle<JS::Value>,bool (*)(JSContext *,JS::Handle<JSObject *>,JS::Handle<jsid>,JS::MutableHandle<JS::Value>),bool (*)(JSContext *,JS::Handle<JSObject *>,JS::Handle<jsid>,bool,JS::MutableHandle<JS::Value>),unsigned int) js/src/jsobj.cpp 16 mozjs.dll DefinePropertyById js/src/jsapi.cpp 17 mozjs.dll JS_DefineProperties(JSContext *,JS::Handle<JSObject *>,JSPropertySpec const *) js/src/jsapi.cpp 18 xul.dll mozilla::dom::DefinePrefable<JSPropertySpec const >(JSContext *,JS::Handle<JSObject *>,mozilla::dom::Prefable<JSPropertySpec const > const *) dom/bindings/BindingUtils.cpp 19 xul.dll mozilla::dom::DefineWebIDLBindingPropertiesOnXPCObject(JSContext *,JS::Handle<JSObject *>,mozilla::dom::NativeProperties const *,bool) dom/bindings/BindingUtils.cpp 20 xul.dll xpc_qsDefineQuickStubs(JSContext *,JSObject *,unsigned int,unsigned int,nsID const * *,unsigned int,xpc_qsHashEntry const *,xpc_qsPropertySpec const *,xpc_qsFunctionSpec const *,char const *) js/xpconnect/src/XPCQuickStubs.cpp 21 xul.dll nsDOMClassInfo::PostCreatePrototype(JSContext *,JSObject *) dom/base/nsDOMClassInfo.cpp 22 xul.dll nsWindowSH::PostCreatePrototype(JSContext *,JSObject *) dom/base/nsDOMClassInfo.cpp 23 xul.dll XPCWrappedNativeProto::CallPostCreatePrototype() js/xpconnect/src/XPCWrappedNativeProto.cpp 24 xul.dll XPCWrappedNativeProto::Init(XPCNativeScriptableCreateInfo const *,bool) js/xpconnect/src/XPCWrappedNativeProto.cpp 25 xul.dll XPCWrappedNativeProto::GetNewOrUsed(XPCWrappedNativeScope *,nsIClassInfo *,XPCNativeScriptableCreateInfo const *,bool) js/xpconnect/src/XPCWrappedNativeProto.cpp 26 xul.dll nsXPConnect::GetWrappedNativePrototype(JSContext *,JSObject *,nsIClassInfo *,nsIXPConnectJSObjectHolder * *) js/xpconnect/src/nsXPConnect.cpp 27 xul.dll GetXPCProto dom/base/nsDOMClassInfo.cpp 28 xul.dll nsWindowSH::GlobalResolve(nsGlobalWindow *,JSContext *,JS::Handle<JSObject *>,JS::Handle<jsid>,JS::MutableHandle<JSPropertyDescriptor>) dom/base/nsDOMClassInfo.cpp 29 xul.dll nsWindowSH::NewResolve(nsIXPConnectWrappedNative *,JSContext *,JSObject *,jsid,unsigned int,JSObject * *,bool *) dom/base/nsDOMClassInfo.cpp 30 xul.dll XPC_WN_Helper_NewResolve js/xpconnect/src/XPCWrappedNativeJSOps.cpp 31 mozjs.dll JSObject::lookupGeneric(JSContext *,JS::Handle<JSObject *>,JS::Handle<jsid>,JS::MutableHandle<JSObject *>,JS::MutableHandle<js::Shape *>) js/src/jsobj.cpp 32 mozjs.dll LookupPropertyById js/src/jsapi.cpp 33 mozjs.dll JS_LookupPropertyById(JSContext *,JS::Handle<JSObject *>,JS::Handle<jsid>,JS::MutableHandle<JS::Value>) js/src/jsapi.cpp 34 mozjs.dll JS_LookupProperty(JSContext *,JS::Handle<JSObject *>,char const *,JS::MutableHandle<JS::Value>) js/src/jsapi.cpp 35 xul.dll ResolvePrototype dom/base/nsDOMClassInfo.cpp 36 xul.dll nsDOMClassInfo::PostCreatePrototype(JSContext *,JSObject *) dom/base/nsDOMClassInfo.cpp 37 xul.dll nsWindowSH::PostCreatePrototype(JSContext *,JSObject *) dom/base/nsDOMClassInfo.cpp 38 xul.dll XPCWrappedNativeProto::CallPostCreatePrototype() js/xpconnect/src/XPCWrappedNativeProto.cpp 39 xul.dll XPCWrappedNative::FinishInitForWrappedGlobal() js/xpconnect/src/XPCWrappedNative.cpp 40 xul.dll nsGlobalWindow::SetNewDocument(nsIDocument *,nsISupports *,bool) dom/base/nsGlobalWindow.cpp 41 xul.dll nsDocumentViewer::InitInternal(nsIWidget *,nsISupports *,nsIntRect const &,bool,bool,bool) layout/base/nsDocumentViewer.cpp 42 xul.dll nsDocumentViewer::Init(nsIWidget *,nsIntRect const &) layout/base/nsDocumentViewer.cpp 43 xul.dll nsDocShell::SetupNewViewer(nsIContentViewer *) docshell/base/nsDocShell.cpp 44 xul.dll nsDocShell::Embed(nsIContentViewer *,char const *,nsISupports *) docshell/base/nsDocShell.cpp 45 xul.dll nsDocShell::CreateAboutBlankContentViewer(nsIPrincipal *,nsIURI *,bool) docshell/base/nsDocShell.cpp 46 xul.dll nsDocShell::CreateAboutBlankContentViewer(nsIPrincipal *) docshell/base/nsDocShell.cpp 47 xul.dll nsWebShellWindow::Initialize(nsIXULWindow *,nsIXULWindow *,nsIURI *,int,int,bool,nsWidgetInitData &) xpfe/appshell/src/nsWebShellWindow.cpp 48 xul.dll nsAppShellService::JustCreateTopWindow(nsIXULWindow *,nsIURI *,unsigned int,int,int,bool,nsWebShellWindow * *) xpfe/appshell/src/nsAppShellService.cpp 49 xul.dll nsAppShellService::CreateTopLevelWindow(nsIXULWindow *,nsIURI *,unsigned int,int,int,nsIXULWindow * *) xpfe/appshell/src/nsAppShellService.cpp 50 xul.dll nsXULWindow::CreateNewContentWindow(int,nsIXULWindow * *) xpfe/appshell/src/nsXULWindow.cpp 51 xul.dll nsXULWindow::CreateNewWindow(int,nsIXULWindow * *) xpfe/appshell/src/nsXULWindow.cpp 52 xul.dll nsAppStartup::CreateChromeWindow2(nsIWebBrowserChrome *,unsigned int,unsigned int,nsIURI *,bool *,nsIWebBrowserChrome * *) toolkit/components/startup/nsAppStartup.cpp 53 xul.dll nsWindowWatcher::OpenWindowInternal(nsIDOMWindow *,char const *,char const *,char const *,bool,bool,bool,nsIArray *,nsIDOMWindow * *) embedding/components/windowwatcher/src/nsWindowWatcher.cpp 54 xul.dll nsWindowWatcher::OpenWindow2(nsIDOMWindow *,char const *,char const *,char const *,bool,bool,bool,nsISupports *,nsIDOMWindow * *) embedding/components/windowwatcher/src/nsWindowWatcher.cpp 55 xul.dll nsGlobalWindow::OpenInternal(nsAString_internal const &,nsAString_internal const &,nsAString_internal const &,bool,bool,bool,bool,bool,nsIArray *,nsISupports *,nsIPrincipal *,JSContext *,nsIDOMWindow * *) dom/base/nsGlobalWindow.cpp 56 xul.dll nsGlobalWindow::OpenInternal(nsAString_internal const &,nsAString_internal const &,nsAString_internal const &,bool,bool,bool,bool,bool,nsIArray *,nsISupports *,nsIPrincipal *,JSContext *,nsIDOMWindow * *) dom/base/nsGlobalWindow.cpp 57 xul.dll nsGlobalWindow::OpenJS(nsAString_internal const &,nsAString_internal const &,nsAString_internal const &,nsIDOMWindow * *) dom/base/nsGlobalWindow.cpp 58 xul.dll nsGlobalWindow::Open(nsAString_internal const &,nsAString_internal const &,nsAString_internal const &,mozilla::ErrorResult &) dom/base/nsGlobalWindow.cpp 59 xul.dll mozilla::dom::WindowBinding::open obj-firefox/dom/bindings/WindowBinding.cpp 60 xul.dll mozilla::dom::WindowBinding::genericMethod obj-firefox/dom/bindings/WindowBinding.cpp 61 mozjs.dll js::Invoke(JSContext *,JS::CallArgs,js::MaybeConstruct) js/src/vm/Interpreter.cpp 62 mozjs.dll Interpret js/src/vm/Interpreter.cpp 63 mozjs.dll js::RunScript(JSContext *,js::RunState &) js/src/vm/Interpreter.cpp 64 mozjs.dll js::ExecuteKernel(JSContext *,JS::Handle<JSScript *>,JSObject &,JS::Value const &,js::ExecuteType,js::AbstractFramePtr,JS::Value *) js/src/vm/Interpreter.cpp 65 mozjs.dll js::Execute(JSContext *,JS::Handle<JSScript *>,JSObject &,JS::Value *) js/src/vm/Interpreter.cpp 66 mozjs.dll Evaluate js/src/jsapi.cpp 67 mozjs.dll JS::Evaluate(JSContext *,JS::Handle<JSObject *>,JS::ReadOnlyCompileOptions const &,wchar_t const *,unsigned int) js/src/jsapi.cpp 68 xul.dll nsJSUtils::EvaluateString(JSContext *,nsAString_internal const &,JS::Handle<JSObject *>,JS::CompileOptions &,nsJSUtils::EvaluateOptions const &,JS::MutableHandle<JS::Value>,void * *) dom/base/nsJSUtils.cpp 69 xul.dll nsJSUtils::EvaluateString(JSContext *,nsAString_internal const &,JS::Handle<JSObject *>,JS::CompileOptions &,void * *) dom/base/nsJSUtils.cpp 70 xul.dll nsScriptLoader::EvaluateScript(nsScriptLoadRequest *,nsString const &,void * *) content/base/src/nsScriptLoader.cpp 71 xul.dll nsScriptLoader::ProcessRequest(nsScriptLoadRequest *,void * *) content/base/src/nsScriptLoader.cpp 72 xul.dll nsScriptLoader::ProcessScriptElement(nsIScriptElement *) content/base/src/nsScriptLoader.cpp 73 xul.dll nsScriptElement::MaybeProcessScript() content/base/src/nsScriptElement.cpp 74 xul.dll nsIScriptElement::AttemptToExecute() obj-firefox/dist/include/nsIScriptElement.h 75 xul.dll nsHtml5TreeOpExecutor::RunScript(nsIContent *) parser/html/nsHtml5TreeOpExecutor.cpp 76 xul.dll nsHtml5TreeOpExecutor::RunFlushLoop() parser/html/nsHtml5TreeOpExecutor.cpp 77 xul.dll nsHtml5ExecutorReflusher::Run() parser/html/nsHtml5TreeOpExecutor.cpp 78 xul.dll nsThread::ProcessNextEvent(bool,bool *) xpcom/threads/nsThread.cpp 79 xul.dll NS_ProcessNextEvent(nsIThread *,bool) xpcom/glue/nsThreadUtils.cpp 80 xul.dll nsXULWindow::CreateNewContentWindow(int,nsIXULWindow * *) xpfe/appshell/src/nsXULWindow.cpp 81 xul.dll nsXULWindow::CreateNewWindow(int,nsIXULWindow * *) xpfe/appshell/src/nsXULWindow.cpp 82 xul.dll nsAppStartup::CreateChromeWindow2(nsIWebBrowserChrome *,unsigned int,unsigned int,nsIURI *,bool *,nsIWebBrowserChrome * *) toolkit/components/startup/nsAppStartup.cpp 83 xul.dll nsWindowWatcher::OpenWindowInternal(nsIDOMWindow *,char const *,char const *,char const *,bool,bool,bool,nsIArray *,nsIDOMWindow * *) embedding/components/windowwatcher/src/nsWindowWatcher.cpp 84 xul.dll nsWindowWatcher::OpenWindow2(nsIDOMWindow *,char const *,char const *,char const *,bool,bool,bool,nsISupports *,nsIDOMWindow * *) embedding/components/windowwatcher/src/nsWindowWatcher.cpp 85 xul.dll nsGlobalWindow::OpenInternal(nsAString_internal const &,nsAString_internal const &,nsAString_internal const &,bool,bool,bool,bool,bool,nsIArray *,nsISupports *,nsIPrincipal *,JSContext *,nsIDOMWindow * *) dom/base/nsGlobalWindow.cpp 86 xul.dll nsGlobalWindow::OpenInternal(nsAString_internal const &,nsAString_internal const &,nsAString_internal const &,bool,bool,bool,bool,bool,nsIArray *,nsISupports *,nsIPrincipal *,JSContext *,nsIDOMWindow * *) dom/base/nsGlobalWindow.cpp 87 xul.dll nsGlobalWindow::OpenJS(nsAString_internal const &,nsAString_internal const &,nsAString_internal const &,nsIDOMWindow * *) dom/base/nsGlobalWindow.cpp 88 xul.dll nsGlobalWindow::Open(nsAString_internal const &,nsAString_internal const &,nsAString_internal const &,mozilla::ErrorResult &) dom/base/nsGlobalWindow.cpp 89 xul.dll mozilla::dom::WindowBinding::open obj-firefox/dom/bindings/WindowBinding.cpp 1015 xul.dll nsGlobalWindow::OpenInternal(nsAString_internal const &,nsAString_internal const &,nsAString_internal const &,bool,bool,bool,bool,bool,nsIArray *,nsISupports *,nsIPrincipal *,JSContext *,nsIDOMWindow * *) dom/base/nsGlobalWindow.cpp 1016 xul.dll nsGlobalWindow::OpenInternal(nsAString_internal const &,nsAString_internal const &,nsAString_internal const &,bool,bool,bool,bool,bool,nsIArray *,nsISupports *,nsIPrincipal *,JSContext *,nsIDOMWindow * *) dom/base/nsGlobalWindow.cpp 1017 xul.dll nsGlobalWindow::OpenJS(nsAString_internal const &,nsAString_internal const &,nsAString_internal const &,nsIDOMWindow * *) dom/base/nsGlobalWindow.cpp 1018 xul.dll nsGlobalWindow::Open(nsAString_internal const &,nsAString_internal const &,nsAString_internal const &,mozilla::ErrorResult &) dom/base/nsGlobalWindow.cpp 1019 xul.dll mozilla::dom::WindowBinding::open obj-firefox/dom/bindings/WindowBinding.cpp 1020 xul.dll mozilla::dom::WindowBinding::genericMethod obj-firefox/dom/bindings/WindowBinding.cpp 1021 mozjs.dll js::Invoke(JSContext *,JS::CallArgs,js::MaybeConstruct) js/src/vm/Interpreter.cpp 1022 mozjs.dll Interpret js/src/vm/Interpreter.cpp 1023 mozjs.dll js::RunScript(JSContext *,js::RunState &) js/src/vm/Interpreter.cpp 1024 mozjs.dll js::ExecuteKernel(JSContext *,JS::Handle<JSScript *>,JSObject &,JS::Value const &,js::ExecuteType,js::AbstractFramePtr,JS::Value *) js/src/vm/Interpreter.cpp
Note that this starting to appear on 2014-04-18 in crash data probably does not point out that it regressed there, as the signature changed from just "js::CrashAtUnhandlableOOM(char const*)" to the current one due to bug 994913 being shipped to production late on the 17th (possibly early 18th UTC).
Blocks: 994589
There are 132 crash reports for this signature in the last 7 days, and it's #10 topcrasher for Firefox 31.0a1.
Keywords: topcrash-win
This is now a top crasher on nightly 32a1.
status-firefox32: --- → affected
Ignore comment #4. I was looking at the wrong set of data in crash stats, thinking this was on 32a1. Removing tracking flags.
status-firefox32: affected → ---
tracking-firefox32: ? → ---
Terrence, do we have anything we can do there? This is showing up as a topcrash in Aurora 31.
Flags: needinfo?(terrence)
Sadly, we don't really have anything to do here in the short term. In general, these crashes would likely have OOMed elsewhere a bit before this point previously: the nursery acts like ballast in that respect. I'll be continuing to think about ways we can improve the situation, however.
Flags: needinfo?(terrence)
Topcrash, tracking.
tracking-firefox31: ? → +
Hrm, this is the #3 crash on Aurora 31 now. Not that OOM being a major issue is news to us...
This is the #2 topcrash on 31, with 332/3819 crashes. It has a few crashes showing up on 32 as well, so I'm marking it as affected.
status-firefox32: --- → affected
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #9) > Hrm, this is the #3 crash on Aurora 31 now. Not that OOM being a major issue > is news to us... This may need to be considered as a blocker to allowing this to ride to Beta. I know OOM isn't anything new to us but we shouldn't knowingly introduce anything which makes that situation any more worse.
Is there a change in the overall number of OOM crashes? If all other OOM crashes are just turning into this, it may not be any worse. Of course, it may be an overall increase.
If there isn't a spike in the overall number of OOMs, bug 1005849 may help by making the allocator much more resistant to unaligned chunks (if these are OOMs from address space fragmentation rather than simply running out). It may also 'help' by making eventual OOMs more likely to happen in another allocator (namely jemalloc, which doesn't have the same logic yet). Bug 1005849 is in Nightlies from 2014-05-28 and on, so not a lot of time for statistics yet (looks like 8 crashes so far on 32.0a1 from builds containing the fix).
I can't speak to whether there's been an overall spike in OOM crashes, perhaps KaiRo can help with that? We did discuss this in the Crashkill meeting today and decided that this would not block GGC from riding to Beta. However, if we see a notable change in stability within the first couple of Betas we may need to back out or pref off GGC.
(In reply to Andrew McCreight [:mccr8] from comment #12) > Is there a change in the overall number of OOM crashes? We do not have a good metric for that at this time, but we're working on that for the future. With the current state, I think it's overall best to let ride ride into beta and see if the overall crash rate there gets worse, only reacting when that is the case. As Anthony mentions, we discussed that at the CrashKill meeting and bsmedberg agrees with what I said here as well. (I'm looking forward to seeing how bug 1005849 and its jemalloc "sister" might help us overall around OOM, though!)
This is the #2 topcrash for Firefox 31 with 391/4251 crashes in the last 7 days.
This is the #3 topcrash for Firefox 31.0b1, with 2342/29222 crashes in the last week. It's also showing up on the explosiveness report with an index of 3.6: https://crash-analysis.mozilla.com/rkaiser/2014-06-15/2014-06-15.firefox.31.explosiveness.html.
Crash Signature: [@ js::CrashAtUnhandlableOOM(char const*) | js::Nursery::MinorGCCallback(JSTracer*, void**, JSGCTraceKind)] → [@ js::CrashAtUnhandlableOOM(char const*) | js::Nursery::MinorGCCallback(JSTracer*, void**, JSGCTraceKind)] [@ OOM | unknown | js::CrashAtUnhandlableOOM(char const*) | js::Nursery::MinorGCCallback(JSTracer*, void**, JSGCTraceKind)]
Given that ends up with an "OOM | unknown | ..." signature, we should at least see to those crashes ending up with an OOMAllocationSize annotation. Terrence, can this be done?
Flags: needinfo?(terrence)
Depends on: 1026545
> can this be done? It will require a bit of plumbing work, but it should be pretty easy. I filed bug 1026545 for the requisite work. Then it will be easy to hook up this particular call site to the OOM reporting mechanism.
Well, I'm not sure how useful bug 1026545 will end up being. It seems like the size you'd want to pass in here would be something like Arena::thingSize(thingKind), but in reality, the GC thing can't be allocated because we couldn't allocate an arena or a chunk or something. The thing that knows the size of the failed allocation is going to be deeper up the stack, which complicates things because it will need to deal with both fallible and infallible allocation. Maybe something could set a value on the runtime when an allocation fails, and when later we decide that we're infallible, the flag could be checked. (This is also a problem in bug 1016388.) Though I'm not entirely sure how useful it would be just to have these reports saying that we failed to allocate a 1MB chunk, as that's probably what this ends up being, way up the stack.
I second what Andrew said, with one caveat. There is one hash-tables we're using where failure to resize results in this crash. In that one case, such an annotation would be helpful, but from what I can tell looking at crash reports this case is incredibly rare.
Flags: needinfo?(terrence)
It sounds like we need to trigger GCs sooner so that we don't hit this situation where we are trying to collect to free memory, but we can't because we don't have enough free memory to tenure the live nursery things. There's a small possibility that bug 1032750 will help here. Apart from that it sounds like we need some kind of memory pressure event to try and free up memory before we get to this point.
Per the relevant email thread, we have decided to disable on beta to stabilize things given that we don't know what the other source of new crashes is yet. (In reply to Jon Coppeard (:jonco) from comment #22) > It sounds like we need to trigger GCs sooner so that we don't hit this > situation where we are trying to collect to free memory, but we can't > because we don't have enough free memory to tenure the live nursery things. > > There's a small possibility that bug 1032750 will help here. > > Apart from that it sounds like we need some kind of memory pressure event to > try and free up memory before we get to this point. We could also mitigate it somewhat without fixing our triggers by adjusting the chunk pool too keep at least one decommitted chunk around in all cases.
This patch adds a minimum size for the free chunk pool and sets it to one if generational GC is enabled.
Attachment #8454465 - Flags: review?(terrence)
Comment on attachment 8454465 [details] [diff] [review] keep-spare-chunk-for-ggc Review of attachment 8454465 [details] [diff] [review]: ----------------------------------------------------------------- r=me Thanks for getting to this sooner than I could!
Attachment #8454465 - Flags: review?(terrence) → review+
sorry had to backout this change for test failures like https://tbpl.mozilla.org/php/getParsedLog.php?id=43733909&tree=Mozilla-Inbound
https://hg.mozilla.org/mozilla-central/rev/6d8732b0aeba
Assignee: nobody → jcoppeard
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla33
Status: RESOLVED → REOPENED
Keywords: leave-open
Resolution: FIXED → ---
GGC has been disabled in 31. Untracking this bug for this release but tracking it for 32 & 33.
status-firefox31: affected → disabled
status-firefox33: --- → affected
tracking-firefox32: --- → +
tracking-firefox33: --- → +
My SO's Firefox 31 has crashed like this: bp-b9fe39a8-6a78-49f3-9d49-5b5a32140707 07/07/2014 01:06 p.m. Commenting so we know when this bug is fixed. Let us know if there's anything we can provide forma a crashing profile.
This is gone from Firefox 31.0b9 and above: https://crash-stats.mozilla.com/topcrasher_ranks_bybug/?bug_number=999158 @alex_mayorga, if your SO still sees this, please make sure Firefox is updated. This should be gone in builds where we backed out GGC (Firefox 31.0b9 and above).
status-firefox31: disabled → verified disabled
Actually, the fix here landed in 33. Crash-stats does not show this as a top-crasher in 33 or 34. I'll nominate the patch here for uplift to 32.
status-firefox33: affected → fixed
status-firefox34: --- → unaffected
Comment on attachment 8454465 [details] [diff] [review] keep-spare-chunk-for-ggc Approval Request Comment [Feature/regressing bug #]: GGC. [User impact if declined]: Increased OOM probability on Windows. [Describe test coverage new/current, TBPL]: On m-i for 1 release. [Risks and why]: Extremely low. This patch simply increases the reserve chunk count from 0 to 1. [String/UUID change made/needed]: None.
Attachment #8454465 - Flags: approval-mozilla-beta?
Comment on attachment 8454465 [details] [diff] [review] keep-spare-chunk-for-ggc beta+ - Let's get this into beta7.
Attachment #8454465 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
This needed a bit of rebasing for beta.
Attachment #8473078 - Flags: review+
Attachment #8454465 - Flags: checkin+
https://hg.mozilla.org/releases/mozilla-beta/rev/97fd0156fdc2
Status: REOPENED → RESOLVED
Closed: 11 years ago → 10 years ago
status-firefox32: affected → fixed
Keywords: checkin-needed, leave-open
Resolution: --- → FIXED
Flags: qe-verify+
Looking in Socorro [1] for crashes over the last week (since this landed in 32.0b8), I can still see many crashes for signature [@ OOM | unknown | js::CrashAtUnhandlableOOM(char const*) | js::Nursery::MinorGCCallback(JSTracer*, void**, JSGCTraceKind)]: - 32.0b8 - 198 crashes - 32.0b9 - 63 crashes This does not look fixed as it still ranks ~50-60th ([2]), same as for beta 6 and 7. Can anyone review these results? [1] - https://crash-stats.mozilla.com/report/list?product=Firefox&signature=OOM+%7C+unknown+%7C+js%3A%3ACrashAtUnhandlableOOM%28char+const%2A%29+%7C+js%3A%3ANursery%3A%3AMinorGCCallback%28JSTracer%2A%2C+void%2A%2A%2C+JSGCTraceKind%29 [2] - https://crash-stats.mozilla.com/topcrasher_ranks_bybug/?bug_number=999158
Well, the name of the crash site is CrashAt*Unhandlable*OOM. The mitigation here took us from #1-2 down to #50-60. That's success.
(In reply to Terrence Cole [:terrence] from comment #39) > Well, the name of the crash site is CrashAt*Unhandlable*OOM. The mitigation > here took us from #1-2 down to #50-60. That's success. Yes, true. Florin, if this is roughly the same much-decresed volume across branches, let's consider it verified.
My problem here is that I do not see any improvement in Beta8 and Beta9 compared to previous versions. Since the fix landed in Beta8 I would expect to see some clear improvement in Beta 8 & 9 compared to previous versions. Looking at [1] which represents the crashes over the past 4 weeks I see that the numbers for Beta8 are really close to those for Beta2 and Beta4, and rather close to Beta6, Beta5, and even Beta1: - 32.0b8 - 4.40 % - 212 crashes - 32.0b4 - 4.81 % - 232 crashes - 32.0b2 - 4.92 % - 237 crashes - 32.0b6 - 4.98 % - 240 crashes - 32.0b5 - 5.18 % - 250 crashes - 32.0b1 - 5.56 % - 268 crashes Looking at [2] I see that the ranking in Beta8 and Beta9 is worse than in Beta1, 4, 5, and 6: Rank Version 56 - 32.0b9 58 - 32.0b8 57 - 32.0b1 48 - 32.0b2 86 - 32.0b5 64 - 32.0b4 47 - 32.0b7 61 - 32.0b6 Given the proximity of numbers above I'm really not sure that we can call this verified when there is no visible improvement in versions prior to the fix in Beta... yes we may have went from #1-2 to #50-60 but this seems to have happened prior to the fix, and the fix itself seems to have not brought much of an improvement. [1] - https://crash-stats.mozilla.com/report/list?product=Firefox&range_unit=days&range_value=28&signature=OOM+%7C+unknown+%7C+js%3A%3ACrashAtUnhandlableOOM%28char+const%2A%29+%7C+js%3A%3ANursery%3A%3AMinorGCCallback%28JSTracer%2A%2C+void%2A%2A%2C+JSGCTraceKind%29 [2] - https://crash-stats.mozilla.com/topcrasher_ranks_bybug/?bug_number=999158
Hmm, yes, it seems like this improved a lot between 31 (before deactivation of GGC) and 32, and it's unclear if this fix helped anything in 32.
Interesting! It's quite possible that one of the fixes we pushed before disabling actually did fix the issue -- that release was total chaos so I wouldn't be terribly surprised if nobody noticed at the time.
No crashes for Firefox 33 on Soccoro with this signature therefore marking this issue verified. We still have 87 crashes over the last week for Firefox 32.0.3.
Status: RESOLVED → VERIFIED
status-firefox33: fixed → verified
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: