731167 - OOM Crash [@ nsFrame::BoxReflow ] due to fallible allocation in nsCSSRuleProcessor.cpp

Status: RESOLVED INCOMPLETE

(Core :: CSS Parsing and Computation, defect)

Description

[Christian Holler (:decoder)]

Tested on m-c revision 66e4d53697c2: The function CascadeEnumData in nsCSSRuleProcessor.cpp uses a HashTableInit (line 2935) that can fail. When this allocation fails, I get the following assertions and crash:

###!!! ASSERTION: Must be a box frame!: '!mScrollCornerBox || mScrollCornerBox->IsBoxFrame()', file /srv/repos/browser/mozilla-central/layout/generic/nsGfxScrollFrame.cpp, line 3410
###!!! ASSERTION: A box layout method was called but InitBoxMetrics was never called: 'metrics', file /srv/repos/browser/mozilla-central/layout/generic/nsFrame.cpp, line 7847

0x00002aaaac6fe855 in nsFrame::BoxReflow (this=0x27e9620, aState=..., aPresContext=0x27c5be0, aDesiredSize=..., aRenderingContext=0x27b9ed0, aX=0, aY=6000, aWidth=0, aHeight=0, aMoveFrame=true) at /srv/repos/browser/mozilla-central/layout/generic/nsFrame.cpp:7748
7748        if (metrics->mLastSize.width != aWidth) {
#0  0x00002aaaac6fe855 in nsFrame::BoxReflow (this=0x27e9620, aState=..., aPresContext=0x27c5be0, aDesiredSize=..., aRenderingContext=0x27b9ed0, aX=0, aY=6000, aWidth=0, aHeight=0, aMoveFrame=true) at /srv/repos/browser/mozilla-central/layout/generic/nsFrame.cpp:7748
#1  0x00002aaaac707d6d in nsFrame::DoLayout (this=0x27e9620, aState=...) at /srv/repos/browser/mozilla-central/layout/generic/nsFrame.cpp:7526
#2  0x00002aaaac80d954 in nsIFrame::Layout (this=0x27e9620, aState=...) at /srv/repos/browser/mozilla-central/layout/xul/base/src/nsBox.cpp:557
#3  0x00002aaaac80f6ac in nsBoxFrame::LayoutChildAt (aState=..., aBox=0x27e9620, aRect=...) at /srv/repos/browser/mozilla-central/layout/xul/base/src/nsBoxFrame.cpp:2030
#4  0x00002aaaac713449 in LayoutAndInvalidate (aState=..., aBox=0x27e9620, aRect=..., aScrollbarIsBeingHidden=false) at /srv/repos/browser/mozilla-central/layout/generic/nsGfxScrollFrame.cpp:3329
#5  0x00002aaaac714c2d in nsGfxScrollFrameInner::LayoutScrollbars (this=0x27e9338, aState=..., aContentArea=..., aOldScrollArea=...) at /srv/repos/browser/mozilla-central/layout/generic/nsGfxScrollFrame.cpp:3434
#6  0x00002aaaac71660f in nsHTMLScrollFrame::Reflow (this=0x27e92b0, aPresContext=0x27c5be0, aDesiredSize=..., aReflowState=..., aStatus=@0x7fffffffa47c) at /srv/repos/browser/mozilla-central/layout/generic/nsGfxScrollFrame.cpp:920
#7  0x00002aaaac6f22f1 in nsContainerFrame::ReflowChild (this=<optimized out>, aKidFrame=0x27e92b0, aPresContext=0x27c5be0, aDesiredSize=..., aReflowState=..., aX=0, aY=0, aFlags=0, aStatus=@0x7fffffffa47c, aTracker=0x0) at /srv/repos/browser/mozilla-central/layout/generic/nsContainerFrame.cpp:942


The backtrace of the failing allocation is as follows:

#0 /srv/repos/browser/mozilla-central/objdir-ff-gcc64dbg/dist/bin/libmozalloc.so(moz_malloc+0x5f) [0x2aaaaab2415c] (aab2415c)
#1 PL_DHashTableInit at objdir-ff-gcc64dbg/xpcom/build/pldhash.cpp:270
#2 CascadeEnumData at layout/style/nsCSSRuleProcessor.cpp:2935
#3 nsCSSRuleProcessor::GetRuleCascade(nsPresContext*) at layout/style/nsCSSRuleProcessor.cpp:3089
#4 nsCSSRuleProcessor::AppendFontFaceRules(nsPresContext*, nsTArray<nsFontFaceRuleContainer, nsTArrayDefaultAllocator>&) at layout/style/nsCSSRuleProcessor.cpp:2604
#5 nsStyleSet::AppendFontFaceRules(nsPresContext*, nsTArray<nsFontFaceRuleContainer, nsTArrayDefaultAllocator>&) at layout/style/nsStyleSet.cpp:1265
#6 nsPresContext::FlushUserFontSet() at layout/base/nsPresContext.cpp:1898
#7 nsPresContext::GetUserFontSetInternal() at layout/base/nsPresContext.cpp:1873
#8 nsDocLoader::DocLoaderIsEmpty(bool) at uriloader/base/nsDocLoader.cpp:801
#9 nsDocLoader::OnStopRequest(nsIRequest*, nsISupports*, unsigned int) at uriloader/base/nsDocLoader.cpp:740
#10 nsLoadGroup::RemoveRequest(nsIRequest*, nsISupports*, unsigned int) at netwerk/base/src/nsLoadGroup.cpp:734
#11 nsDocument::DoUnblockOnload() at content/base/src/nsDocument.cpp:7246
#12 nsDocument::DispatchContentLoadedEvents() at content/base/src/nsDocument.cpp:4198
#13 nsRunnableMethodImpl<void (nsPACMan::*)(), true>::Run() at objdir-ff-gcc64dbg/netwerk/base/src/../../../dist/include/nsThreadUtils.h:347
#14 nsThread::ProcessNextEvent(bool, bool*) at xpcom/threads/nsThread.cpp:657
#15 NS_ProcessNextEvent_P(nsIThread*, bool) at objdir-ff-gcc64dbg/xpcom/build/nsThreadUtils.cpp:245


I assume something must be wrong with the OOM handling in nsCSSRuleProcessor.cpp.

Comment 1

[Mats Palmgren (inactive)]

nsCSSRuleProcessor no longer exist and I believe all these allocations
are infallible nowadays.