Last Comment Bug 732955 - OOM Crash [@ mozilla::FramePropertyTable::PropertyValue::IsArray] due to unhandled alloc in FramePropertyTable::Set
: OOM Crash [@ mozilla::FramePropertyTable::PropertyValue::IsArray] due to unha...
Status: RESOLVED FIXED
[fixed by bug 734847]
: crash
Product: Core
Classification: Components
Component: Layout (show other bugs)
: Trunk
: x86_64 Linux
: -- critical (vote)
: mozilla15
Assigned To: Nobody; OK to take it and work on it
:
Mentors:
Depends on: 734847
Blocks: 687256
  Show dependency treegraph
 
Reported: 2012-03-05 06:52 PST by Christian Holler (:decoder)
Modified: 2012-05-19 07:06 PDT (History)
6 users (show)
See Also:
Crash Signature:
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
fix (861 bytes, patch)
2012-03-12 12:27 PDT, Mats Palmgren (:mats)
no flags Details | Diff | Review

Description Christian Holler (:decoder) 2012-03-05 06:52:40 PST
Tested on m-c revision 8ea5c983743f: It seems that an OOM condition in | FramePropertyTable::Set | can leave | mLastEntry | initialized to NULL which causes the following crash:


Program received signal SIGSEGV, Segmentation fault.0x00002aaaac64dcf8 in mozilla::FramePropertyTable::PropertyValue::IsArray (this=0x10) at /srv/repos/browser/mozilla-central/layout/base/FramePropertyTable.h:167
167         bool IsArray() { return !mProperty && mValue; }
#0  0x00002aaaac64dcf8 in mozilla::FramePropertyTable::PropertyValue::IsArray (this=0x10) at /srv/repos/browser/mozilla-central/layout/base/FramePropertyTable.h:167
#1  0x00002aaaac64e102 in mozilla::FramePropertyTable::Set (this=<optimized out>, aFrame=0x4c537a0, aProperty=0x2aaaadbd52d0, aValue=0x4c55790) at /srv/repos/browser/mozilla-central/layout/base/FramePropertyTable.cpp:56
#2  0x00002aaaac6f9287 in nsIFrame::SetView (this=0x4c537a0, aView=0x4c55790) at /srv/repos/browser/mozilla-central/layout/generic/nsFrame.cpp:4138
#3  0x00002aaaac82746d in nsMenuPopupFrame::CreatePopupView (this=0x4c537a0) at /srv/repos/browser/mozilla-central/layout/xul/base/src/nsMenuPopupFrame.cpp:1938
#4  0x00002aaaac828f52 in nsMenuPopupFrame::Init (this=0x4c537a0, aContent=0x3e6dc50, aParent=<optimized out>, aPrevInFlow=<optimized out>) at /srv/repos/browser/mozilla-central/layout/xul/base/src/nsMenuPopupFrame.cpp:159
#5  0x00002aaaac65308b in InitAndRestoreFrame (aAllowCounters=true, aNewFrame=0x4c537a0, aPrevInFlow=0x0, aParentFrame=0x48be6d0, aContent=0x3e6dc50, aState=..., this=0x2aaac005ed00) at /srv/repos/browser/mozilla-central/layout/base/nsCSSFrameConstructor.cpp:4461
#6  nsCSSFrameConstructor::InitAndRestoreFrame (this=0x2aaac005ed00, aState=..., aContent=0x3e6dc50, aParentFrame=0x48be6d0, aPrevInFlow=0x0, aNewFrame=0x4c537a0, aAllowCounters=true) at /srv/repos/browser/mozilla-central/layout/base/nsCSSFrameConstructor.cpp:4444
#7  0x00002aaaac65baa1 in nsCSSFrameConstructor::ConstructFrameFromItemInternal (this=0x2aaac005ed00, aItem=..., aState=..., aParentFrame=0x48befd8, aFrameItems=...) at /srv/repos/browser/mozilla-central/layout/base/nsCSSFrameConstructor.cpp:3637
(gdb) f 1
#1  0x00002aaaac64e102 in mozilla::FramePropertyTable::Set (this=<optimized out>, aFrame=0x4d14a20, aProperty=0x2aaaadbd52d0, aValue=0x4d16a10)
    at /srv/repos/browser/mozilla-central/layout/base/FramePropertyTable.cpp:56
56        if (!entry->mProp.IsArray()) {
(gdb) p entry
$1 = (mozilla::FramePropertyTable::Entry *) 0x0


The backtrace of the failing allocation is as follows:

#0 /srv/repos/browser/mozilla-central/objdir-ff-gcc64dbg/dist/bin/libmozalloc.so(moz_malloc+0x5f)
#1 ChangeTable at objdir-ff-gcc64dbg/xpcom/build/pldhash.cpp:565
#2 PL_DHashTableOperate at objdir-ff-gcc64dbg/xpcom/build/pldhash.cpp:649
#3 mozilla::FramePropertyTable::Set(nsIFrame*, mozilla::FramePropertyDescriptor const*, void*) at layout/base/FramePropertyTable.cpp:52
#4 nsFrame::InitBoxMetrics(bool) at layout/generic/nsFrame.cpp:7890
#5 nsFrame::Init(nsIContent*, nsIFrame*, nsIFrame*) at layout/generic/nsFrame.cpp:549
#6 nsPlaceholderFrame::SetOutOfFlowFrame(nsIFrame*) at layout/base/../generic/nsPlaceholderFrame.h:117
#7 nsFrameConstructorState::AddChild(nsIFrame*, nsFrameItems&, nsIContent*, nsStyleContext*, nsIFrame*, bool, bool, bool, bool, nsIFrame*) at layout/base/nsCSSFrameConstructor.cpp:1170
#8 nsCSSFrameConstructor::ConstructFrameFromItemInternal(nsCSSFrameConstructor::FrameConstructionItem&, nsFrameConstructorState&, nsIFrame*, nsFrameItems&) at layout/base/nsCSSFrameConstructor.cpp:3691
Comment 1 Mats Palmgren (:mats) 2012-03-12 12:27:17 PDT
Created attachment 605053 [details] [diff] [review]
fix

I suspect the reported null-pointer crash on OOM here is intentional.
I think we should make this explicit by using an infallible type.

https://tbpl.mozilla.org/?usebuildbot=1&tree=Try&rev=079034b54a86
Comment 2 Mats Palmgren (:mats) 2012-03-12 13:23:20 PDT
Comment on attachment 605053 [details] [diff] [review]
fix

Benjamin didn't want a separate infallible type (in bug 734847),
so I'll make a new patch...
Comment 3 Mats Palmgren (:mats) 2012-05-19 07:06:00 PDT
nsTHashtable::PutEntry(key) is now infallible by default. (bug 734847)

Note You need to log in before you can comment on or make changes to this bug.