610040 - OOM abort under nsBufferedStream::Init

Status: RESOLVED FIXED

(Core :: Networking: JAR, defect)

Description

[Jason Orendorff [:jorendorff]]

Apparently caused by bug 594172:

Do anything that causes the firefox process to get huge and you'll eventually abort. In short, nsZipWriter::Open calls NS_BufferedOutputStream with the hard-coded argument 3 * 1024 * 1024. That ends up calling moz_xmalloc(3 * 1024 * 1024), which aborts if there's not enough memory.

mozalloc.dll!mozalloc_abort(const char * const msg)  Line 77	C++
mozalloc.dll!mozalloc_handle_oom()  Line 54 + 0xa bytes	C++
mozalloc.dll!moz_xmalloc(unsigned int size)  Line 101	C++
xul.dll!operator new[](unsigned int size)  Line 238 + 0xa bytes	C++
xul.dll!nsBufferedStream::Init(nsISupports * stream, unsigned int bufferSize)  Line 105 + 0x9 bytes	C++
xul.dll!nsBufferedOutputStream::Init(nsIOutputStream * stream, unsigned int bufferSize)  Line 562	C++
xul.dll!NS_NewBufferedOutputStream(nsIOutputStream * * result, nsIOutputStream * str, unsigned int bufferSize)  Line 1096 + 0x21 bytes	C++
xul.dll!nsZipWriter::Open(nsIFile * aFile, int aIoFlags)  Line 304 + 0x2e bytes	C++
xul.dll!mozilla::scache::StartupCache::PutBuffer(const char * id, const char * inbuf, unsigned int len)  Line 289 + 0x39 bytes	C++
xul.dll!mozJSComponentLoader::WriteScript(mozilla::scache::StartupCache * cache, JSScript * script, nsIFile * component, nsIURI * uri, JSContext * cx)  Line 934 + 0x1e bytes	C++
xul.dll!mozJSComponentLoader::GlobalForLocation(nsILocalFile * aComponentFile, nsIURI * aURI, JSObject * * aGlobal, char * * aLocation, jsval_layout * exception)  Line 1221 + 0x2d bytes	C++
xul.dll!mozJSComponentLoader::LoadModuleImpl(nsILocalFile * aSourceFile, nsAString_internal & aKey, nsIURI * aComponentURI)  Line 729 + 0x2d bytes	C++
xul.dll!mozJSComponentLoader::LoadModule(nsILocalFile * aComponentFile)  Line 664 + 0x1f bytes	C++
xul.dll!nsComponentManagerImpl::KnownModule::Load()  Line 952 + 0x27 bytes	C++
xul.dll!nsFactoryEntry::GetFactory()  Line 1936 + 0xb bytes	C++
xul.dll!nsComponentManagerImpl::CreateInstance(const nsID & aClass, nsISupports * aDelegate, const nsID & aIID, void * * aResult)  Line 1213 + 0xc bytes	C++
xul.dll!nsJSCID::CreateInstance(nsISupports * * _retval)  Line 771 + 0x43 bytes	C++
xul.dll!NS_InvokeByIndex_P(nsISupports * that, unsigned int methodIndex, unsigned int paramCount, nsXPTCVariant * params)  Line 103	C++
xul.dll!CallMethodHelper::Invoke()  Line 3054 + 0x1c bytes	C++
xul.dll!CallMethodHelper::Call()  Line 2321 + 0x8 bytes	C++
xul.dll!XPCWrappedNative::CallMethod(XPCCallContext & ccx, XPCWrappedNative::CallMode mode)  Line 2285 + 0x16 bytes	C++
xul.dll!XPC_WN_CallMethod(JSContext * cx, unsigned int argc, jsval_layout * vp)  Line 1631 + 0xe bytes	C++
mozjs.dll!js::CallJSNative(JSContext * cx, int (JSContext *, unsigned int, js::Value *)* native, unsigned int argc, js::Value * vp)  Line 684 + 0xf bytes	C++
mozjs.dll!js::Interpret(JSContext * cx, JSStackFrame * entryFrame, unsigned int inlineCallCount, JSInterpMode interpMode)  Line 4765 + 0x21 bytes	C++
mozjs.dll!js::RunScript(JSContext * cx, JSScript * script, JSStackFrame * fp)  Line 665 + 0x11 bytes	C++
mozjs.dll!js::Invoke(JSContext * cx, const js::CallArgs & argsRef, unsigned int flags)  Line 768 + 0x11 bytes	C++
mozjs.dll!js::ExternalInvoke(JSContext * cx, const js::Value & thisv, const js::Value & fval, unsigned int argc, js::Value * argv, js::Value * rval)  Line 881 + 0xf bytes	C++
mozjs.dll!js::ExternalInvoke(JSContext * cx, JSObject * obj, const js::Value & fval, unsigned int argc, js::Value * argv, js::Value * rval)  Line 954 + 0x2a bytes	C++
mozjs.dll!JS_CallFunctionValue(JSContext * cx, JSObject * obj, jsval_layout fval, unsigned int argc, jsval_layout * argv, jsval_layout * rval)  Line 4961 + 0x38 bytes	C++
xul.dll!nsXPCWrappedJSClass::CallMethod(nsXPCWrappedJS * wrapper, unsigned short methodIndex, const XPTMethodDescriptor * info, nsXPTCMiniVariant * nativeParams)  Line 1694 + 0x35 bytes	C++
xul.dll!nsXPCWrappedJS::CallMethod(unsigned short methodIndex, const XPTMethodDescriptor * info, nsXPTCMiniVariant * params)  Line 578	C++
xul.dll!PrepareAndDispatch(nsXPTCStubBase * self, unsigned int methodIndex, unsigned int * args, unsigned int * stackBytesToPop)  Line 114 + 0x21 bytes	C++
xul.dll!SharedStub()  Line 142	C++
xul.dll!nsHttpChannel::CallOnStartRequest()  Line 770 + 0x44 bytes	C++
xul.dll!nsHttpChannel::ContinueProcessNormal(unsigned int rv)  Line 1228 + 0x8 bytes	C++
xul.dll!nsHttpChannel::ProcessNormal()  Line 1166	C++
xul.dll!nsHttpChannel::ProcessResponse()  Line 1052 + 0x8 bytes	C++
xul.dll!nsHttpChannel::OnStartRequest(nsIRequest * request, nsISupports * ctxt)  Line 3805 + 0xe bytes	C++
xul.dll!nsInputStreamPump::OnStateStart()  Line 441 + 0x2c bytes	C++
xul.dll!nsInputStreamPump::OnInputStreamReady(nsIAsyncInputStream * stream)  Line 397 + 0xb bytes	C++
xul.dll!nsInputStreamReadyEvent::Run()  Line 113	C++
xul.dll!nsThread::ProcessNextEvent(int mayWait, int * result)  Line 609 + 0x19 bytes	C++
xul.dll!NS_ProcessNextEvent_P(nsIThread * thread, int mayWait)  Line 250 + 0x16 bytes	C++
xul.dll!mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate * aDelegate)  Line 134 + 0xe bytes	C++
xul.dll!MessageLoop::RunInternal()  Line 220	C++
xul.dll!MessageLoop::RunHandler()  Line 203	C++
xul.dll!MessageLoop::Run()  Line 177	C++
xul.dll!nsBaseAppShell::Run()  Line 187	C++
xul.dll!nsAppShell::Run()  Line 243 + 0x9 bytes	C++
xul.dll!nsAppStartup::Run()  Line 191 + 0x1c bytes	C++
xul.dll!XRE_main(int argc, char * * argv, const nsXREAppData * aAppData)  Line 3682 + 0x25 bytes	C++
firefox.exe!NS_internal_main(int argc, char * * argv)  Line 158 + 0x12 bytes	C++
firefox.exe!wmain(int argc, wchar_t * * argv)  Line 129 + 0xd bytes	C++
firefox.exe!__tmainCRTStartup()  Line 552 + 0x19 bytes	C
firefox.exe!wmainCRTStartup()  Line 371	C

Comment 1

[(dormant account)]

Did this actually happen in the wild? Julian pointed this out from his valgrind exploration today.

Comment 2

[Jason Orendorff [:jorendorff]]

Yeah, I had the ES5 spec loaded (about 1.5MB of HTML) and was using jQuery on it. firefox bloated to about 1.5GB and then aborted.

Comment 3

[(dormant account)]

a) What sort of large buffer size is safer? I don't really want to cap it at 64K

b)Sounds like we should mod bufferedoutputstream to use fallible malloc.

Comment 5

[Julian Seward [:jseward]]

(In reply to comment #1)
Yeah, I also saw that; stats at
https://bugzilla.mozilla.org/show_bug.cgi?id=610064#c0

Not only is the buffer 3MB, which seems a bit extravagant,
at least in the case I was looking at 98% of the bytes in
it were neither read nor written.

(In reply to comment #3)
> I don't really want to cap it at 64K

Why not?  Originally it was 2048 bytes.  Changing it to 64k will
reduce the per-buffer-fill costs by a factor of 32 without 
spiking heap usage in this way.

Comment 6

[(dormant account)]

(In reply to comment #5)
> (In reply to comment #1)
> Yeah, I also saw that; stats at
> https://bugzilla.mozilla.org/show_bug.cgi?id=610064#c0
> 
> Not only is the buffer 3MB, which seems a bit extravagant,
> at least in the case I was looking at 98% of the bytes in
> it were neither read nor written.
> 
> (In reply to comment #3)
> > I don't really want to cap it at 64K
> 
> Why not?  Originally it was 2048 bytes.  Changing it to 64k will
> reduce the per-buffer-fill costs by a factor of 32 without 
> spiking heap usage in this way.

I'm not concerned about fill-costs. This was a workaround for crappy operating systems fragmenting the filesystem more if lots of little writes are used(big buffer is a way to reduce the number of write calls). I was curious to see if 3mb allocation would cause any issues, now I know :)

I'll change it to 64K and use another way of dealing with the filesystem.

Comment 7

[(dormant account)]

Attachment: oom fix

I'll let bufferedwriter be for now

Comment 8

[Julian Seward [:jseward]]

Can I nominate this fix to make it into Fx 4.0 ?  Considering that
the mobile folks just this week have been battling some space
problems, iirc.

Comment 9

[Dão Gottwald [:dao]]

http://hg.mozilla.org/mozilla-central/rev/f8cfc4c02709