Closed Bug 1254649 Opened 9 years ago Closed 5 years ago

crash reporter can race on reading/writing from the annotation table

Categories

(Toolkit :: Crash Reporting, defect)

Product:
Toolkit
Component:
Crash Reporting
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED

People

(Reporter: froydnj, Unassigned)

Details

Found this on try: 11:07:54 INFO - Assertion failure: IsIdle(oldState), at /builds/slave/try-l64-d-00000000000000000000/build/src/xpcom/glue/PLDHashTable.h:132 11:07:54 INFO - Assertion failure: IsRead(oldState), at /builds/slave/try-l64-d-00000000000000000000/build/src/xpcom/glue/PLDHashTable.h:125 11:08:22 INFO - #01: CrashReporter::WriteExtraData [toolkit/crashreporter/nsExceptionHandler.cpp:2650] 11:08:22 INFO - #02: CrashReporter::WriteExtraForMinidump [toolkit/crashreporter/nsExceptionHandler.cpp:2690] 11:08:22 INFO - #03: CrashReporter::OnChildProcessDumpRequested [toolkit/crashreporter/nsExceptionHandler.cpp:2754] 11:08:22 INFO - #04: google_breakpad::CrashGenerationServer::ClientEvent(short) [toolkit/crashreporter/google-breakpad/src/client/linux/crash_generation/crash_generation_server.cc:279] 11:08:22 INFO - #05: google_breakpad::CrashGenerationServer::Run() [toolkit/crashreporter/google-breakpad/src/client/linux/crash_generation/crash_generation_server.cc:178] 11:08:22 INFO - #06: google_breakpad::CrashGenerationServer::ThreadMain(void*) [toolkit/crashreporter/google-breakpad/src/client/linux/crash_generation/crash_generation_server.cc:328] 11:08:22 INFO - #07: libpthread.so.0 + 0x7e9a 11:08:22 INFO - #08: libc.so.6 + 0xf338d 11:08:22 INFO - #01: PLDHashTable::Add(void const*, mozilla::fallible_t const&) [xpcom/glue/PLDHashTable.cpp:538] 11:08:22 INFO - #02: PLDHashTable::Add(void const*) [xpcom/glue/PLDHashTable.cpp:595] 11:08:22 INFO - #03: CrashReporter::AnnotateCrashReport [xpcom/glue/nsBaseHashtable.h:134] 11:08:22 INFO - #04: NS_InvokeByIndex [xpcom/reflect/xptcall/md/unix/xptcinvoke_x86_64_unix.cpp:182] 11:08:22 INFO - #05: CallMethodHelper::Call() [js/xpconnect/src/xpcprivate.h:859] 11:08:22 INFO - #06: XPCWrappedNative::CallMethod(XPCCallContext&, XPCWrappedNative::CallMode) [js/xpconnect/src/XPCWrappedNative.cpp:1367] 11:08:22 INFO - #07: XPC_WN_CallMethod(JSContext*, unsigned int, JS::Value*) [js/xpconnect/src/XPCWrappedNativeJSOps.cpp:1115] 11:08:22 INFO - #08: js::CallJSNative(JSContext*, bool (*)(JSContext*, unsigned int, JS::Value*), JS::CallArgs const&) [js/src/jscntxtinlines.h:236] 11:08:22 INFO - #09: js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) [js/src/vm/Interpreter.cpp:478] 11:08:22 INFO - #10: Interpret [js/src/vm/Interpreter.cpp:2802] 11:08:22 INFO - #11: js::RunScript(JSContext*, js::RunState&) [js/src/vm/Interpreter.cpp:428] 11:08:22 INFO - #12: js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) [js/src/vm/Interpreter.cpp:496] 11:08:22 INFO - #13: js::Invoke(JSContext*, JS::Value const&, JS::Value const&, unsigned int, JS::Value const*, JS::MutableHandle<JS::Value>) [js/src/vm/Interpreter.cpp:530] 11:08:22 INFO - #14: js::DirectProxyHandler::call(JSContext*, JS::Handle<JSObject*>, JS::CallArgs const&) const [js/public/RootingAPI.h:666] 11:08:22 INFO - #15: js::CrossCompartmentWrapper::call(JSContext*, JS::Handle<JSObject*>, JS::CallArgs const&) const [js/src/proxy/CrossCompartmentWrapper.cpp:289] 11:08:22 INFO - #16: js::Proxy::call(JSContext*, JS::Handle<JSObject*>, JS::CallArgs const&) [js/src/proxy/Proxy.cpp:391] 11:08:22 INFO - #17: js::proxy_Call(JSContext*, unsigned int, JS::Value*) [js/public/RootingAPI.h:666] 11:08:22 INFO - #18: js::CallJSNative(JSContext*, bool (*)(JSContext*, unsigned int, JS::Value*), JS::CallArgs const&) [js/src/jscntxtinlines.h:236] 11:08:22 INFO - #19: js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) [js/src/vm/Interpreter.cpp:466] 11:08:22 INFO - #20: Interpret [js/src/vm/Interpreter.cpp:2802] 11:08:22 INFO - #21: js::RunScript(JSContext*, js::RunState&) [js/src/vm/Interpreter.cpp:428] 11:08:22 INFO - #22: js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) [js/src/vm/Interpreter.cpp:496] 11:08:22 INFO - #23: js::fun_call(JSContext*, unsigned int, JS::Value*) [js/src/jsfun.cpp:1206] 11:08:22 INFO - #24: js::CallJSNative(JSContext*, bool (*)(JSContext*, unsigned int, JS::Value*), JS::CallArgs const&) [js/src/jscntxtinlines.h:236] 11:08:22 INFO - #25: js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) [js/src/vm/Interpreter.cpp:478] 11:08:22 INFO - #26: js::Invoke(JSContext*, JS::Value const&, JS::Value const&, unsigned int, JS::Value const*, JS::MutableHandle<JS::Value>) [js/src/vm/Interpreter.cpp:530] 11:08:22 INFO - #27: js::DirectProxyHandler::call(JSContext*, JS::Handle<JSObject*>, JS::CallArgs const&) const [js/public/RootingAPI.h:666] 11:08:22 INFO - #28: js::CrossCompartmentWrapper::call(JSContext*, JS::Handle<JSObject*>, JS::CallArgs const&) const [js/src/proxy/CrossCompartmentWrapper.cpp:289] 11:08:22 INFO - #29: js::Proxy::call(JSContext*, JS::Handle<JSObject*>, JS::CallArgs const&) [js/src/proxy/Proxy.cpp:391] 11:08:22 INFO - #30: js::proxy_Call(JSContext*, unsigned int, JS::Value*) [js/public/RootingAPI.h:666] 11:08:22 INFO - #31: js::CallJSNative(JSContext*, bool (*)(JSContext*, unsigned int, JS::Value*), JS::CallArgs const&) [js/src/jscntxtinlines.h:236] 11:08:22 INFO - #32: js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) [js/src/vm/Interpreter.cpp:466] 11:08:22 INFO - #33: Interpret [js/src/vm/Interpreter.cpp:2802] 11:08:22 INFO - #34: js::RunScript(JSContext*, js::RunState&) [js/src/vm/Interpreter.cpp:428] 11:08:22 INFO - #35: js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) [js/src/vm/Interpreter.cpp:496] 11:08:22 INFO - #36: js::Invoke(JSContext*, JS::Value const&, JS::Value const&, unsigned int, JS::Value const*, JS::MutableHandle<JS::Value>) [js/src/vm/Interpreter.cpp:530] 11:08:22 INFO - #37: JS_CallFunctionValue(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::HandleValueArray const&, JS::MutableHandle<JS::Value>) [js/src/jsapi.cpp:2845] 11:08:22 INFO - #38: nsXPCWrappedJSClass::CallMethod(nsXPCWrappedJS*, unsigned short, XPTMethodDescriptor const*, nsXPTCMiniVariant*) [js/xpconnect/src/XPCWrappedJSClass.cpp:1237] 11:08:22 INFO - #39: PrepareAndDispatch [xpcom/reflect/xptcall/md/unix/xptcstubs_x86_64_linux.cpp:124] 11:08:22 INFO - #40: NS_InvokeByIndex [xpcom/reflect/xptcall/md/unix/xptcinvoke_x86_64_unix.cpp:184] 11:08:22 INFO - ExceptionHandler::GenerateDump cloned child 2066 11:08:22 INFO - ExceptionHandler::SendContinueSignalToChild sent continue signal to child 11:08:22 INFO - ExceptionHandler::WaitForContinueSignal waiting for continue signal... 11:08:22 INFO - [Child 2042] ###!!! ABORT: Aborting on channel error.: file /builds/slave/try-l64-d-00000000000000000000/build/src/ipc/glue/MessageChannel.cpp, line 1824 11:08:22 INFO - #01: mozilla::ipc::ProcessLink::OnChannelError() [ipc/glue/MessageLink.cpp:428] 11:08:22 INFO - #02: IPC::Channel::ChannelImpl::OnFileCanReadWithoutBlocking(int) [ipc/chromium/src/chrome/common/ipc_channel_posix.cc:881] 11:08:22 INFO - #03: event_base_loop [ipc/chromium/src/third_party/libevent/event.c:1355] 11:08:22 INFO - #04: base::MessagePumpLibevent::Run(base::MessagePump::Delegate*) [ipc/chromium/src/base/message_pump_libevent.cc:370] 11:08:22 INFO - #05: MessageLoop::RunInternal() [ipc/chromium/src/base/message_loop.cc:235] 11:08:22 INFO - #06: MessageLoop::Run() [ipc/chromium/src/base/message_loop.cc:520] 11:08:22 INFO - #07: base::Thread::ThreadMain() [ipc/chromium/src/base/thread.cc:175] 11:08:22 INFO - #08: ThreadFunc [ipc/chromium/src/base/platform_thread_posix.cc:38] 11:08:22 INFO - #09: libpthread.so.0 + 0x7e9a 11:08:22 INFO - #10: libc.so.6 + 0xf338d 11:08:22 INFO - [Child 2042] ###!!! ABORT: Aborting on channel error.: file /builds/slave/try-l64-d-00000000000000000000/build/src/ipc/glue/MessageChannel.cpp, line 1824 11:08:22 INFO - Hit MOZ_CRASH() at /builds/slave/try-l64-d-00000000000000000000/build/src/memory/mozalloc/mozalloc_abort.cpp:33 11:08:22 INFO - TEST-INFO | Main app process: exit 11 Those assertions in PLDHashTable are there to try and ensure that we're not writing to the hash table at the same time we're iterating through it. The first assertions show that they caught something interesting inside the crash reporter: we're trying to add a crash annotation at the same time we're trying to write a minidump. Not sure if there's an easy solution here.
I didn't *think* anything was accessing the annotations off the main thread, but I can't say I know for sure nowadays.

The hash table was removed, it's now safe to add or remove annotations while iterating over existing ones.

Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.