Closed Bug 1282185 Opened 3 years ago Closed 3 years ago

DMD crashes when --mode=cumulative is set

Categories

(Core :: DMD, defect)

defect
Not set

Tracking

()

RESOLVED FIXED
mozilla50
Tracking Status
firefox50 --- fixed

People

(Reporter: erahm, Assigned: njn)

References

Details

Attachments

(2 files)

At least on OSX with a debug build DMD crashes pretty much immediately when cumulative mode is enabled.

> Process 2355 launched: '/Users/ericrahm/dev/mozilla-central/obj-x86_64-apple-darwin15.3.0-debug/dist/NightlyDebug.app/Contents/MacOS/firefox' (x86_64)
> DMD[2355] $DMD = '--mode=cumulative'
> Assertion failure: AllocStackTrace(), at /Users/ericrahm/dev/mozilla-central/memory/replace/dmd/DMD.cpp:1068
> Process 2355 stopped
> * thread #1: tid = 0x10cd3, 0x000000010000a5f9 libdmd.dylib`mozilla::dmd::FreeCallback(void*, mozilla::dmd::Thread*, mozilla::dmd::DeadBlock*) [inlined] mozilla::dmd::DeadBlock::DeadBlock(this=0x00007fff5fbff1f0, aLb=<unavailable>) + 52 at DMD.cpp:1068, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
>    frame #0: 0x000000010000a5f9 libdmd.dylib`mozilla::dmd::FreeCallback(void*, mozilla::dmd::Thread*, mozilla::dmd::DeadBlock*) [inlined] mozilla::dmd::DeadBlock::DeadBlock(this=0x00007fff5fbff1f0, aLb=<unavailable>) + 52 at DMD.cpp:1068
>    1065	    , mSlopSize(aLb.SlopSize())
>    1066	    , mAllocStackTrace(aLb.AllocStackTrace())
>    1067	  {
> -> 1068	    MOZ_ASSERT(AllocStackTrace());
>    1069	  }
>    1070	
>    1071	  ~DeadBlock() {}
> (lldb) bt
> * thread #1: tid = 0x10cd3, 0x000000010000a5f9 libdmd.dylib`mozilla::dmd::FreeCallback(void*, mozilla::dmd::Thread*, mozilla::dmd::DeadBlock*) [inlined] mozilla::dmd::DeadBlock::DeadBlock(this=0x00007fff5fbff1f0, aLb=<unavailable>) + 52 at DMD.cpp:1068, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
>   * frame #0: 0x000000010000a5f9 libdmd.dylib`mozilla::dmd::FreeCallback(void*, mozilla::dmd::Thread*, mozilla::dmd::DeadBlock*) [inlined] mozilla::dmd::DeadBlock::DeadBlock(this=0x00007fff5fbff1f0, aLb=<unavailable>) + 52 at DMD.cpp:1068
>     frame #1: 0x000000010000a5c5 libdmd.dylib`mozilla::dmd::FreeCallback(void*, mozilla::dmd::Thread*, mozilla::dmd::DeadBlock*) [inlined] mozilla::dmd::DeadBlock::DeadBlock(aLb=<unavailable>) at DMD.cpp:1067
>     frame #2: 0x000000010000a5c5 libdmd.dylib`mozilla::dmd::FreeCallback(aPtr=<unavailable>, aT=<unavailable>, aDeadBlock=<unavailable>) + 1525 at DMD.cpp:1229
>     frame #3: 0x000000010000a6bb libdmd.dylib`::replace_free(aPtr=0x000000010200f068) + 107 at DMD.cpp:1387
>     frame #4: 0x00000001000015db firefox`NS_CompareVersions(A=<unavailable>, B=<unavailable>) + 427 at MacQuirks.h:200
>     frame #5: 0x00000001000017fc firefox`main [inlined] TriggerQuirks() + 92 at MacQuirks.h:222
>     frame #6: 0x00000001000017a0 firefox`main(argc=5, argv=0x00007fff5fbff7f0, envp=0x00007fff5fbff820) + 64 at nsBrowserApp.cpp:362
>     frame #7: 0x0000000100001424 firefox`start + 52
Depends on: 1282980
This happens on Linux as well. I wonder if we should just remove the assert. It seems like we're just seeing things that were allocated before our malloc hook is added, but freed after. Nick any thoughts?

> ./mach run --dmd --mode cumulative --debug
> ...
> DMD[6268] $DMD = '--mode=cumulative'
> Assertion failure: AllocStackTrace(), at /home/erahm/dev/mozilla-central/memory/replace/dmd/DMD.cpp:1068
> 
> Program received signal SIGSEGV, Segmentation fault.
> mozilla::dmd::DeadBlock::DeadBlock (this=<optimized out>, aLb=...) at /home/erahm/dev/mozilla-central/memory/replace/dmd/DMD.cpp:1068
> 1068	    MOZ_ASSERT(AllocStackTrace());
> (gdb) bt
> #0  mozilla::dmd::DeadBlock::DeadBlock (this=<optimized out>, aLb=...) at /home/erahm/dev/mozilla-central/memory/replace/dmd/DMD.cpp:1068
> #1  0x00007ffff7fe2818 in mozilla::dmd::FreeCallback (aPtr=<optimized out>, aT=<optimized out>, aDeadBlock=0x7fffffffa6a0)
>     at /home/erahm/dev/mozilla-central/memory/replace/dmd/DMD.cpp:1229
> #2  0x00007ffff7fe2ade in replace_free (aPtr=0x7ffff6ae7040) at /home/erahm/dev/mozilla-central/memory/replace/dmd/DMD.cpp:1387
> #3  0x00007ffff7ded61a in _dl_scope_free (old=0x7ffff6ae7040) at dl-scope.c:31
> #4  0x00007ffff7deeff0 in dl_open_worker (a=a@entry=0x7fffffffa908) at dl-open.c:490
> #5  0x00007ffff7de9fc4 in _dl_catch_error (objname=objname@entry=0x7fffffffa8f8, errstring=errstring@entry=0x7fffffffa900, 
>     mallocedp=mallocedp@entry=0x7fffffffa8f0, operate=operate@entry=0x7ffff7dee960 <dl_open_worker>, args=args@entry=0x7fffffffa908)
>     at dl-error.c:187
> #6  0x00007ffff7dee37b in _dl_open (
>     file=0x7fffffffbb80 "/home/erahm/dev/mozilla-central/obj-x86_64-pc-linux-gnu-clang/dist/bin/libssl3.so", mode=-2147483391, 
>     caller_dlopen=<optimized out>, nsid=-2, argc=4, argv=0x7fffffffddd8, env=0x7ffff6be7400) at dl-open.c:661
> #7  0x00007ffff79b902b in dlopen_doit (a=a@entry=0x7fffffffab20) at dlopen.c:66
> #8  0x00007ffff7de9fc4 in _dl_catch_error (objname=0x7ffff6b030d0, errstring=0x7ffff6b030d8, mallocedp=0x7ffff6b030c8, 
>     operate=0x7ffff79b8fd0 <dlopen_doit>, args=0x7fffffffab20) at dl-error.c:187
> #9  0x00007ffff79b962d in _dlerror_run (operate=operate@entry=0x7ffff79b8fd0 <dlopen_doit>, args=args@entry=0x7fffffffab20)
>     at dlerror.c:163
> #10 0x00007ffff79b90c1 in __dlopen (file=<optimized out>, mode=<optimized out>) at dlopen.c:87
> #11 0x0000000000421e49 in GetLibHandle (aDependentLib=<optimized out>)
>     at /home/erahm/dev/mozilla-central/xpcom/glue/standalone/nsXPCOMGlue.cpp:165
> #12 ReadDependentCB (aDependentLib=<optimized out>, aDoPreload=<optimized out>)
>     at /home/erahm/dev/mozilla-central/xpcom/glue/standalone/nsXPCOMGlue.cpp:214
> #13 XPCOMGlueLoad (aXPCOMFile=<optimized out>) at /home/erahm/dev/mozilla-central/xpcom/glue/standalone/nsXPCOMGlue.cpp:383
> #14 XPCOMGlueStartup (aXPCOMFile=<optimized out>) at /home/erahm/dev/mozilla-central/xpcom/glue/standalone/nsXPCOMGlue.cpp:485
> #15 0x0000000000404dc4 in InitXPCOMGlue (argv0=<optimized out>, xreDirectory=<optimized out>)
>     at /home/erahm/dev/mozilla-central/browser/app/nsBrowserApp.cpp:296
> #16 main (argc=4, argv=0x7fffffffddd8, envp=0x7fffffffde00) at /home/erahm/dev/mozilla-central/browser/app/nsBrowserApp.cpp:390
Flags: needinfo?(n.nethercote)
OS: Unspecified → All
Hardware: Unspecified → All
Removing that assert gets me further, but now it's decided to blow up when building the dmd output file name. It looks like the newly allocated chunk of memory is already in the hashtable.

> Assertion failure: !lookup(l).found(), at /home/erahm/dev/mozilla-central/obj-x86_64-pc-linux-gnu-clang/dist/include/js/HashTable.h:1746
> 
> #01: AsmJSFaultHandler(int, siginfo_t*, void*) (/home/erahm/dev/mozilla-central/js/src/asmjs/WasmSignalHandlers.cpp:1169)
> #02: __restore_rt (sigaction.c:?)
> #03: void js::detail::HashTable<mozilla::dmd::LiveBlock const, js::HashSet<mozilla::dmd::LiveBlock, mozilla::dmd::LiveBlock, mozilla::dmd::InfallibleAllocPolicy>::SetOps, mozilla::dmd::InfallibleAllocPolicy>::putNewInfallible<mozilla::dmd::LiveBlock&>(void const* const&, mozilla::dmd::LiveBlock&) (/home/erahm/dev/mozilla-central/obj-x86_64-pc-linux-gnu-clang/dist/include/js/HashTable.h:1746 (discriminator 6))
> #04: bool js::detail::HashTable<mozilla::dmd::LiveBlock const, js::HashSet<mozilla::dmd::LiveBlock, mozilla::dmd::LiveBlock, mozilla::dmd::InfallibleAllocPolicy>::SetOps, mozilla::dmd::InfallibleAllocPolicy>::putNew<mozilla::dmd::LiveBlock&>(void const* const&, mozilla::dmd::LiveBlock&) (/home/erahm/dev/mozilla-central/obj-x86_64-pc-linux-gnu-clang/dist/include/js/HashTable.h:1762)
> #05: mozilla::dmd::AllocCallback(void*, unsigned long, mozilla::dmd::Thread*) (/home/erahm/dev/mozilla-central/memory/replace/dmd/DMD.cpp:1213)
> #06: replace_malloc (/home/erahm/dev/mozilla-central/memory/replace/dmd/DMD.cpp:1287)
> #07: nsStringBuffer::Alloc(unsigned long) (/home/erahm/dev/mozilla-central/xpcom/string/nsSubstring.cpp:217)
> #08: already_AddRefed<nsStringBuffer>::take() (/home/erahm/dev/mozilla-central/obj-x86_64-pc-linux-gnu-clang/dist/include/mozilla/AlreadyAddRefed.h:116)
> #09: nsACString_internal::ReplacePrepInternal(unsigned int, unsigned int, unsigned int, unsigned int) (/home/erahm/dev/mozilla-central/xpcom/string/nsTSubstring.cpp:195)
> #10: nsACString_internal::ReplacePrep(unsigned int, unsigned int, unsigned int) (/home/erahm/dev/mozilla-central/xpcom/string/nsTSubstring.cpp:187)
> #11: nsACString_internal::Replace(unsigned int, unsigned int, char) (/home/erahm/dev/mozilla-central/xpcom/string/nsTSubstring.cpp:510)
> #12: nsACString_internal::Append(nsACString_internal const&) (/home/erahm/dev/mozilla-central/obj-x86_64-pc-linux-gnu-clang/dist/include/nsTSubstring.h:547)
> #13: nsLocalFile::AppendNative(nsACString_internal const&) (/home/erahm/dev/mozilla-central/xpcom/io/nsLocalFileUnix.cpp:520)
> #14: nsDumpUtils::OpenTempFile(nsACString_internal const&, nsIFile**, nsACString_internal const&, nsDumpUtils::Mode) (/home/erahm/dev/mozilla-central/xpcom/base/nsDumpUtils.cpp:484)
> #15: nsMemoryInfoDumper::OpenDMDFile(nsAString_internal const&, int, _IO_FILE**) (/home/erahm/dev/mozilla-central/xpcom/base/nsMemoryInfoDumper.cpp:791)
> #16: NS_warn_if_impl(bool, char const*, char const*, int) (/home/erahm/dev/mozilla-central/obj-x86_64-pc-linux-gnu-clang/dist/include/nsDebug.h:40)
> #17: nsMemoryReporterManager::EndProcessReport(unsigned int, bool) (/home/erahm/dev/mozilla-central/xpcom/base/nsMemoryReporterManager.cpp:1988)
> #18: nsMemoryReporterManager::EndReport() (/home/erahm/dev/mozilla-central/xpcom/base/nsMemoryReporterManager.cpp:1851)
> #19: nsRunnableFunction<nsMemoryReporterManager::DispatchReporter(nsIMemoryReporter*, bool, nsIMemoryReporterCallback*, nsISupports*, bool)::$_0>::Run() (/home/erahm/dev/mozilla-central/obj-x86_64-pc-linux-gnu-clang/dist/include/nsThreadUtils.h:277)
> #20: nsThread::ProcessNextEvent(bool, bool*) (/home/erahm/dev/mozilla-central/xpcom/threads/nsThread.cpp:1029)
> #21: NS_ProcessNextEvent(nsIThread*, bool) (/home/erahm/dev/mozilla-central/xpcom/glue/nsThreadUtils.cpp:290)
> #22: mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) (/home/erahm/dev/mozilla-central/ipc/glue/MessagePump.cpp:100)
> #23: ~AutoRunState (/home/erahm/dev/mozilla-central/ipc/chromium/src/base/message_loop.cc:493)
> #24: nsBaseAppShell::Run() (/home/erahm/dev/mozilla-central/widget/nsBaseAppShell.cpp:158)
> #25: nsAppStartup::Run() (/home/erahm/dev/mozilla-central/toolkit/components/startup/nsAppStartup.cpp:284)
> #26: XREMain::XRE_mainRun() (/home/erahm/dev/mozilla-central/toolkit/xre/nsAppRunner.cpp:4391)
> #27: XREMain::XRE_main(int, char**, nsXREAppData const*) (/home/erahm/dev/mozilla-central/toolkit/xre/nsAppRunner.cpp:4495)
> #28: XRE_main (/home/erahm/dev/mozilla-central/toolkit/xre/nsAppRunner.cpp:4603)
> #29: do_main(int, char**, char**, nsIFile*) (/home/erahm/dev/mozilla-central/browser/app/nsBrowserApp.cpp:254)
> #30: __libc_start_main (/build/eglibc-oGUzwX/eglibc-2.19/csu/libc-start.c:321)
> #31: _start (/home/erahm/dev/mozilla-central/obj-x86_64-pc-linux-gnu-clang/dist/bin/firefox)
The first assertion is just bogus since bug 1253512 landed; not sure why testing didn't flush it out already.

The second assertion while dumping doesn't depend on using cumulative mode. I haven't worked out the problem there yet.
Flags: needinfo?(n.nethercote)
> The second assertion while dumping doesn't depend on using cumulative mode.
> I haven't worked out the problem there yet.

There's a block that we allocate normally, and DMD records its existence. We then free it somehow with intercepts blocked, which means that DMD ignores the free, and so thinks the block is still alive. Then another block gets allocated at the same address and we hit the problem.

This only occurs to a single block.
> There's a block that we allocate normally, and DMD records its existence. We
> then free it somehow with intercepts blocked, which means that DMD ignores
> the free, and so thinks the block is still alive. Then another block gets
> allocated at the same address and we hit the problem.

It's the JSONWriter. It's created in nsMemoryInfoDumper::DumpDMDToFile() and then, because of the use of UniquePtr, it's destroyed inside DMD's Analyze(), where intercepts are blocked. I have a fix to come.
Since bug 1253512 landed, it's possible for DeadBlocks to lack an allocation
stack.
Attachment #8770441 - Flags: review?(erahm)
Assignee: nobody → n.nethercote
Status: NEW → ASSIGNED
Comment on attachment 8770441 [details] [diff] [review]
(part 1) - Remove a bogus assertion in DMD

Review of attachment 8770441 [details] [diff] [review]:
-----------------------------------------------------------------

r=me
Attachment #8770441 - Flags: review?(erahm) → review+
Comment on attachment 8770465 [details] [diff] [review]
(part 2) - Fix an assertion failure in DMD

Review of attachment 8770465 [details] [diff] [review]:
-----------------------------------------------------------------

r=me. Thanks for tracking this down.
Attachment #8770465 - Flags: review?(erahm) → review+
https://hg.mozilla.org/mozilla-central/rev/d8e2cf1f7079
https://hg.mozilla.org/mozilla-central/rev/2f8d5f889b47
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla50
You need to log in before you can comment on or make changes to this bug.