1699321 - DMD crashes @HashTable<...>::putNewInfallible on Windows debug builds

Status: RESOLVED FIXED

(Core :: DMD, defect)

Description

[Mike Hommey [:glandium]]

STR:

• Build with --enable-debug for either Windows 32-bits or Windows 64-bits.
• Run ./mach run --dmd
• In the browser, open about:memory
• In the "Save DMD output" box, click "Save"

Actual result:

• The browser crashes on this assertion: https://searchfox.org/mozilla-central/rev/b7b156e53643f0237f3e98a76e5fc7fa9e3b4e71/mfbt/HashTable.h#2160
  coming from this: https://searchfox.org/mozilla-central/rev/b7b156e53643f0237f3e98a76e5fc7fa9e3b4e71/memory/replace/dmd/DMD.cpp#1035

Expected result:

• DMD reports and a running browser

This doesn't happen on neither mac nor linux.

Comment 1

[Mike Hommey [:glandium]]

The full stack trace that I could get from this crash:

#01: MozStackWalkThread (c:\Users\glandium\gecko\mozglue\misc\StackWalk.cpp:412)
#02: WalkTheStack (c:\Users\glandium\gecko\mozglue\misc\StackWalk.cpp:967)
#03: MOZ_ReportAssertionFailure (c:\Users\glandium\gecko\obj-x86_64-pc-mingw32\dist\include\mozilla\Assertions.h:114)
#04: mozilla::detail::HashTable<const mozilla::dmd::LiveBlock,mozilla::HashSet<mozilla::dmd::LiveBlock,mozilla::dmd::LiveBlock,mozilla::dmd::InfallibleAllocPolicy>::SetHashPolicy,mozilla::dmd::InfallibleAllocPolicy>::putNewInfallible<mozilla::dmd::LiveBlock &> (c:\Users\glandium\gecko\obj-x86_64-pc-mingw32\dist\include\mozilla\HashTable.h:2160)
#05: mozilla::dmd::AllocCallback (c:\Users\glandium\gecko\memory\replace\dmd\DMD.cpp:1060)
#06: replace_malloc (c:\Users\glandium\gecko\memory\replace\dmd\DMD.cpp:1109)
#07: nsTSubstring<char>::StartBulkWriteImpl (c:\Users\glandium\gecko\xpcom\string\nsTSubstring.cpp:202)
#08: nsTSubstring<char>::Assign (c:\Users\glandium\gecko\xpcom\string\nsTSubstring.cpp:409)
#09: nsTSubstring<char>::Assign (c:\Users\glandium\gecko\xpcom\string\nsTSubstring.cpp:455)
#10: mozilla::dom::quota::ScopedLogExtraInfo::ScopedLogExtraInfo<6> (c:\Users\glandium\gecko\obj-x86_64-pc-mingw32\dist\include\mozilla\dom\quota\QuotaCommon.h:1056)
#11: mozilla::dom::quota::CachingDatabaseConnection::GetCachedStatement (c:\Users\glandium\gecko\dom\quota\CachingDatabaseConnection.cpp:39)
#12: mozilla::dom::quota::CachingDatabaseConnection::BorrowCachedStatement (c:\Users\glandium\gecko\dom\quota\CachingDatabaseConnection.cpp:75)
#13: mozilla::dom::quota::CachingDatabaseConnection::ExecuteCachedStatement (c:\Users\glandium\gecko\dom\quota\CachingDatabaseConnection.cpp:82)
#14: mozilla::dom::indexedDB::`anonymous namespace'::DatabaseConnection::CheckpointInternal (c:\Users\glandium\gecko\dom\indexedDB\ActorsParent.cpp:7069)
#15: mozilla::dom::indexedDB::`anonymous namespace'::DatabaseConnection::DoIdleProcessing (c:\Users\glandium\gecko\dom\indexedDB\ActorsParent.cpp:7119)
#16: mozilla::dom::indexedDB::`anonymous namespace'::ConnectionPool::IdleConnectionRunnable::Run (c:\Users\glandium\gecko\dom\indexedDB\ActorsParent.cpp:8658)
#17: nsThread::ProcessNextEvent (c:\Users\glandium\gecko\xpcom\threads\nsThread.cpp:1152)
#18: NS_ProcessNextEvent (c:\Users\glandium\gecko\xpcom\threads\nsThreadUtils.cpp:548)
#19: mozilla::dom::indexedDB::`anonymous namespace'::ConnectionPool::ThreadRunnable::Run (c:\Users\glandium\gecko\dom\indexedDB\ActorsParent.cpp:8872)
#20: nsThread::ProcessNextEvent (c:\Users\glandium\gecko\xpcom\threads\nsThread.cpp:1152)
#21: NS_ProcessNextEvent (c:\Users\glandium\gecko\xpcom\threads\nsThreadUtils.cpp:548)
#22: mozilla::ipc::MessagePumpForNonMainThreads::Run (c:\Users\glandium\gecko\ipc\glue\MessagePump.cpp:303)
#23: MessageLoop::RunHandler (c:\Users\glandium\gecko\ipc\chromium\src\base\message_loop.cc:329)
#24: MessageLoop::Run (c:\Users\glandium\gecko\ipc\chromium\src\base\message_loop.cc:311)
#25: nsThread::ThreadFunc (c:\Users\glandium\gecko\xpcom\threads\nsThread.cpp:393)
#26: _PR_NativeRunThread (c:\Users\glandium\gecko\nsprpub\pr\src\threads\combined\pruthr.c:408)
#27: pr_root (c:\Users\glandium\gecko\nsprpub\pr\src\md\windows\w95thred.c:140)
#28: beginthreadex[C:\Windows\System32\ucrtbase.dll +0x210b2]
#29: BaseThreadInitThunk[C:\Windows\System32\KERNEL32.DLL +0x17c24]
#30: RtlUserThreadStart[C:\Windows\SYSTEM32\ntdll.dll +0x6d721]

However, this trace is always different.

If I add a lookup before the failing putNew, and use --stacks=full, I get a LiveBlock with the following stack trace:

#00: MozStackWalk (c:\Users\glandium\gecko\mozglue\misc\StackWalk.cpp:433)
#01: mozilla::dmd::StackTrace::Get (c:\Users\glandium\gecko\memory\replace\dmd\DMD.cpp:679)
#02: mozilla::dmd::AllocCallback (c:\Users\glandium\gecko\memory\replace\dmd\DMD.cpp:1040)
#03: replace_malloc (c:\Users\glandium\gecko\memory\replace\dmd\DMD.cpp:1109)
#04: nsTSubstring<char16_t>::StartBulkWriteImpl (c:\Users\glandium\gecko\xpcom\string\nsTSubstring.cpp:202)
#05: nsTSubstring<char16_t>::Assign (c:\Users\glandium\gecko\xpcom\string\nsTSubstring.cpp:409)
#06: nsTSubstring<char16_t>::Assign (c:\Users\glandium\gecko\xpcom\string\nsTSubstring.cpp:455)
#07: `anonymous namespace'::WinIOAutoObservation::WinIOAutoObservation (c:\Users\glandium\gecko\xpcom\build\PoisonIOInterposerWin.cpp:154)
#08: `anonymous namespace'::InterposedNtCreateFile (c:\Users\glandium\gecko\xpcom\build\PoisonIOInterposerWin.cpp:270)
#09: CreateFileW[C:\Windows\System32\KERNELBASE.dll +0x2d8d4]
#10: CreateFileW[C:\Windows\System32\KERNELBASE.dll +0x2d5c6]
#11: AddSIDToBoundaryDescriptor[C:\Windows\System32\KERNELBASE.dll +0x7d048]
#12: GetFinalPathNameByHandleW[C:\Windows\System32\KERNELBASE.dll +0x355db]
#13: mozilla::widget::WinUtils::ResolveJunctionPointsAndSymLinks (c:\Users\glandium\gecko\widget\windows\WinUtils.cpp:1959)
#14: mozilla::widget::WinUtils::ResolveJunctionPointsAndSymLinks (c:\Users\glandium\gecko\widget\windows\WinUtils.cpp:1990)
#15: mozilla::net::ExtensionProtocolHandler::NewStream (c:\Users\glandium\gecko\netwerk\protocol\res\ExtensionProtocolHandler.cpp:766)
#16: mozilla::net::NeckoParent::RecvGetExtensionStream (c:\Users\glandium\gecko\netwerk\ipc\NeckoParent.cpp:801)
#17: mozilla::net::PNeckoParent::OnMessageReceived (c:\Users\glandium\gecko\obj-x86_64-pc-mingw32\ipc\ipdl\PNeckoParent.cpp:3302)
#18: mozilla::dom::PContentParent::OnMessageReceived (c:\Users\glandium\gecko\obj-x86_64-pc-mingw32\ipc\ipdl\PContentParent.cpp:6764)
#19: mozilla::ipc::MessageChannel::DispatchAsyncMessage (c:\Users\glandium\gecko\ipc\glue\MessageChannel.cpp:2158)
#20: mozilla::ipc::MessageChannel::DispatchMessage (c:\Users\glandium\gecko\ipc\glue\MessageChannel.cpp:2081)
#21: mozilla::ipc::MessageChannel::RunMessage (c:\Users\glandium\gecko\ipc\glue\MessageChannel.cpp:1930)
#22: mozilla::ipc::MessageChannel::MessageTask::Run (c:\Users\glandium\gecko\ipc\glue\MessageChannel.cpp:1962)
#23: mozilla::RunnableTask::Run (c:\Users\glandium\gecko\xpcom\threads\TaskController.cpp:473)

Here too the trace changes every time, except for the first few frames up to InterposedNtCreateFile, so this has to be related to file interposition.

Comment 2

[Simon Giesecke [:sg] [he/him]]

So IIUC, this would happen if some malloc is intercepted (and an entry for the allocated address key is inserted into gLiveBlockTable), but the corresponding free is not intercepted (so the entry is not removed from gLiveBlockTable), and then another malloc allocates memory at the same address (attempting to insert an entry with the same address key into gLiveBlockTable, which triggers the assertion failure). Right?

It seems there are already precautions for the opposite case (malloc was not intercepted, but free is) at https://searchfox.org/mozilla-central/rev/f07a609a76136ef779c65185165ff5ac513cc172/memory/replace/dmd/DMD.cpp#1052-1055, so maybe this situation should also be taken care of?

Comment 3

[Mike Hommey [:glandium]]

Yeah, I /think/ what's happening is that the IO interception ends up freeing memory while the JSON is being created, which happens with DMD interceptions disabled/blocked, so the LiveBlockTable is not updated for that freed memory. I'll test this theory tomorrow.

Comment 4

[Mike Hommey [:glandium]]

I confirmed this is what happens. The sad part is that IO interception is enabled during xpcshell tests, but doesn't cause a failure to the dmd test, presumably because nothing is doing IO on a separate thread.

Comment 5

[Mike Hommey [:glandium]]

Attachment: Bug 1699321 - Avoid DMD crashes on Windows debug builds.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b1c2bbaef047
Avoid DMD crashes on Windows debug builds. r=mccr8

Comment 7

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/b1c2bbaef047