Closed Bug 1299516 Opened 9 years ago Closed 5 years ago

Crash in shutdownhang | mozilla::dom::PBlobParent::DestroySubtree

Categories

(Toolkit :: General, defect, P2)

Product:
Toolkit
Component:
General
Version:
51 Branch
Platform:
x86
Windows 7
Type:
defect

Tracking

()

Status:
RESOLVED WORKSFORME
Tracking Flags:
Tracking Status
e10s + ---
firefox49 - wontfix
firefox50 --- wontfix
firefox51 --- wontfix
firefox52 --- wontfix
firefox-esr68 --- wontfix
firefox53 --- wontfix

People

(Reporter: alex_mayorga, Unassigned)

References

Details

(Keywords: crash, nightly-community)

Crash Data

This bug was filed from the Socorro interface and is report bp-2a288f86-70e2-4b4c-b8af-4c7562160831. ============================================================= ¡Hola! Just crashed like this on Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:51.0) Gecko/20100101 Firefox/51.0 ID:20160831030224 CSet: 506facea63169a29e04eb140663da1730052db64: bp-2a288f86-70e2-4b4c-b8af-4c7562160831 31/08/2016 09:25 a.m. Found 4365 more crashes in the past week at https://crash-stats.mozilla.com/signature/?product=Firefox&signature=shutdownhang%20%7C%20mozilla%3A%3A%60anonymous%20namespace%27%27%3A%3ARunWatchdog dating back to Firefox 37.0.2 so filing a bug and guessing a component. ¡Gracias! Alex Crashing Thread (55) Frame Module Signature Source 0 xul.dll mozilla::`anonymous namespace'::RunWatchdog toolkit/components/terminator/nsTerminator.cpp:158 1 nss3.dll PR_NativeRunThread nsprpub/pr/src/threads/combined/pruthr.c:397 2 nss3.dll pr_root nsprpub/pr/src/md/windows/w95thred.c:95 3 ucrtbase.dll crt_at_quick_exit 4 kernel32.dll BaseThreadInitThunk 5 ntdll.dll RtlUserThreadStart
[Tracking Requested - why for this release]: This has risen to #1 to crash on 49 Beta 8. It has been present in other releases as well.
status-firefox49: --- → affected
status-firefox50: --- → affected
tracking-firefox49: --- → ?
This is also currently the #2 top browser crash on 48.0.2.
Keywords: topcrash
See also Bug 1299769 where apparently shutdownhang signatures are incorrectly generated.
See Also: → 1299769
David, can you help with this? Thanks
Flags: needinfo?(dbolter)
Yep. Benjamin is this basically the crash we expect to see when FF takes too long to shutdown and we force a crash? Anything we can investigate?
Flags: needinfo?(dbolter) → needinfo?(benjamin)
These crashes need to be investigated by what the main thread was doing (not the watchdog thread, which is just doing its job crashing on purpose). e.g. the crash from comment 0 the main thread is firing shutdown observers, which are spinning a nested event loop (so far relatively normal, although I hate the nested-eventloop shutdown set we're using). Then we're processing a mozilla::ipc::MessageChannel::OnNotifyMaybeChannelError() from a content process that has shut down earlier. I can't be sure whether this is "normal" shutdown or abnormal. That triggers mozilla::dom::PContentParent::OnChannelError() which tears down the IPDL actor tree. Which shouldn't take a long time. So either tearing down the actor tree is taking far longer than it should/caused an infinite loop, or we just happened to hit the 60-second shutdown timer at this random point in time. In case it's not clear, these are hard to debug ;-) One thing that might make it better is to annotate when we started the shutdown timer and at various shutdown milestones after that, so that we can know whether particular phases of shutdown are taking a long time. A went through a few others and the stacks are broken-enough that I couldn't figure out what's going on. I'd have to load them in VS to see if that has better stacks.
Flags: needinfo?(benjamin)
Jimm, do you know who might be able to investigate the cause of these shutdown issues? (I think your team has chases stuff like this before?)
Flags: needinfo?(jmathies)
there was a ~24 hour window when all shutdown hangs were showing up under the same signature. now that bug 1299769 got fixed and all shutdownhang reports get reprocessed, the particular crash from comment #0 turns out to be [@ shutdownhang | mozilla::dom::PBlobParent::DestroySubtree] which is of quite minor volume and probably doesn't need tracking for 49 anymore.
Crash Signature: [@ shutdownhang | mozilla::`anonymous namespace''::RunWatchdog] → [@ shutdownhang | mozilla::dom::PBlobParent::DestroySubtree ]
tracking-firefox49: + → ?
Summary: Crash in shutdownhang | mozilla::`anonymous namespace''::RunWatchdog → Crash in shutdownhang | mozilla::dom::PBlobParent::DestroySubtree
I see three reports across 7 days so this isn't critical. Kanru's team has been looking at shutdown crashes with their stability work. Marking the bug such that it'll get on their radar.
tracking-e10s: --- → +
Flags: needinfo?(jmathies)
Priority: -- → P2
(In reply to Jim Mathies [:jimm] from comment #10) > I see three reports across 7 days so this isn't critical. > > Kanru's team has been looking at shutdown crashes with their stability work. > Marking the bug such that it'll get on their radar. Thanks. Note this was still called "Crash in shutdownhang | mozilla::`anonymous namespace''::RunWatchdog" when I asked, which is a topcrash (although assumed to be many types of shutdown hangs)... is there investigation (via Kanru?) happening as per comment 6 somewhere else? Do we still want a "Crash in shutdownhang | mozilla::`anonymous namespace''::RunWatchdog" (perhaps meta) bug?
Flags: needinfo?(jmathies)
Sounds like maybe this is from bug 1299769 and the signatures were incorrectly generated.
tracking-firefox49: ? → -
I don't see why we would spend time on this, low volume crash appears to be a shutdownhang, which the shutdown hang thread catches. https://crash-stats.mozilla.com/signature/?signature=shutdownhang%20%7C%20mozilla%3A%3Adom%3A%3APBlobParent%3A%3ADestroySubtree&_columns=date&_columns=product&_columns=version&_columns=build_id&_columns=platform&_columns=reason&_columns=address&_sort=-date&page=1#reports 13 crashes over 7 days
Flags: needinfo?(jmathies)
Keywords: topcrash
(In reply to David Bolter [:davidb] from comment #11) > (In reply to Jim Mathies [:jimm] from comment #10) > > I see three reports across 7 days so this isn't critical. > > > > Kanru's team has been looking at shutdown crashes with their stability work. > > Marking the bug such that it'll get on their radar. > > Thanks. Note this was still called "Crash in shutdownhang | > mozilla::`anonymous namespace''::RunWatchdog" when I asked, which is a > topcrash (although assumed to be many types of shutdown hangs)... is there > investigation (via Kanru?) happening as per comment 6 somewhere else? > > Do we still want a "Crash in shutdownhang | mozilla::`anonymous > namespace''::RunWatchdog" (perhaps meta) bug? The reports should have the crashing (hung) thread as the main signature. So namespace''::RunWatchdog should show up.
(In reply to Jim Mathies [:jimm] from comment #14) > (In reply to David Bolter [:davidb] from comment #11) > > (In reply to Jim Mathies [:jimm] from comment #10) > > > I see three reports across 7 days so this isn't critical. > > > > > > Kanru's team has been looking at shutdown crashes with their stability work. > > > Marking the bug such that it'll get on their radar. > > > > Thanks. Note this was still called "Crash in shutdownhang | > > mozilla::`anonymous namespace''::RunWatchdog" when I asked, which is a > > topcrash (although assumed to be many types of shutdown hangs)... is there > > investigation (via Kanru?) happening as per comment 6 somewhere else? > > > > Do we still want a "Crash in shutdownhang | mozilla::`anonymous > > namespace''::RunWatchdog" (perhaps meta) bug? > > The reports should have the crashing (hung) thread as the main signature. So > namespace''::RunWatchdog should show up. s/should/shouldn't
OK. Setting wontfix for 49 (for "Crash in shutdownhang | mozilla::dom::PBlobParent::DestroySubtree"). Thanks.
status-firefox49: affected → wontfix
¡Hola! FWIW This is low volume but still a thing on Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:52.0) Gecko/20100101 Firefox/52.0 ID:20161024030205 CSet: 215f9686117673a2c914ed207bc7da9bb8d741ad bp-9b1bab88-41a4-434b-967d-508922161025 25/10/2016 08:56 a.m. ¡Gracias! Alex Crashing Thread (0) Frame Module Signature Source 0 xul.dll mozilla::dom::PBlobParent::DestroySubtree(mozilla::ipc::IProtocolManager<mozilla::ipc::IProtocol>::ActorDestroyReason) obj-firefox/ipc/ipdl/PBlobParent.cpp:603 1 xul.dll mozilla::dom::PContentParent::DestroySubtree(mozilla::ipc::IProtocolManager<mozilla::ipc::IProtocol>::ActorDestroyReason) obj-firefox/ipc/ipdl/PContentParent.cpp:9787 2 xul.dll mozilla::dom::PContentParent::OnChannelError() obj-firefox/ipc/ipdl/PContentParent.cpp:9694 3 xul.dll mozilla::dom::ContentParent::OnChannelError() dom/ipc/ContentParent.cpp:1634 4 xul.dll mozilla::ipc::MessageChannel::NotifyMaybeChannelError() ipc/glue/MessageChannel.cpp:2104 5 xul.dll mozilla::ipc::MessageChannel::OnNotifyMaybeChannelError() ipc/glue/MessageChannel.cpp:2133 6 xul.dll mozilla::detail::RunnableMethodImpl<void ( mozilla::ipc::MessageChannel::*)(void), 0, 1>::Run() obj-firefox/dist/include/nsThreadUtils.h:764 7 xul.dll nsThread::ProcessNextEvent(bool, bool*) xpcom/threads/nsThread.cpp:1082 8 xul.dll nsThread::Shutdown() xpcom/threads/nsThread.cpp:941 9 xul.dll mozilla::detail::RunnableMethodImpl<void ( mozilla::storage::Connection::*)(nsIThread*), 1, 0, nsCOMPtr<nsIThread> >::Run() obj-firefox/dist/include/nsThreadUtils.h:764 10 xul.dll nsThread::ProcessNextEvent(bool, bool*) xpcom/threads/nsThread.cpp:1082 11 xul.dll nsThread::Shutdown() xpcom/threads/nsThread.cpp:941 12 xul.dll mozilla::detail::RunnableMethodImpl<void ( mozilla::storage::Connection::*)(nsIThread*), 1, 0, nsCOMPtr<nsIThread> >::Run() obj-firefox/dist/include/nsThreadUtils.h:764 13 xul.dll nsThread::ProcessNextEvent(bool, bool*) xpcom/threads/nsThread.cpp:1082 14 xul.dll nsThread::Shutdown() xpcom/threads/nsThread.cpp:941 15 xul.dll mozilla::DataStorage::Observe(nsISupports*, char const*, char16_t const*) security/manager/ssl/DataStorage.cpp:886 16 xul.dll nsObserverList::NotifyObservers(nsISupports*, char const*, char16_t const*) xpcom/ds/nsObserverList.cpp:112 17 xul.dll nsObserverService::NotifyObservers(nsISupports*, char const*, char16_t const*) xpcom/ds/nsObserverService.cpp:281 18 xul.dll nsXREDirProvider::DoShutdown() toolkit/xre/nsXREDirProvider.cpp:1250 19 xul.dll ScopedXPCOMStartup::~ScopedXPCOMStartup() toolkit/xre/nsAppRunner.cpp:1396 20 xul.dll mozilla::UniquePtr<ScopedXPCOMStartup, mozilla::DefaultDelete<ScopedXPCOMStartup> >::reset(ScopedXPCOMStartup*) obj-firefox/dist/include/mozilla/UniquePtr.h:345 21 xul.dll XREMain::XRE_main(int, char** const, nsXREAppData const*) toolkit/xre/nsAppRunner.cpp:4611 22 xul.dll XRE_main toolkit/xre/nsAppRunner.cpp:4674 23 firefox.exe do_main browser/app/nsBrowserApp.cpp:282 24 firefox.exe wmain toolkit/xre/nsWindowsWMain.cpp:115 25 firefox.exe __scrt_common_main_seh f:/dd/vctools/crt/vcstartup/src/startup/exe_common.inl:253 26 kernel32.dll BaseThreadInitThunk 27 ntdll.dll RtlUserThreadStart
status-firefox52: --- → affected
I hit this today, and part of the cause may be that Test Pilot's vertical tabs extension triggers some leaky behavior of blobs, which then causes us to accumulate lots of PBlob* things, which then causes the shutdown hang...
¡Hola! Just crashed like this on Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:53.0) Gecko/20100101 Firefox/53.0 ID:20161201030205 CSet: cd4cdcc9ad6c45dad8b8d8c0d40e459db2bca8a1 bp-7c54c745-6be7-4452-8a4c-69c902161201 12/1/16 9:35 AM As Nathan stated, this install does have Test Pilot's Tab Center and Activity Stream enabled. ¡Gracias! Alex Crashing Thread (0) Frame Module Signature Source 0 xul.dll mozilla::dom::PBlobParent::DestroySubtree(mozilla::ipc::IProtocol::ActorDestroyReason) obj-firefox/ipc/ipdl/PBlobParent.cpp:463 1 xul.dll mozilla::dom::PContentParent::DestroySubtree(mozilla::ipc::IProtocol::ActorDestroyReason) obj-firefox/ipc/ipdl/PContentParent.cpp:8953 2 xul.dll mozilla::dom::PContentParent::OnChannelError() obj-firefox/ipc/ipdl/PContentParent.cpp:8930 3 xul.dll mozilla::dom::ContentParent::OnChannelError() dom/ipc/ContentParent.cpp:1317 4 xul.dll mozilla::ipc::MessageChannel::NotifyMaybeChannelError() ipc/glue/MessageChannel.cpp:2193 5 xul.dll mozilla::ipc::MessageChannel::OnNotifyMaybeChannelError() ipc/glue/MessageChannel.cpp:2222 6 xul.dll mozilla::detail::RunnableMethodImpl<void ( mozilla::ipc::MessageChannel::*)(void), 0, 1>::Run() obj-firefox/dist/include/nsThreadUtils.h:814 7 xul.dll nsThread::ProcessNextEvent(bool, bool*) xpcom/threads/nsThread.cpp:1213 8 xul.dll nsThread::Shutdown() xpcom/threads/nsThread.cpp:983 9 xul.dll mozilla::detail::RunnableMethodImpl<void ( mozilla::storage::Connection::*)(nsIThread*), 1, 0, nsCOMPtr<nsIThread> >::Run() obj-firefox/dist/include/nsThreadUtils.h:814 10 xul.dll nsThread::ProcessNextEvent(bool, bool*) xpcom/threads/nsThread.cpp:1213 11 xul.dll nsThread::Shutdown() xpcom/threads/nsThread.cpp:983 12 xul.dll mozilla::detail::RunnableMethodImpl<void ( mozilla::storage::Connection::*)(nsIThread*), 1, 0, nsCOMPtr<nsIThread> >::Run() obj-firefox/dist/include/nsThreadUtils.h:814 13 xul.dll nsThread::ProcessNextEvent(bool, bool*) xpcom/threads/nsThread.cpp:1213 14 xul.dll nsThread::Shutdown() xpcom/threads/nsThread.cpp:983 15 xul.dll mozilla::DataStorage::Observe(nsISupports*, char const*, char16_t const*) security/manager/ssl/DataStorage.cpp:886 16 xul.dll nsObserverList::NotifyObservers(nsISupports*, char const*, char16_t const*) xpcom/ds/nsObserverList.cpp:112 17 xul.dll nsObserverService::NotifyObservers(nsISupports*, char const*, char16_t const*) xpcom/ds/nsObserverService.cpp:281 18 xul.dll nsXREDirProvider::DoShutdown() toolkit/xre/nsXREDirProvider.cpp:1250 19 xul.dll ScopedXPCOMStartup::~ScopedXPCOMStartup() toolkit/xre/nsAppRunner.cpp:1409 20 xul.dll mozilla::UniquePtr<ScopedXPCOMStartup, mozilla::DefaultDelete<ScopedXPCOMStartup> >::reset(ScopedXPCOMStartup*) obj-firefox/dist/include/mozilla/UniquePtr.h:345 21 xul.dll XREMain::XRE_main(int, char** const, nsXREAppData const*) toolkit/xre/nsAppRunner.cpp:4641 22 xul.dll XRE_main toolkit/xre/nsAppRunner.cpp:4704 23 firefox.exe do_main browser/app/nsBrowserApp.cpp:328 24 firefox.exe wmain toolkit/xre/nsWindowsWMain.cpp:115 25 firefox.exe __scrt_common_main_seh f:/dd/vctools/crt/vcstartup/src/startup/exe_common.inl:253 Ø 26 kernel32.dll kernel32.dll@0x159cc Ø 27 ntdll.dll ntdll.dll@0x2a2e0
status-firefox53: --- → affected
Keywords: nightly-community
Mass wontfix for bugs affecting firefox 52.
status-firefox52: affected → wontfix

¡Hola!

There are just a handful of 52.9.0esr crashes over at https://crash-stats.mozilla.org/signature/?signature=shutdownhang%20%7C%20mozilla%3A%3Adom%3A%3APBlobParent%3A%3ADestroySubtree&date=%3E%3D2019-12-11T00%3A37%3A00.000Z&date=%3C2020-03-11T00%3A37%3A00.000Z

Resolving.

¡Gracias!
Alex

Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.