MediaStreamGraph shutdown can hang if happening in conjunction with a memory report

RESOLVED FIXED in Firefox 32, Firefox OS v2.0

Status

()

Core
Audio/Video
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: erahm, Assigned: erahm)

Tracking

Trunk
mozilla33
Points:
---

Firefox Tracking Flags

(blocking-b2g:2.0+, firefox31 wontfix, firefox32 fixed, firefox33 fixed, b2g-v2.0 fixed, b2g-v2.1 fixed)

Details

Attachments

(1 attachment)

What I'm seeing is:

1) MediaStreamGraphShutDownRunnable is processed on main thread
2) That causes the MSG thread to be shutdown
3) During shutdown nsThread processes other pending main thread events
4) One of those is CollectReports for MSG
5) Collect reports waits for MSG thread to give it reports, but it's already shut down

And the stack:
Thread 1 (Thread 295.295):
#0  __futex_syscall3 () at bionic/libc/arch-arm/bionic/futex_arm.S:40
#1  0xb6e9a894 in __pthread_cond_timedwait_relative (cond=0xad035b54, mutex=0xad0ffb00, reltime=0x0) at bionic/libc/bionic/pthread.c:1127
#2  0xb6e9a8f4 in __pthread_cond_timedwait (cond=0xad035b54, mutex=0xad0ffb00, abstime=<optimized out>, clock=<optimized out>) at bionic/libc/bionic/pthread.c:1150
#3  0xb6e38cde in __wrap_pthread_cond_wait (cond=0xad035b54, mtx=0xad0ffb00) at ../../../gecko/mozglue/build/Nuwa.cpp:1049
#4  0xb67036ba in PR_WaitCondVar (cvar=0xad035b50, timeout=4294967295) at ../../../../../gecko/nsprpub/pr/src/pthreads/ptsynch.c:385
#5  0xb3e57828 in mozilla::CondVar::Wait (this=0xb01e0680, interval=4294967295) at ../../../gecko/xpcom/glue/BlockingResourceBase.cpp:350
#6  0xb4ad56f8 in Wait (interval=4294967295, this=<optimized out>) at ../../dist/include/mozilla/Monitor.h:47
#7  Wait (interval=4294967295, this=0xbec7c5b8) at ../../dist/include/mozilla/Monitor.h:102
#8  mozilla::MediaStreamGraphImpl::CollectReports (this=0xb01e05c0, aHandleReport=0xad3807e0, aData=0x0) at ../../../gecko/content/media/MediaStreamGraph.cpp:2809
#9  0xb3e7bcd2 in nsMemoryReporterManager::GetReportsForThisProcessExtended (this=<optimized out>, aHandleReport=0xad3807e0, aHandleReportData=0x0, aDMDDumpIdent=...) at ../../../gecko/xpcom/base/nsMemoryReporterManager.cpp:1147
#10 0xb3e7ab14 in nsMemoryReporterManager::StartGettingReports (this=0xb64255c0) at ../../../gecko/xpcom/base/nsMemoryReporterManager.cpp:1083
#11 0xb3e78dbc in nsRunnableMethodImpl<tag_nsresult (nsMemoryReporterManager::*)(), void, true>::Run (this=<optimized out>) at ../../dist/include/nsThreadUtils.h:387
#12 0xb3e7c54e in (anonymous namespace)::MinimizeMemoryUsageRunnable::Run (this=0xad380f40) at ../../../gecko/xpcom/base/nsMemoryReporterManager.cpp:1743
#13 0xb3ea71ca in ProcessNextEvent (aResult=0xbec7c6f7, aMayWait=true, this=0xb6a55700) at ../../../gecko/xpcom/threads/nsThread.cpp:766
#14 nsThread::ProcessNextEvent (this=0xb6a55700, aMayWait=<optimized out>, aResult=0xbec7c6f7) at ../../../gecko/xpcom/threads/nsThread.cpp:685
#15 0xb3e5d67c in NS_ProcessNextEvent (thread=0xb6a55700, mayWait=<optimized out>) at ../../../gecko/xpcom/glue/nsThreadUtils.cpp:263
#16 0xb3ea767c in Shutdown (this=0xafba9080) at ../../../gecko/xpcom/threads/nsThread.cpp:598
#17 nsThread::Shutdown (this=0xafba9080) at ../../../gecko/xpcom/threads/nsThread.cpp:554
#18 0xb4ad3842 in mozilla::MediaStreamGraphImpl::ShutdownThreads (this=0xb01e05c0) at ../../../gecko/content/media/MediaStreamGraph.cpp:1484
#19 0xb4ad389c in mozilla::(anonymous namespace)::MediaStreamGraphShutDownRunnable::Run (this=0xab8597b0) at ../../../gecko/content/media/MediaStreamGraph.cpp:1543
#20 0xb3ea71ca in ProcessNextEvent (aResult=0xbec7c7c7, aMayWait=false, this=0xb6a55700) at ../../../gecko/xpcom/threads/nsThread.cpp:766
#21 nsThread::ProcessNextEvent (this=0xb6a55700, aMayWait=<optimized out>, aResult=0xbec7c7c7) at ../../../gecko/xpcom/threads/nsThread.cpp:685
#22 0xb3e5d67c in NS_ProcessNextEvent (thread=0xb6a55700, mayWait=<optimized out>) at ../../../gecko/xpcom/glue/nsThreadUtils.cpp:263
#23 0xb4064560 in mozilla::ipc::MessagePump::Run (this=0xb6a01e80, aDelegate=0xb6a701a0) at ../../../gecko/ipc/glue/MessagePump.cpp:95
#24 0xb40513ca in MessageLoop::RunInternal (this=0xb6a701a0) at ../../../gecko/ipc/chromium/src/base/message_loop.cc:229
#25 0xb40513e2 in RunHandler (this=0xb6a701a0) at ../../../gecko/ipc/chromium/src/base/message_loop.cc:222
#26 MessageLoop::Run (this=0xb6a701a0) at ../../../gecko/ipc/chromium/src/base/message_loop.cc:196
#27 0xb46996ea in nsBaseAppShell::Run (this=0xb289f340) at ../../../gecko/widget/xpwidgets/nsBaseAppShell.cpp:164
#28 0xb4f9bbae in nsAppStartup::Run (this=0xb28a16d0) at ../../../../gecko/toolkit/components/startup/nsAppStartup.cpp:278
#29 0xb4f83696 in XREMain::XRE_mainRun (this=0xbec7c95c) at ../../../gecko/toolkit/xre/nsAppRunner.cpp:4012
#30 0xb4f8389a in XREMain::XRE_main (this=0xbec7c95c, argc=<optimized out>, argv=<optimized out>, aAppData=<optimized out>) at ../../../gecko/toolkit/xre/nsAppRunner.cpp:4083
#31 0xb4f839f4 in XRE_main (argc=1, argv=0xbec7eb14, aAppData=0x24938, aFlags=<optimized out>) at ../../../gecko/toolkit/xre/nsAppRunner.cpp:4297
#32 0x0000ae9c in do_main (argv=0xbec7eb14, argc=1) at ../../../gecko/b2g/app/nsBrowserApp.cpp:163
#33 main (argc=<optimized out>, argv=<optimized out>) at ../../../gecko/b2g/app/nsBrowserApp.cpp:256
Proposed solution:
#1 - Add a timeout to mozilla::MediaStreamGraphImpl::CollectReports
#2 - set a flag (if not already set, I need to look at the code) that indicates shutdown, check that flag in CollectReports
Assignee: nobody → erahm
I've confirmed that this is what was consistently blocking getting memory reports on 2.0+ for me and :justindarc. I believe it was the result of using --minimize, which caused MSG to be cycle collected, which led to the shutdown-and-then-hang situation.
Created attachment 8453469 [details] [diff] [review]
MediaStreamGraph shutdown can hang if happening in conjunction with a memory report

This adds a check to CollectReports to make sure the MSG thread is not being shut down. It also adds a max wait of 1 second for the memory usage to be tallied.
Attachment #8453469 - Flags: review?(roc)
This potentially blocks retrieving memory reports on 2.0, nomming.
blocking-b2g: --- → 2.0?
https://hg.mozilla.org/mozilla-central/rev/c95a18275e1d
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla33

Updated

4 years ago
blocking-b2g: 2.0? → 2.0+
https://hg.mozilla.org/releases/mozilla-aurora/rev/0061c1fe7e28
status-b2g-v2.0: --- → fixed
status-b2g-v2.1: --- → fixed
status-firefox31: --- → wontfix
status-firefox32: --- → fixed
status-firefox33: --- → fixed
You need to log in before you can comment on or make changes to this bug.