Closed Bug 1542321 Opened 1 year ago Closed 4 months ago

AddressSanitizer: stack-overflow /builds/worker/workspace/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:2168:13 in clock_gettime

Categories

(Core :: Audio/Video, defect, P2, critical)

defect

Tracking

()

RESOLVED FIXED
mozilla73
Tracking Status
firefox-esr68 --- wontfix
firefox68 --- wontfix
firefox69 --- wontfix
firefox70 --- wontfix
firefox71 --- wontfix
firefox72 --- wontfix
firefox73 --- fixed

People

(Reporter: jkratzer, Assigned: pehrsons)

References

(Blocks 2 open bugs)

Details

(Keywords: crash, testcase-wanted, Whiteboard: [fuzzblocker])

Attachments

(1 file)

Found while fuzzing mozilla-central rev aa4c97d22712. I'm currently reducing the testcase and will update once complete.

==5970==ERROR: AddressSanitizer: stack-overflow on address 0x7fffc58c2b88 (pc 0x55a9ae7dc55b bp 0x7fffc58c33f0 sp 0x7fffc58c2b90 T0)
    #0 0x55a9ae7dc55a in clock_gettime /builds/worker/workspace/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:2168:13
    #1 0x55a9ae897203 in ClockTimeNs /builds/worker/workspace/build/src/mozglue/misc/TimeStamp_posix.cpp:73:3
    #2 0x55a9ae897203 in mozilla::TimeStamp::Now(bool) /builds/worker/workspace/build/src/mozglue/misc/TimeStamp_posix.cpp:185
    #3 0x7f44b09dc52b in NowLoRes /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/TimeStamp.h:453:40
    #4 0x7f44b09dc52b in XPCJSContext::BeforeProcessTask(bool) /builds/worker/workspace/build/src/js/xpconnect/src/XPCJSContext.cpp:1258
    #5 0x7f44ae9d6b56 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1087:22
    #6 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #7 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #8 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #9 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #10 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #11 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #12 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #13 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #14 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #15 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #16 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #17 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #18 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #19 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #20 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #21 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #22 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #23 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #24 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #25 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #26 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #27 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #28 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #29 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #30 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #31 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #32 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #33 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #34 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #35 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #36 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #37 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #38 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #39 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #40 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #41 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #42 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #43 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #44 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #45 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #46 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #47 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #48 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #49 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #50 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #51 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #52 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #53 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #54 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #55 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #56 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #57 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #58 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #59 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #60 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #61 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #62 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #63 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #64 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #65 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #66 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #67 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #68 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #69 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #70 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #71 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #72 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #73 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #74 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #75 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #76 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #77 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #78 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #79 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #80 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #81 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #82 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #83 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #84 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #85 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #86 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #87 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #88 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #89 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #90 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #91 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #92 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #93 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #94 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #95 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #96 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #97 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #98 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #99 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #100 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #101 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #102 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #103 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #104 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #105 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #106 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #107 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #108 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #109 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #110 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #111 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #112 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #113 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #114 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #115 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #116 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #117 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #118 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #119 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #120 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #121 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #122 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #123 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #124 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #125 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #126 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #127 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #128 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #129 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #130 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #131 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #132 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #133 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #134 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #135 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #136 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #137 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #138 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #139 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #140 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #141 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #142 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #143 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #144 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #145 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #146 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #147 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #148 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #149 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #150 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #151 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #152 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #153 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #154 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #155 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #156 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #157 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #158 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #159 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #160 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #161 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #162 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #163 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #164 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #165 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #166 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #167 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #168 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #169 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #170 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #171 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #172 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #173 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #174 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #175 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #176 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #177 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #178 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #179 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #180 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #181 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #182 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #183 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #184 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #185 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #186 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #187 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #188 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #189 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #190 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #191 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #192 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #193 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #194 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #195 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #196 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #197 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #198 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #199 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #200 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #201 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #202 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #203 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #204 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #205 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #206 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #207 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #208 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #209 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #210 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #211 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #212 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #213 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #214 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #215 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #216 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #217 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #218 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #219 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #220 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #221 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #222 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #223 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #224 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #225 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #226 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #227 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #228 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #229 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #230 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #231 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #232 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #233 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #234 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #235 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #236 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #237 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #238 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #239 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #240 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #241 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #242 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #243 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #244 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #245 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #246 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #247 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #248 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #249 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #250 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #251 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #252 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #253 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #254 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #255 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #256 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #257 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #258 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #259 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #260 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #261 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #262 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #263 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #264 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #265 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #266 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #267 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #268 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #269 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #270 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #271 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #272 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #273 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #274 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #275 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #276 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #277 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #278 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #279 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #280 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #281 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #282 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #283 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #284 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #285 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #286 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #287 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #288 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #289 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #290 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #291 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #292 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32
    #293 0x7f44ae9d7dc1 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1180:14
    #294 0x7f44ae9df974 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:486:10
    #295 0x7f44ae9d59ce in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:348:25
    #296 0x7f44ae9d59ce in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:881
    #297 0x7f44b67635b1 in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:158:14
    #298 0x7f44ae99af35 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:295:32

Paul, given your experience with the MSG, does this mean anything to you? Please change the priority as you see fit, as I'm not particularly familiar with the domain.

Flags: needinfo?(padenot)
Priority: -- → P2

Let's wait for the test-case.

Flags: needinfo?(padenot)

I don't think we are going to get a test case for this. This issue is reported frequently but almost all of the test cases result in a hang or very long pause where the browser is unresponsive before closing. Even if we could get a test case that does repo the issue it would take a very long time to reduce if at all.

I did manage to get an rr trace that captures an abort after a long hang. Hopefully this is enough to get an idea of what is happening. The Pernosco session can be found here: https://pernos.co/debug/tbNBE_fdMkHAxiPZf-745Q/index.html

It would be great to get this fixed since this is a fuzz blocker on one of our most important fuzzers.

Flags: needinfo?(padenot)

Tyson, this is very useful thanks.

Assignee: nobody → padenot
Flags: needinfo?(padenot)

Paul, any chance you can have a look at this and if not could you possibly redirect it to someone else? This issue is a fuzzblocker and is limiting the performance of a fuzzer.

Flags: needinfo?(padenot)

Andreas, do you think you could have a look at this? There is a recording.

Flags: needinfo?(padenot) → needinfo?(apehrson)

I've requested for the pernosco database to be rebuilt so I can take a look.

Hmm, though I'm not sure we need it. This is an old classic (we saw it on B2G or Android emulator?) showing its face again. Many many graphs were created during a relatively short time. Short enough that they're not getting GCed. That, or something is blocking them from getting GCed. Then at shutdown they all get shut down at the same time, including their nsThreads. nsThread::Shutdown spins the event loop until the thread is done shutting down. But spinning the event loop will shut down the next graph's nsThread, and on it goes, higher and higher up the stack.

I'll take a look at whether there's something unexpected happening in the recording, but chances are we need to handle a legit case of this.

There's only one graph involved, so yeah this is very unexpected.

These SystemClockDrivers are all fallbacks, so something in that code appears broken. We shouldn't switch from a fallback driver back to an audio driver. We switch here and this code does not check whether the current driver is a fallback driver.

Bug 1289678 comment 2 sheds some light on this in that the fallback mechanism was put in place so that we don't over-report errors, not to prevent flip-flopping between an audio driver and a clock driver.

I am currently looking into moving the fallback mechanism around in bug 1586370 and might make it so that once there's an error we avoid flip-flopping, because the graph will still think it's on the audio driver. Do you have any concerns with this, Paul?

Flags: needinfo?(apehrson) → needinfo?(padenot)

FWIW the fallback happens because the graph's output channel count is 0, because cubeb returns CUBEB_ERROR when queried for its max channel count. See https://pernos.co/debug/tbNBE_fdMkHAxiPZf-745Q/index.html#f{m[AVCH,qmM_,t[UQ,OEE_,f{e[AVCH,qlw_,s{af28gxKAA,bAWI,oIIoQQw,uIIlN/g___

I'm not sure where to look to see what's happening in the server. Kinetik, could you take a look?

Flags: needinfo?(kinetik)

Tyson, do you know if the machine here have audio capabilities?

Flags: needinfo?(padenot) → needinfo?(twsmith)

(In reply to Andreas Pehrson [:pehrsons] from comment #9)

Do you have any concerns with this, Paul?

It's pretty useful to recover from transient errors, would be nice if the graph would re-try periodically to re-open an audio driver if it needs one. But clearly it can be better than the current situation.

(In reply to Paul Adenot (:padenot) from comment #12)

Tyson, do you know if the machine here have audio capabilities?

Tyson checked on those host (different from the regular treeherder linux hosts), and they do not run the PulseAudio daemon.

Flags: needinfo?(kinetik)
Flags: needinfo?(twsmith)

(In reply to Paul Adenot (:padenot) from comment #13)

(In reply to Andreas Pehrson [:pehrsons] from comment #9)

Do you have any concerns with this, Paul?

It's pretty useful to recover from transient errors, would be nice if the graph would re-try periodically to re-open an audio driver if it needs one. But clearly it can be better than the current situation.

Cool, I can make something work, but with some back-off logic so that we try more and more seldom if the problems continue.

I'll run with "media.cubeb.force_null_context" when testing, that should be able to repro. Per pernosco it could take some 2 minutes, but we'll see.

Assignee: padenot → apehrson
Status: NEW → ASSIGNED

I wasn't able to reproduce this, but I could observe flip-flopping drivers at a very high rate. With bug 1586370 and the patch (to come soon) here that rate is way down. The initial (100ms) and max (30s) rates of the back-off logic is very much up for bikeshedding.

Pushed by pehrsons@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/cb00f09b36fc
Re-init a failed audio driver at increasing intervals. r=padenot
Status: ASSIGNED → RESOLVED
Closed: 4 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla73
You need to log in before you can comment on or make changes to this bug.