Closed Bug 1426185 Opened 2 years ago Closed 2 years ago

Recursion in SpinEventLoopUntil<> (potential stack exhaustion)

Categories

(Core :: Audio/Video: MediaStreamGraph, defect, P2)

59 Branch
defect

Tracking

()

RESOLVED FIXED
Tracking Status
firefox59 --- fixed

People

(Reporter: jkratzer, Unassigned)

References

(Blocks 1 open bug)

Details

(Keywords: crash, sec-other, testcase, Whiteboard: [fuzzblocker][post-critsmash-triage][adv-main59-])

Attachments

(3 files)

Found while fuzzing mozilla-central rev f1329009bf0d.  I don't currently have a reproducible test case for this but will update if one becomes available.

==15445==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000018 (pc 0x7f6408b394c4 bp 0x7fff1fcb22b0 sp 0x7fff1fcb1e00 T0)
==15445==The signal is caused by a READ memory access.
==15445==Hint: address points to the zero page.
    #0 0x7f6408b394c3 in DoGetOrCreateDOMReflector<mozilla::dom::MediaStreamTrack, mozilla::dom::binding_detail::GetOrCreateReflectorWrapBehavior::eWrapIntoContextCompartment> /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/dom/BindingUtils.h:1095:46
    #1 0x7f6408b394c3 in GetOrCreateDOMReflector<mozilla::dom::MediaStreamTrack> /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/dom/BindingUtils.h:1179
    #2 0x7f6408b394c3 in GetOrCreate /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/dom/BindingUtils.h:1876
    #3 0x7f6408b394c3 in GetOrCreateDOMReflector<mozilla::dom::MediaStreamTrack> /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/dom/BindingUtils.h:1886
    #4 0x7f6408b394c3 in mozilla::dom::PeerConnectionObserverJSImpl::OnDTMFToneChange(mozilla::dom::MediaStreamTrack&, nsTSubstring<char16_t> const&, mozilla::ErrorResult&, JSCompartment*) /builds/worker/workspace/build/src/obj-firefox/dom/bindings/PeerConnectionObserverBinding.cpp:2488
    #5 0x7f6406d2219e in mozilla::PeerConnectionImpl::DTMFSendTimerCallback_m(nsITimer*, void*) /builds/worker/workspace/build/src/media/webrtc/signaling/src/peerconnection/PeerConnectionImpl.cpp:3841:8
    #6 0x7f6404fe2353 in nsTimerImpl::Fire(int) /builds/worker/workspace/build/src/xpcom/threads/nsTimerImpl.cpp:701:7
    #7 0x7f6404fb1c59 in nsTimerEvent::Run() /builds/worker/workspace/build/src/xpcom/threads/TimerThread.cpp:286:11
    #8 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #9 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #10 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #11 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #12 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #13 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #14 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #15 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #16 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #17 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #18 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #19 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #20 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #21 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #22 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #23 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #24 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #25 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #26 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #27 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #28 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #29 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #30 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #31 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #32 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #33 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #34 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #35 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #36 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #37 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #38 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #39 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #40 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #41 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #42 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #43 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #44 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #45 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #46 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #47 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #48 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #49 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #50 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #51 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #52 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #53 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #54 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #55 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #56 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #57 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #58 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #59 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #60 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #61 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #62 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #63 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #64 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #65 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #66 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #67 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #68 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #69 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #70 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #71 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #72 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #73 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #74 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #75 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #76 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #77 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #78 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #79 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #80 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #81 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #82 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #83 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #84 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #85 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #86 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #87 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #88 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #89 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #90 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #91 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #92 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #93 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #94 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #95 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #96 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #97 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #98 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #99 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #100 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #101 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #102 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #103 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #104 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #105 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #106 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #107 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #108 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #109 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #110 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #111 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #112 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #113 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #114 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #115 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #116 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #117 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #118 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #119 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #120 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #121 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #122 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #123 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #124 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #125 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #126 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #127 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #128 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #129 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #130 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #131 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #132 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #133 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #134 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #135 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #136 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #137 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #138 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #139 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #140 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #141 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #142 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #143 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #144 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #145 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #146 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #147 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #148 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #149 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #150 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #151 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #152 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #153 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #154 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #155 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #156 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #157 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #158 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #159 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #160 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #161 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #162 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #163 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #164 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #165 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #166 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #167 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #168 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #169 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #170 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #171 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #172 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #173 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #174 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #175 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #176 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #177 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #178 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #179 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #180 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #181 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #182 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #183 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #184 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #185 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #186 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #187 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #188 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #189 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #190 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #191 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #192 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #193 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #194 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #195 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #196 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #197 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #198 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #199 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #200 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #201 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #202 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #203 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #204 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #205 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #206 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #207 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #208 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #209 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #210 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #211 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #212 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #213 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #214 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #215 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #216 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #217 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #218 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #219 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #220 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #221 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #222 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #223 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #224 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #225 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #226 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #227 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #228 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #229 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #230 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #231 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #232 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #233 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #234 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #235 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #236 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #237 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #238 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #239 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #240 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #241 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #242 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #243 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #244 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #245 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #246 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #247 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #248 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #249 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #250 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #251 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #252 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #253 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #254 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #255 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #256 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #257 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #258 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #259 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #260 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #261 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #262 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #263 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #264 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #265 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #266 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #267 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #268 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #269 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #270 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #271 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #272 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #273 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #274 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #275 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #276 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #277 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #278 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #279 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #280 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #281 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #282 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #283 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #284 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #285 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #286 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32
    #287 0x7f6404f9b010 in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/workspace/build/src/xpcom/threads/SchedulerGroup.cpp:395:25
    #288 0x7f6404fc1886 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1039:14
    #289 0x7f6404fdd370 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10
    #290 0x7f6404fc02e2 in SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, (lambda at /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796:22)> /builds/worker/workspace/build/src/obj-firefox/dist/include/nsThreadUtils.h:323:25
    #291 0x7f6404fc02e2 in nsThread::Shutdown() /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:796
    #292 0x7f640aca84da in mozilla::MediaStreamGraphShutdownThreadRunnable::Run() /builds/worker/workspace/build/src/dom/media/GraphDriver.cpp:139:14
    #293 0x7f6404f9dc4b in mozilla::EventTargetWrapper::Runner::Run() /builds/worker/workspace/build/src/xpcom/threads/AbstractThread.cpp:150:32

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/dom/BindingUtils.h:1095:46 in DoGetOrCreateDOMReflector<mozilla::dom::MediaStreamTrack, mozilla::dom::binding_detail::GetOrCreateReflectorWrapBehavior::eWrapIntoContextCompartment>
==15445==ABORTING
Rank: 35
Priority: -- → P4
Attached file trigger.html
The attached testcase must be provided via a local webserver.
Attached file prefs.js
This is super sketchy [1]:

> pco->OnDTMFToneChange(*state->mTransceiver->GetSendTrack(), eventTone, jrv);

We're dereferencing a temporary RefPtr (we actually attempt to prevent that [2]). We probably want:

> RefPtr<...> track = state->mTransceiver->GetSendTrack();
> pco->OnDTMFToneChange(*track, eventTone, jrv);

[1] https://searchfox.org/mozilla-central/rev/ff462a7f6b4dd3c3fdc53c9bc0b328f42a5b3d2b/media/webrtc/signaling/src/peerconnection/PeerConnectionImpl.cpp#3841
[2] https://searchfox.org/mozilla-central/rev/f6f1731b1b7fec332f86b55fa40e2c9ae67ac39b/mfbt/RefPtr.h#303-306
Sorry erahm; mostly I wanted you to look at the recursion in SpinEventLoopUntil<>; we suspected this was a dup of another sec bug already.

We might want to file a new bug about SpinEventLoopUntil<>; I'd be surprised if it was expected to recurse in this fashion.
Group: media-core-security
Flags: needinfo?(erahm)
want to dup this?
Flags: needinfo?(docfaraday)
(In reply to Randell Jesup [:jesup] from comment #4)
> Sorry erahm; mostly I wanted you to look at the recursion in
> SpinEventLoopUntil<>; we suspected this was a dup of another sec bug already.
> 
> We might want to file a new bug about SpinEventLoopUntil<>; I'd be surprised
> if it was expected to recurse in this fashion.

It's not really recursion here...well it kind of is. Spinning the event loop is intentional when shutting down threads (we have to wait for an ack that the thread actually shuts down, it's possible the thread still has some cleanup events pending, and we don't want to jank the main thread), the problem is you have a ton of other shutdown events pending. You should probably look into why that is.
Flags: needinfo?(erahm)
(In reply to Randell Jesup [:jesup] from comment #5)
> want to dup this?

Let me see if the fix from bug 1423086 also fixes this. I have a sneaking suspicion that it will.
The fix from bug 1423086 is in nightly now, so give it a shot.
Flags: needinfo?(docfaraday) → needinfo?(jkratzer)
(In reply to Byron Campen [:bwc] from comment #8)
> The fix from bug 1423086 is in nightly now, so give it a shot.

It's possible that makes comment 3 less likely, but we should still probably fix that and make it so that RefPtr doesn't allow grabbing a pointer to a temporary.
Agreed, we should fix this (re: comment 3)

erahm: Spinning the event loop is fine, but I don't expect it to fill the stack with SpinEventLoopUntil<> frames - I wish there was some way for it to know that it was called from SpinEventLoopUntil<> so we could collapse them.
I am still able to reproduce this issue using m-c rev 5b1fdaa14d35 (20171221).
Ok, so this is a new problem.
Flags: needinfo?(jkratzer)
Keywords: sec-high
(In reply to Randell Jesup [:jesup] from comment #10)
> Agreed, we should fix this (re: comment 3)
> 
> erahm: Spinning the event loop is fine, but I don't expect it to fill the
> stack with SpinEventLoopUntil<> frames - I wish there was some way for it to
> know that it was called from SpinEventLoopUntil<> so we could collapse them.

This really does look like recursion, not a simple spin on the event loop. I would need more time to look at it though, and I have my hands full at the moment.
(In reply to Byron Campen [:bwc] from comment #13)
> (In reply to Randell Jesup [:jesup] from comment #10)
> > Agreed, we should fix this (re: comment 3)
> > 
> > erahm: Spinning the event loop is fine, but I don't expect it to fill the
> > stack with SpinEventLoopUntil<> frames - I wish there was some way for it to
> > know that it was called from SpinEventLoopUntil<> so we could collapse them.
> 
> This really does look like recursion, not a simple spin on the event loop. I
> would need more time to look at it though, and I have my hands full at the
> moment.

So this is roughly:

#1 - We post `MediaStreamGraphShutdownThreadRunnable`
#2 - That does a thread shutdown on the main thread [1] 
#3 - Thread shutdown spins the loop [2] waiting `nsThreadShutdownAckEvent` to run [3] when the nsThread finishes
#4 - We process the next event, it happens to be a `MediaStreamGraphShutdownThreadRunnable`, repeat

I'm reasonably sure that the `MediaStreamGraphShutdownThreadRunnable` in step 4 is a unique runnable (not the one we were just running), so we need to figure out why we keep posting `MediaStreamGraphShutdownThreadRunnable`s.

[1] https://searchfox.org/mozilla-central/rev/ff462a7f6b4dd3c3fdc53c9bc0b328f42a5b3d2b/dom/media/GraphDriver.cpp#139
[2] https://searchfox.org/mozilla-central/rev/f6f1731b1b7fec332f86b55fa40e2c9ae67ac39b/xpcom/threads/nsThread.cpp#796-798
[3] https://searchfox.org/mozilla-central/rev/f6f1731b1b7fec332f86b55fa40e2c9ae67ac39b/xpcom/threads/nsThread.cpp#454-463
Repro'd in a debug build, but didn't get the crazy stack:

> Assertion failure: mRawPtr != nullptr (You can't dereference a NULL RefPtr with operator*().), at /var/dev/erahm/mozilla-unified/obj-x86_64-pc-linux-gnu-clang-5-debug/dist/include/mozilla/RefPtr.h:370
> ...
> #6  0x00007fffe65213c3 in RefPtr<mozilla::dom::MediaStreamTrack>::operator* (this=<optimized out>)
>     at /var/dev/erahm/mozilla-unified/obj-x86_64-pc-linux-gnu-clang-5-debug/dist/include/mozilla/RefPtr.h:369
> #7  0x00007fffe6512af9 in mozilla::PeerConnectionImpl::DTMFState::Notify (this=<optimized out>, timer=<optimized out>)
Attachment #8938525 - Flags: review?(docfaraday)
Assignee: nobody → erahm
Status: NEW → ASSIGNED
Comment on attachment 8938525 [details] [diff] [review]
Add warning about MediaStreamTrack

Review of attachment 8938525 [details] [diff] [review]:
-----------------------------------------------------------------

dminor fixed this earlier today

https://hg.mozilla.org/integration/autoland/rev/2e99f273f7b7
Attachment #8938525 - Flags: review?(docfaraday)
Any progress on this?  perhaps throw rr at it?
Flags: needinfo?(erahm)
This ended up being a dup right?
Flags: needinfo?(erahm) → needinfo?(docfaraday)
Maybe? Bug 1426678 had a patch that was similar to yours, that should fix the stack in comment 15, but it is unclear to me whether this is the same bug or not. What's your take?
Flags: needinfo?(docfaraday) → needinfo?(erahm)
(In reply to Randell Jesup [:jesup] from comment #4)
> Sorry erahm; mostly I wanted you to look at the recursion in
> SpinEventLoopUntil<>; we suspected this was a dup of another sec bug already.
> 
> We might want to file a new bug about SpinEventLoopUntil<>; I'd be surprised
> if it was expected to recurse in this fashion.

So I think bug 1426678 covers the DTMF issue; I'm going to make it clear this bug is (now) about the SpinEventLoopUntil<> recursion.  If this continues it will overrun the stack (and slow things down, though that's not a huge issue).

Keeping this hidden since the other bug discussed here is a sec issue.
Rank: 35
Component: WebRTC → XPCOM
Keywords: sec-highsec-other
Priority: P4 → --
Summary: Crash near null at [@ DoGetOrCreateDOMReflector<mozilla::dom::MediaStreamTrack, mozilla::dom::binding_detail::GetOrCreateReflectorWrapBehavior::eWrapIntoContextCompartment>] → Recursion in SpinEventLoopUntil<> (potential stack exhaustion)
Copied from IRC in response to comment 14:

We only dispatch them [MediaStreamGraphShutdownThreadRunnable] in ~ThreadedDriver; which can only really happen once per driver (a MSGraph can switch between CallbackDrivers and SystemClock drivers, but this wouldn't happen continually let alone pile up in a queue).  It is possible that on system shutdown, when the gGraphs are all shut down, and in a mochitest or fuzz test there might be lots of graphs.

We'll ForceShutdown all the graphs in gGraphs, which will send MediaStreamGraphShutDownRunnable  for each graph, which will call mGraph->mDriver->Shutdown(), which clears mThread -- if mThread is cleared, ~ThreadedDriver() won't even send a MediaStreamGraphShutdownThreadRunnable

We need an rr run here...
Component: XPCOM → Audio/Video: MediaStreamGraph
Flags: needinfo?(padenot)
Reading the test-case, this is fine, it's just that the DTMF code is wrong.
Flags: needinfo?(padenot)
(In reply to Paul Adenot (:padenot) from comment #23)
> Reading the test-case, this is fine, it's just that the DTMF code is wrong.

The DMTF issue was resolved in another bug; this is about the (apparent) slew of shutdown messages causing a deep recursion of SpinEventLoopUntil
So, we're cleaning up a _huge_ number of MSGs (this is expected, from the test case, it keeps reloading the page). In addition, this machine does not have working audio, so we create normal threads (and not audio streams).

Then it goes lie this:
1. We want to shutdown an MSG A. It has a `SystemClockDriver`, that has owns a thread.
2. We dispatch an event to the main thread `MediaStreamGraphShutDownRunnable`, telling it to shut down
3. We receive this on the main thread, and we shut down the `SystemClockDriver`.
4. This calls `Shutdown` on the `SystemClockDriver` thread.
5. This spins the even loop of the main thread, supposedly to, at some point, receive the confirmation that the thread can be joined
6. The main thread finds another `MediaStreamGraphShutDownRunnable`, so we jump back to 3., and loop until the threads are joined and everything unwinds properly.

This seems sound. The stack is a bit deep, and the event loops are nested like crazy, but it's unclear if we can do something else really.
In that case, this may be "as expected", though for this test edge-case it may drive the stack to a very deep point.
Flags: needinfo?(erahm)
Rank: 15
Priority: -- → P2
Assignee: erahm → nobody
Status: ASSIGNED → NEW
Closing this, I don't see anything for anyone to do here, now that Eric has patched the DTMF code. Please reopen if you disagree.
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Group: media-core-security → core-security-release
Flags: qe-verify-
Whiteboard: [fuzzblocker] → [fuzzblocker][post-critsmash-triage]
Whiteboard: [fuzzblocker][post-critsmash-triage] → [fuzzblocker][post-critsmash-triage][adv-main59-]
Group: core-security-release
You need to log in before you can comment on or make changes to this bug.