824359 - Cubeb code isn't shut down in/before XPCOM shutdown and occasionally crashes in shutdown

Status: RESOLVED DUPLICATE of bug 802538

(Core :: Audio/Video, defect, P1)

Description

[Eric Rescorla (:ekr)]

This appears to be entirely in gUM.

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000008
[Switching to process 74069 thread 0xff0b]
0x00000001035e2ab4 in mozilla::DeadlockDetector<mozilla::BlockingResourceBase::DeadlockDetectorEntry>::CheckAcquisition (this=0x0, aLast=0x0, aProposed=0x11731d4e0, aCallContext=@0x146e804b0) at DeadlockDetector.h:402
402	        PRAutoLock _(mLock);
(gdb) bt
#0  0x00000001035e2ab4 in mozilla::DeadlockDetector<mozilla::BlockingResourceBase::DeadlockDetectorEntry>::CheckAcquisition (this=0x0, aLast=0x0, aProposed=0x11731d4e0, aCallContext=@0x146e804b0) at DeadlockDetector.h:402
#1  0x00000001035e1cb9 in mozilla::BlockingResourceBase::CheckAcquire (this=0x11731d468, aCallContext=@0x146e804b0) at BlockingResourceBase.cpp:105
#2  0x00000001035e233a in mozilla::Mutex::Lock (this=0x11731d468) at BlockingResourceBase.cpp:227
#3  0x00000001023230a5 in mozilla::Monitor::Lock (this=0x11731d468) at Monitor.h:37
#4  0x0000000102323083 in mozilla::MonitorAutoLock::MonitorAutoLock (this=0x146e805a0, aMonitor=@0x11731d468) at Monitor.h:92
#5  0x000000010232136d in mozilla::MonitorAutoLock::MonitorAutoLock (this=0x146e805a0, aMonitor=@0x11731d468) at Monitor.h:93
#6  0x00000001025750a5 in mozilla::BufferedAudioStream::DataCallback (this=0x11731d3e0, aBuffer=0x125851e00, aFrames=186) at /Users/ekr/dev/mozilla-inbound/content/media/AudioStream.cpp:997
#7  0x000000010257577c in mozilla::BufferedAudioStream::DataCallback_S (aThis=0x11731d3e0, aBuffer=0x125851e00, aFrames=186) at /Users/ekr/dev/mozilla-inbound/content/media/AudioStream.cpp:610
#8  0x0000000103e05c59 in audio_unit_output_callback (user_ptr=0x12383b720, flags=0x146e806ec, tstamp=0x1205c4270, bus=0, nframes=186, bufs=0x120543370) at /Users/ekr/dev/mozilla-inbound/media/libcubeb/src/cubeb_audiounit.c:57
#9  0x0000000123c7fb47 in AUInputElement::PullInput ()
#10 0x0000000123c7f014 in AUInputFormatConverter2::InputProc ()
#11 0x00007fff970d2973 in AudioConverterChain::CallInputProc ()
#12 0x00007fff970d26a5 in AudioConverterChain::FillBufferFromInputProc ()
#13 0x00007fff970d262b in BufferedAudioConverter::GetInputBytes ()
#14 0x00007fff970d2c04 in Resampler2Wrapper::RenderOutput ()
#15 0x00007fff970d2305 in BufferedAudioConverter::FillBuffer ()
#16 0x00007fff970d260a in BufferedAudioConverter::GetInputBytes ()
#17 0x00007fff970d253f in CBRConverter::RenderOutput ()
#18 0x00007fff970d2305 in BufferedAudioConverter::FillBuffer ()
#19 0x00007fff970d2435 in AudioConverterChain::RenderOutput ()
#20 0x00007fff970d2305 in BufferedAudioConverter::FillBuffer ()
#21 0x00007fff970d209d in AudioConverterFillComplexBuffer ()
#22 0x0000000123c7f0c7 in AUInputFormatConverter2::PullAndConvertInput ()
#23 0x0000000123c7e0c0 in AUConverterBase::RenderBus ()
#24 0x0000000123c7d544 in AUBase::DoRenderBus ()
#25 0x0000000123c7a9cd in AUBase::DoRender ()
#26 0x0000000123c81c13 in AUHAL::AUIOProc ()
#27 0x00007fff95b959fa in HALC_ProxyIOContext::IOWorkLoop ()
#28 0x00007fff95b95151 in HALC_ProxyIOContext::IOThreadEntry ()
#29 0x00007fff95b9500c in HALB_IOThread::Entry ()
#30 0x00007fff8ebc28bf in _pthread_start ()
#31 0x00007fff8ebc5b75 in thread_start ()

Comment 1

[Jason Smith [:jsmith]]

Do we know which mochitest in particular is causing this crash?

Comment 2

[Eric Rescorla (:ekr)]

I don't. I just see it occasionally and haven't narrowed it down

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Hm, I haven't seen this before. Might this be another variation of bug 821292? I mean since that one has been fixed?

Comment 4

[Eric Rescorla (:ekr)]

No, I don't think it's a variation of bug 821292. It doesn't like it has any connection to PeerConnection at all.

Comment 5

[Randell Jesup [:jesup] (needinfo me)]

This is crashing *deep* in cubeb - What could cause this?  Looking deeper:

Seems like some sort of shutdown race condition with playing audio when shutdown is invoked; the error seems to be due to sDeadlockDetector being null, which happens in LogTerm(), which is called at the *end* of XPCOM shutdown.  This means almost certainly something in the audio code isn't being shut down cleanly, and once in a while happens to process a buffer and touch a lock after LogTerm() nulls it but before we exit (like when processing some of the stats).

Not a gUM bug; reassigning component

Comment 7

[Jason Smith [:jsmith]]

It might not be a gum bug, but it's blocking automation associated with gum. We should keep the whiteboard flags on.

Comment 8

[Jason Smith [:jsmith]]

Adam's comment in another bug implies that this is killing us in the automation world right now for WebRTC - 50% to 75% we get this crash.

Comment 9

[Randell Jesup [:jesup] (needinfo me)]

We need to retest with the patch from bug 802538 in (landed this afternoon, after adam's comment).

Discussion with kinetik implies a left-open BufferedAudioStream likely coming from MediaStreamGraph.  Until bug 802538 landed, we were failing to shut down the MediaStreamGraph when the page navigated, and so it would continue until (and into?) shutdown.

Comment 10

[Randell Jesup [:jesup] (needinfo me)]

We look good, no more failures seen.  Caused by bug 802538