1347067 - Shutdown hang in audiounit_close_stream

Status: RESOLVED FIXED

(Core :: Audio/Video: cubeb, defect, P1)

Description

[JW Wang [:jwwang] [:jw_wang]]

https://crash-stats.mozilla.com/search/?signature=~MediaShutdownManager&version=55.0a1&version=54.0a2&version=54.0a1&version=53.0b&version=53.0b2&version=53.0b1&platform=Mac%20OS%20X&date=%3E%3D2017-02-14T07%3A17%3A23.000Z&date=%3C2017-03-14T07%3A17%3A23.000Z&_sort=-date&_facets=signature&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#crash-reports

Comment 1

[JW Wang [:jwwang] [:jw_wang]]

640b7ecb-7c06-4cf8-85bc-bfe872170314

Thread 36
6 	AudioToolbox 	APComponentInstance::DisposeInstance() 	
7 	XUL 	audiounit_close_stream 	media/libcubeb/src/cubeb_audiounit.cpp:1801
8 	XUL 	___ZL36audiounit_property_listener_callbackjjPK26AudioObjectPropertyAddressPv_block_invoke 	media/libcubeb/src/cubeb_audiounit.cpp:566

Thread 38
6 	AudioToolbox 	APComponentInstance::DisposeInstance() 	
7 	XUL 	audiounit_close_stream 	media/libcubeb/src/cubeb_audiounit.cpp:1801
8 	XUL 	audiounit_stream_destroy 	media/libcubeb/src/cubeb_audiounit.cpp:1817
9 	XUL 	mozilla::AudioStream::Shutdown() 	dom/media/AudioStream.h:26

Comment 2

[JW Wang [:jwwang] [:jw_wang]]

https://crash-stats.mozilla.com/report/index/a34735ee-ffe8-47a9-be77-76f5f2170314#allthreads

Thread 47
6 	AudioToolbox 	APComponentInstance::DisposeInstance() 	
7 	XUL 	audiounit_close_stream 	media/libcubeb/src/cubeb_audiounit.cpp:1801
8 	XUL 	___ZL36audiounit_property_listener_callbackjjPK26AudioObjectPropertyAddressPv_block_invoke 	media/libcubeb/src/cubeb_audiounit.cpp:566

Thread 50
6 	AudioToolbox 	APComponentInstance::DisposeInstance() 	
7 	XUL 	audiounit_close_stream 	media/libcubeb/src/cubeb_audiounit.cpp:1801
8 	XUL 	audiounit_stream_destroy 	media/libcubeb/src/cubeb_audiounit.cpp:1817
9 	XUL 	mozilla::AudioStream::Shutdown() 	dom/media/AudioStream.h:26

Comment 3

[JW Wang [:jwwang] [:jw_wang]]

https://crash-stats.mozilla.com/report/index/e3494569-df78-474e-b658-219ad2170313#allthreads

Thread 48
8 	XUL 	audiounit_close_stream 	media/libcubeb/src/cubeb_audiounit.cpp:1801
9 	XUL 	___ZL36audiounit_property_listener_callbackjjPK26AudioObjectPropertyAddressPv_block_invoke 	media/libcubeb/src/cubeb_audiounit.cpp:566

Thread 54
8 	XUL 	audiounit_close_stream 	media/libcubeb/src/cubeb_audiounit.cpp:1801
9 	XUL 	audiounit_stream_destroy 	media/libcubeb/src/cubeb_audiounit.cpp:1817
10 	XUL 	mozilla::AudioStream::Shutdown() 	dom/media/AudioStream.h:26

Comment 4

[JW Wang [:jwwang] [:jw_wang]]

It looks like deadlock will happen if both audiounit_reinit_stream() and audiounit_stream_destroy() call audiounit_close_stream() at the same time.

Comment 6

[JW Wang [:jwwang] [:jw_wang]]

bp-aef242fa-0842-4e04-8e8f-c992f2170314

Thread 53
4 	XUL 	audiounit_close_stream 	media/libcubeb/src/cubeb_audiounit.cpp:765
5 	XUL 	___ZL36audiounit_property_listener_callbackjjPK26AudioObjectPropertyAddressPv_block_invoke 	media/libcubeb/src/cubeb_audiounit.cpp:642


Thread 57
1 	XUL 	audiounit_stream_stop 	media/libcubeb/src/cubeb_utils_unix.h:56
2 	XUL 	mozilla::AudioStream::Shutdown() 	dom/media/AudioStream.cpp:468

We have this crash on 55.0a1 even with bug 1343930 fixed.

Comment 7

[JW Wang [:jwwang] [:jw_wang]]

One more instance: bp-f245e015-5dc0-4cd6-981d-084aa2170320

Comment 8

[JW Wang [:jwwang] [:jw_wang]]

bp-e607904e-7a2a-45d2-9f5d-953b92170315

Comment 9

[Alex Chronopoulos [:achronop]]

Patch of bug 1345049 landed in m-c 2017-03-10. Most of crash-stats mentioned above have build date before landing. There are 2 crash-stats though that are indeed after landing. 

2017-03-13: aef242fa-0842-4e04-8e8f-c992f2170314
2017-03-19: f245e015-5dc0-4cd6-981d-084aa2170320

Comment 10

[Alex Chronopoulos [:achronop]]

Given crash-stat: 2017-03-13: aef242fa-0842-4e04-8e8f-c992f2170314

thread 52
0 	libsystem_kernel.dylib 	libsystem_kernel.dylib@0x15746 	
1 	XUL 	audiounit_property_listener_callback 	media/libcubeb/src/cubeb_utils_unix.h:56

thread 53
0 	libsystem_kernel.dylib 	libsystem_kernel.dylib@0x15716 	
1 	CoreAudio 	CAGuard::WaitFor(unsigned long long) 	
2 	CoreAudio 	HALObject::RemovePropertyListener(AudioObjectPropertyAddress const&, HALPropertyListener const&) 	
3 	CoreAudio 	AudioObjectRemovePropertyListener 	
4 	XUL 	audiounit_close_stream 	media/libcubeb/src/cubeb_audiounit.cpp:765
5 	XUL 	___ZL36audiounit_property_listener_callbackjjPK26AudioObjectPropertyAddressPv_block_invoke 	media/libcubeb/src/cubeb_audiounit.cpp:642

thread 57
0 	libsystem_kernel.dylib 	libsystem_kernel.dylib@0x15746 	
1 	XUL 	audiounit_stream_stop 	media/libcubeb/src/cubeb_utils_unix.h:56
2 	XUL 	mozilla::AudioStream::Shutdown() 	dom/media/AudioStream.cpp:468

Cubeb threads 52 and 57 are waiting for 53 to finish which makes sense. Cubeb thread 53 is blocked with many other non-cubeb threads of the system: 3 - 16, 18 - 31, 33 - 35, 37 -41, 45 - 51, 54, 56

I am not sure where the deadlock is created but I believe is not caused by cubeb. Cubeb is blocked along with the most of the system.

The other crash-stat from 2017-03-19 is similar.

Can you please double check the above? Do you have any idea where the deadlock is created?

Comment 11

[JW Wang [:jwwang] [:jw_wang]]

No, I have no idea where deadlock is from. It just appears to me audiounit_stream_stop() is waiting forever to acquire a lock.

Comment 12

[Emma Humphries ☕️🎸🧞‍♀️✨ (she/they) [:emceeaich] (Pacific Time) use needinfo]

This is an assigned P1 bug without activity in two weeks. 

If you intend to continue working on this bug for the current release/iteration/sprint, remove the 'stale-bug' keyword.

Otherwise we'll reset the priority of the bug back to '--' on Monday, August 28th.

Comment 13

[Alex Chronopoulos [:achronop]]

I haven't seen that crash for a long time. Feel free to reopen if you have it again.