It looks like there is a racing issue here on my macOS. There are two sources that can notify the devices-changed event: 1. [`InputObserver::OnDeviceChange()`][0] in `CamerasParent`, by listening [`AVCaptureDeviceWasDisconnectedNotification`][1] 2. [`CubebDeviceEnumerator::InputAudioDeviceListChanged_s`][2] for input, [`CubebDeviceEnumerator::OutputAudioDeviceListChanged_s`][4] for output, by listening [`kAudioHardwarePropertyDevices` on `kAudioObjectSystemObject`][3] The listeners in (2) are registered as the callbacks for devices-changed events in the content process directly. When the listener is fired, it will clear the cached input or output devices in [`CubebDeviceEnumerator`][5] (specifically, `mInputDevices` and `mOutputDevices`). On the other hand, the notification by (1) will end up getting to [`MediaManager::OnDeviceChange`][6] via an IPC channel from `CamerasParent`(parent process) to `CamerasChild`(content process). Then `MediaManager::OnDeviceChange` will create a task to check if there are missing devices (code path below). To check if there are missing devices, `MediaManager` will get the current devices by `CubebDeviceEnumerator` and then compare the current devices with the its cached devices (`mDeviceIDs` specifically). The missing device is the device that is in the cached device list but there is not the current device list. In the end, [the media sources for the missing devices will be stopped][7]. Therefore, it looks like we intend to close the media source if the device tied to the source is unplugged. Thus, the microphone shouldn't be switched automatically. However, the interesting thing is that the mic can be switched automatically when the device is not the default input device in the system. When the mic used in WebRTC is system default, the listeners in (2) are always run __before__ the `CubebDeviceEnumerator::EnumerateAudioDevices` triggered by (1). Therefore, the cached devices in the `CubebDeviceEnumerator` will be cleared before `MediaManager` asks for the current devices. If there is no cached device, the `CubebDeviceEnumerator` will get the devices from cubeb. Currently, if the mic is [closed][10] properly, `MediaStreamGraph::mInputDeviceID` will be set to `nullptr` and the `AudioCallbackDriver::mInputChannelCount` will [become `0`][11], so the `AudioCallbackDriver` for next iteration will be a output-only stream if there is an output device. In contrast, when the mic used in WebRTC is __not__ system default, the listeners in (2) are always run __after__ the `CubebDeviceEnumerator::EnumerateAudioDevices` triggered by (1). That is, at the time `CubebDeviceEnumerator::EnumerateAudioDevices` is called, `CubebDeviceEnumerator` will [return its cached devices directly without querying from cubeb][9]. As a result, the media source tied to the unplugged mic won't be stopped. Since [cubeb can reinitialize stream silently][8], the WebRTC works fine. I am not sure if we intend to switch the mic automatically when the active mic is unplugged. It looks like we try stopping the media source stream tied to the missing mic. If we need to make sure the media stream for the missing device will be close, maybe we could try: 1. Use only one listener to monitor the audio device change 2. Make sure callbacks in (2) run before `CubebDeviceEnumerator::EnumerateAudioDevices` triggered by (1) 3. Always querying devices from cubeb [0]: https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/systemservices/CamerasParent.cpp#104 [1]: https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/media/webrtc/trunk/webrtc/modules/video_capture/objc/device_info_objc.mm#127 [2]: https://searchfox.org/mozilla-central/rev/8ea946dcf51f0d6400362cc1d49c8d4808eeacf1/dom/media/webrtc/CubebDeviceEnumerator.cpp#295 [3]: https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/media/libcubeb/src/cubeb_audiounit.cpp#3545-3548 [4]: https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/webrtc/CubebDeviceEnumerator.cpp#301 [5]: https://searchfox.org/mozilla-central/rev/8ea946dcf51f0d6400362cc1d49c8d4808eeacf1/dom/media/webrtc/CubebDeviceEnumerator.cpp#311,314 [6]: https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/MediaManager.cpp#2151 [7]: https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/MediaManager.cpp#2193-2207 [8]: https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/media/libcubeb/src/cubeb_audiounit.cpp#878,894 [9]: https://searchfox.org/mozilla-central/rev/8ea946dcf51f0d6400362cc1d49c8d4808eeacf1/dom/media/webrtc/CubebDeviceEnumerator.cpp#224-228 [10]: https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/MediaStreamGraph.cpp#813 [11]: https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/MediaStreamGraphImpl.h#473-477 Devices enumeration triggered by device changed events from parent process ------------------ 1) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/media/webrtc/trunk/webrtc/modules/video_capture/objc/device_info_objc.mm#127 2) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/media/webrtc/trunk/webrtc/modules/video_capture/video_capture.h#85 3) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/systemservices/CamerasParent.cpp#113 4) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/systemservices/CamerasChild.cpp#583 5) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/MediaManager.cpp#2165 6) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/MediaManager.cpp#1689 7) https://searchfox.org/mozilla-central/source/dom/media/MediaManager.cpp#1324 8) https://searchfox.org/mozilla-central/source/dom/media/webrtc/MediaEngineWebRTC.cpp#233 9) https://searchfox.org/mozilla-central/source/dom/media/webrtc/MediaEngineWebRTC.cpp#148 10) https://searchfox.org/mozilla-central/source/dom/media/webrtc/CubebDeviceEnumerator.cpp#75 11) https://searchfox.org/mozilla-central/source/dom/media/webrtc/CubebDeviceEnumerator.cpp#177 Stop media sources for the missing devices ------------------------------ 1) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/MediaManager.cpp#2206 2) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/MediaManager.cpp#4562 3) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/MediaManager.cpp#4211 4) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/MediaManager.cpp#1052 5) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/webrtc/MediaEngineWebRTCAudio.cpp#582 6) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/MediaStreamGraph.cpp#2413 7) https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/MediaStreamGraph.cpp#841 8) https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/MediaStreamGraph.cpp#785 Close input device when there is an active output device -------------------------- 1) https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/MediaStreamGraph.cpp#813,828-830 - https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/MediaStreamGraphImpl.h#465,473-478 2) https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/GraphDriver.cpp#65,81 3) https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/GraphDriver.cpp#125,138 4) https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/GraphDriver.cpp#977 5) https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/GraphDriver.cpp#109 6) https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/GraphDriver.cpp#722 7) https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/GraphDriver.cpp#438 8) https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/GraphDriver.cpp#653
Bug 1572281 Comment 2 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
It looks like there is a racing issue here on my macOS. There are two sources that can notify the devices-changed event: 1. [`InputObserver::OnDeviceChange()`][0] in `CamerasParent`, by listening [`AVCaptureDeviceWasDisconnectedNotification`][1] 2. [`CubebDeviceEnumerator::InputAudioDeviceListChanged_s`][2] for input, [`CubebDeviceEnumerator::OutputAudioDeviceListChanged_s`][4] for output, by listening [`kAudioHardwarePropertyDevices` on `kAudioObjectSystemObject`][3] The listeners in (2) are registered as the callbacks for devices-changed events in the content process directly. When the listener is fired, it will clear the cached input or output devices in [`CubebDeviceEnumerator`][5] (specifically, `mInputDevices` and `mOutputDevices`). On the other hand, the notification by (1) will end up getting to [`MediaManager::OnDeviceChange`][6] via an IPC channel from `CamerasParent`(parent process) to `CamerasChild`(content process). Then `MediaManager::OnDeviceChange` will create a task to check if there are missing devices (code path below). To check if there are missing devices, `MediaManager` will get the current devices by `CubebDeviceEnumerator` and then compare the current devices with the its cached devices (`mDeviceIDs` specifically). The missing device is the device that is in the cached device list but there is not the current device list. In the end, [the media sources for the missing devices will be stopped][7]. Therefore, it looks like we intend to close the media source if the device tied to the source is unplugged. Thus, the microphone shouldn't be switched automatically. However, the interesting thing is that the mic can be switched automatically when the device is not the default input device in the system. When the mic used in WebRTC is system default, the listeners in (2) are always run __before__ the `CubebDeviceEnumerator::EnumerateAudioDevices` triggered by (1). Therefore, the cached devices in the `CubebDeviceEnumerator` will be cleared before `MediaManager` asks for the current devices. If there is no cached device, the `CubebDeviceEnumerator` will get the devices from cubeb. Currently, if the mic is [closed][10] properly, `MediaStreamGraph::mInputDeviceID` will be set to `nullptr` and the `AudioCallbackDriver::mInputChannelCount` will [become `0`][11], so the `AudioCallbackDriver` for next iteration will be a output-only stream if there is an output device. In contrast, when the mic used in WebRTC is __not__ system default, the listeners in (2) are always run __after__ the `CubebDeviceEnumerator::EnumerateAudioDevices` triggered by (1). That is, at the time `CubebDeviceEnumerator::EnumerateAudioDevices` is called, `CubebDeviceEnumerator` will [return its cached devices directly without querying from cubeb][9]. As a result, the media source tied to the unplugged mic won't be stopped. Since [cubeb can reinitialize stream silently][8], the WebRTC works fine. I am not sure if we intend to switch the mic automatically when the active mic is unplugged. It looks like we try stopping the media source stream tied to the missing mic. If we need to make sure the media stream for the missing device will be close, maybe we could try: 1. Use only one listener to monitor the audio device change 2. Make sure callbacks in (2) run before `CubebDeviceEnumerator::EnumerateAudioDevices` triggered by (1) 3. Add a parameter `bool aManualInvalidation` to `CubebDeviceEnumerator::EnumerateAudioDevices`, if it's true then querying devices from cubeb 4. Always querying devices from cubeb [0]: https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/systemservices/CamerasParent.cpp#104 [1]: https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/media/webrtc/trunk/webrtc/modules/video_capture/objc/device_info_objc.mm#127 [2]: https://searchfox.org/mozilla-central/rev/8ea946dcf51f0d6400362cc1d49c8d4808eeacf1/dom/media/webrtc/CubebDeviceEnumerator.cpp#295 [3]: https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/media/libcubeb/src/cubeb_audiounit.cpp#3545-3548 [4]: https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/webrtc/CubebDeviceEnumerator.cpp#301 [5]: https://searchfox.org/mozilla-central/rev/8ea946dcf51f0d6400362cc1d49c8d4808eeacf1/dom/media/webrtc/CubebDeviceEnumerator.cpp#311,314 [6]: https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/MediaManager.cpp#2151 [7]: https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/MediaManager.cpp#2193-2207 [8]: https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/media/libcubeb/src/cubeb_audiounit.cpp#878,894 [9]: https://searchfox.org/mozilla-central/rev/8ea946dcf51f0d6400362cc1d49c8d4808eeacf1/dom/media/webrtc/CubebDeviceEnumerator.cpp#224-228 [10]: https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/MediaStreamGraph.cpp#813 [11]: https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/MediaStreamGraphImpl.h#473-477 Devices enumeration triggered by device changed events from parent process ------------------ 1) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/media/webrtc/trunk/webrtc/modules/video_capture/objc/device_info_objc.mm#127 2) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/media/webrtc/trunk/webrtc/modules/video_capture/video_capture.h#85 3) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/systemservices/CamerasParent.cpp#113 4) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/systemservices/CamerasChild.cpp#583 5) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/MediaManager.cpp#2165 6) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/MediaManager.cpp#1689 7) https://searchfox.org/mozilla-central/source/dom/media/MediaManager.cpp#1324 8) https://searchfox.org/mozilla-central/source/dom/media/webrtc/MediaEngineWebRTC.cpp#233 9) https://searchfox.org/mozilla-central/source/dom/media/webrtc/MediaEngineWebRTC.cpp#148 10) https://searchfox.org/mozilla-central/source/dom/media/webrtc/CubebDeviceEnumerator.cpp#75 11) https://searchfox.org/mozilla-central/source/dom/media/webrtc/CubebDeviceEnumerator.cpp#177 Stop media sources for the missing devices ------------------------------ 1) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/MediaManager.cpp#2206 2) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/MediaManager.cpp#4562 3) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/MediaManager.cpp#4211 4) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/MediaManager.cpp#1052 5) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/webrtc/MediaEngineWebRTCAudio.cpp#582 6) https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/dom/media/MediaStreamGraph.cpp#2413 7) https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/MediaStreamGraph.cpp#841 8) https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/MediaStreamGraph.cpp#785 Close input device when there is an active output device -------------------------- 1) https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/MediaStreamGraph.cpp#813,828-830 - https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/MediaStreamGraphImpl.h#465,473-478 2) https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/GraphDriver.cpp#65,81 3) https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/GraphDriver.cpp#125,138 4) https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/GraphDriver.cpp#977 5) https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/GraphDriver.cpp#109 6) https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/GraphDriver.cpp#722 7) https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/GraphDriver.cpp#438 8) https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/dom/media/GraphDriver.cpp#653