1522488 - WebRTC: cannot switch microphone to another one without page refresh

Status: RESOLVED FIXED

(Core :: WebRTC: Audio/Video, defect, P2)

Description

[Victor Portnov]

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36

Steps to reproduce:

Equipment:
I use two USB headsets (earphone +microphone) to reproduce the issue but probably just two microphones are required.
Please note that some of devices are fine (I have one no name headset which works fine in the scenario below) but most of them fail in the scenario below (I tried Plantronics, Sennheiser and Logitech additionally to no name one and those three failed the test).

Initial conditions:

1. there is code ( see here https://jsbin.com/requzamofi/1/edit?js,console and also below) which:

• takes a microphone stream via windows.navigator.mediaDevices.getUserMedia and store it
• uses window.navigator.mediaDevices.ondevicechange callback to detect when the new device is plugged / unplugged.
• if the notification comes, it re-requests microphone stream

2. microphone permission is not set for the site, so Firefox asks the user to allow to use this or that microphone each time when windows.navigator.mediaDevices.getUserMedia is called

3. headset 1 and headset 2 are plugged in; headset 1 is default device in the system

Steps:

1. launch the code ( https://jsbin.com/requzamofi/1/edit?js,console )
2. when the page asks an access to microphone, allow to use microphone of headset 1
3. look at console output
4. unplug headset 1
5. when the page asks an access to microphone, allow to use microphone of headset 2
6. look at console output
7. plug headset 1 back
8. look at console output

JavaScript to test with:

var microphone = null;

var micName = function (mic) {
return mic.getTracks()[0].label;
};

var connectMicrophone = function () {
console.log('connecting microphone ...');

window.navigator.mediaDevices.getUserMedia({audio: true, video: false})
.then(function (mic) {
console.log('got microphone ' + micName(mic));
microphone = mic;
})
.catch(function (error) {
console.error('cannot get microphone due to error ' + error);
});
};

window.navigator.mediaDevices.ondevicechange = function () {
console.log('devices changed');

microphone = null;
connectMicrophone();
};

connectMicrophone();

Actual results:

1. on step 3 console output is like the following:
   "connecting microphone ..."
   "got microphone Headset Microphone (Plantronics BT600)"

2. on step 6 console output is like the following:
   "devices changed"
   "connecting microphone ..."
   "cannot get microphone due to error AbortError: Starting audio failed"

... and microphone does not work for the page

3. on step 8 console output is like the following:
   "devices changed"
   "connecting microphone ..."

... and after that it does not show microphone request dialog and does not connect the microphone silently.

Expected results:

1. on step 3 it works as expected

2. on step 6 it should be something like the following in console:
   "devices changed"
   "connecting microphone ..."
   "got microphone Microphone (USB Audio Device)"

3. on step 8 it should be something like the following in console:
   "devices changed"
   "connecting microphone ..."
   "got microphone Microphone (USB Audio Device)"

... (I'm assuming that if device is given already, no reason to change it; however, up to you to think differently and to switch to default device back; also probably it makes sense to show microphone request dialog again)

Comment 1

[Victor Portnov]

Sorry, I was not aware about markdown and was not able to preview the report.
Just in case, code sample is the following:

var microphone = null;

var micName = function (mic) {
  return mic.getTracks()[0].label;
};

var connectMicrophone = function () {
  console.log('connecting microphone ...');
  
  window.navigator.mediaDevices.getUserMedia({audio: true, video: false})
    .then(function (mic) {
      console.log('got microphone ' + micName(mic));
      microphone = mic;
    })
    .catch(function (error) {
      console.error('cannot get microphone due to error ' + error);
    });
};

window.navigator.mediaDevices.ondevicechange = function () {
  console.log('devices changed');

  microphone = null;
  connectMicrophone();
};

connectMicrophone();

Comment 3

[Victor Portnov]

Sorry, I don't agree that this is duplicate of bug880312.

bug880312 is about the way to handle things automatically.
So it is about the new feature of Firefox.
It would be nice to have but I'm not asking it in the bug I have created.

In the bug I'm asking about a fix for functionality which is already in Firefox.
getUserMedia method works with the bug and it is good to have it fixed without introducing the new functionality into the product.
Please note that for some headsets it works already.
So it is worth to verify what is wrong in the code when it initiates a stream for other headsets.

BTW, in Firefox 65 the behaviour is the same as Actual result for the bug.

Comment 4

[Alex Chronopoulos [:achronop]]

I can reproduce this one. I am comparing with Linux and it does not happen there. On Linux I get the expected begavior. It appears on Windows though.

Comment 5

[Victor Portnov]

(In reply to Alex Chronopoulos [:achronop] from comment #4)

I can reproduce this one. I am comparing with Linux and it does not happen there. On Linux I get the expected begavior. It appears on Windows though.

I was able to reproduce it on Mac either.
Please note still that with some headsets it is not reproducible.

I'm not sure, what is the reason for that but I have a guess:
it is not reproducible with very simple headsets which most probably use a system driver.
On the other hand, Plantronics, Sennheiser and Logitech most probably use custom drivers.
It could be that this difference is worth in understanding why it does not work for branded devices.

Comment 6

[Alex Chronopoulos [:achronop]]

Hmm I am using Logitech Webcam C930e (which contains mic) and I cannot repro on Mac. Then I tried Plantronics .Audio 648 USB and I can repro on Mac, latest Nightly. That's a strange behavior ...

Comment 7

[Alex Chronopoulos [:achronop]]

We abort here [1]. It looks like a timing issue between MSG and MediaManager thread. That must be the reasons it does not repro under the debugger.

After the device has been unplugged the method MediaStreamGraphImpl::InputDeviceID(), which returns the value of MediaStreamGraphImpl::mInputDeviceID, must return null. On the failed case it return the previous device id instead. The MediaStreamGraphImpl::mInputDeviceID is cleaned up (reset to NULL) in MediaStreamGraphImpl::CloseAudioInputImpl. From the logs I see that the close method is called in the good case but it is not called in the bad case. So, the question is why.

[1] https://searchfox.org/mozilla-central/rev/4faab2f1b697827b93e16cb798b22b197e5235c9/dom/media/webrtc/MediaEngineWebRTCAudio.cpp#569

Comment 8

[Alex Chronopoulos [:achronop]]

Attachment: Bug 1522488 - Increase wait time before a new device enumeration after a device change event to avoid getting outdated list. r?padenot

The problem here is that the MediaStreamGraphImpl::mInputDeviceID is not reset after a device unplug. On a device-remove event of the active device MediaStreamGraphImpl::mInputDeviceID is reset when MediaStreamGraphImpl::CloseAudioInput is being executed. This method will not be executed if the new enumeration, on the MediaManager::OnDeviceChange, is returning the old outdated list of devices. This patch increases the wait time before the new enumeration in order to allow the system to update the provided device list.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fb45e3d534cd
Increase wait time before a new device enumeration after a device change event to avoid getting outdated list. r=padenot

Comment 10

[Alex Chronopoulos [:achronop]]

In the review Paul pointed out that the best mechanism for that would be to wait on the device-change-callback from cubeb instead of a static wait time, in order to enumerate a new device list. This requires us to implement that cubeb method in wasapi backend, which is missing. Linux and OSX have it. Also, it requires some MediaManager changes in order to call MediaManager::OnDeviceChange from the cubeb callback for audio devices. I will start doing that in the following days.

Comment 11

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/fb45e3d534cd

Comment 12

[Alex Chronopoulos [:achronop]]

The implementation described in comment 10 is blocked by https://github.com/djg/audioipc-2/issues/50