1546237 - WebRTC output going to "Default Communications Device"

Status: RESOLVED FIXED

(Core :: WebRTC: Audio/Video, defect, P2)

Description

[Greg Fodor]

In Nightly 68 as of 4/22 it seems audio output goes to the windows "default communications device" when a microphone is active. To reproduce, go to:

https://janus.conf.meetecho.com/echotest.html

On Stable, the echo from your microphone will go to whatever device you have set as the default output in Windows. On nightly, it will go to the "default communications device". You can see what your default communications device is via the legacy audio panel via run->mmsys.cpl -- right clicking on output devices allows selection of the default communications device.

Comment 1

[Nils Ohlmeier [:drno]]

Paul, are you aware of any changes we landed recently which would explains Firefox now apparently using the default communication device?

Comment 2

[Alex Chronopoulos [:achronop]]

I think this is intentional per Bug 1531833, but Paul knows better.

Comment 3

[Paul Adenot (:padenot)]

Yes, this is intentional. When using the input device for voice, we now prefer communication devices.

If we use gUM with voice-filtering disabled, we'll use the default audio devices.

Comment 4

[Greg Fodor]

This seems really problematic, because currently it doesn't seem that windows 10 offers a way for the user to change the default communication device other than manually running the legacy sound control panel mmsys.cpl. In practice, this has resulted in a lot of audio failure modes for Hubs (https://hubs.mozilla.com) users who are using nightly, who often unknowingly have the default communication device set to something other than their PC speakers, such as an HMD or bluetooth headset. So, as soon as we try to access their computer's microphone to let them choose which microphone they want to use, audio cuts out from their speakers and its unclear what is going on or how to fix it.

I'd highly suggest re-considering this change. It seems that at the very least the browser should surface audio output options here, since currently the only way for a user to correct the issue is to manually run the legacy control panel. Previously, a user could just use the user-facing windows 10 output chooser to direct audio to the proper output, but now with this change the output is effectively locked to a specific device (typically a pair of headphones or an attached VR HMD) with no clear way to change it.

Comment 5

[Greg Fodor]

Two specific scenarios here we've been hitting:

• User has a bluetooth headset paired with their computer that registered itself as the default communications device (often unknown to them) but they want to use their PC speakers + mic for a session (and may not even realize the headset is connected.) As soon as we try to access their mic, audio will cut over to the headset they are not wearing so it just cuts out audio.

• User has an HMD attached to their computer (eg Rift or Vive) and, sometimes they want to use it for audio input/output, sometimes they don't. If the communication device is set to their HMD, it's impossible for them to use their desktop speakers and mic, bluetooth headset or mic, etc without changing the setting via the legacy control panel. Alternatively, if its not set to their HMD, it ends up being 'impossible' to get audio to go to their HMD when they want to use VR, unless they know how to switch it manually

Comment 6

[Greg Fodor]

One thing that may have not been clear from the writeup above -- the concern here isn't with the use of the communications device as the default input/microphone. That certainly seems like a good default. The issue here is that it seems that when using a voice input of any kind (including a microphone that does not correspond to the default communications device) the output is forcibly set to the default communications device's output in windows, which cannot be changed other than manually running mmsys.cpl.

Perhaps this is actually just a bug. Was it intentional for the output to change as well? If so, that doesn't seem like a bad idea either, as long as the output is also only switched if the user is actually using the default communication device as input. As it is right now, it seems like the output is always switched to the default communication device, regardless of what input the user is currently using for voice.

Comment 7

[Paul Adenot (:padenot)]

Alex, is the physical matching device ready enough to use in this case, to use the audio output device that matches the audio input device picked by the user?

Comment 8

[Alex Chronopoulos [:achronop]]

Regarding audio devices physical matching is 100%. You can use the group_id member of cubeb_device_info [1]. However my understanding is that you don't need that. You need to update the following set of output.pref in [2] to CUBEB_DEVICE_PREF_NONE when the selected input is not the default communication device.

[1] https://searchfox.org/mozilla-central/rev/b756e6d00728dda4121f8278a744381d8643317a/media/libcubeb/include/cubeb.h#343
[2] https://searchfox.org/mozilla-central/rev/b756e6d00728dda4121f8278a744381d8643317a/dom/media/GraphDriver.cpp#619-622

Comment 9

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Paul, do we have a document outlining the overall design here? Having Firefox's audio stack make low-level inferences about where audio should go based on microphone usage in the moment seems surprising: See "Alice can only hear Bob when she is self muted" from bug 1549467.

Setting aside how the Dell monitor in bug 1549467 got configured as a "default communication device" when it doesn't have audio inputs, even if Alice were to have an audible default communications device configured, AND even if she were aware of this, she would still experience Bob's audio as bouncing between coming out of that device vs. her regular speakers whenever she mutes/unmutes her own microphone (recall we relinquish mic on mute), which seems undesirable and surprising.

Also, what happens if Alice opens 2 mics? Where should audio output go then? These feel like app decisions. The browser here seems to be assuming too much about the nature of the app, specifically that it is a calling app. The spec model so far has been to outsource decisions like these to the app, using setSinkId.

While there might be some room when element.sinkId == "" to say the user-agent default device refers to some smart browser default not in the enumerated list—Firefox recently got rid of its "Default (xxx)" choice in the list from enumerateDevices—I think most users and browser vendors will assume the default to be a dumb device and one of the ones listed. I therefore worry smart behaviors here, as useful as they may seem to the targeted calling use-cases, are actually web compat issues.

I think we need to work out APIs to limit these behaviors to apps that want them, and I think we need to restore behavior—at least on Windows,I haven't looked at android—until we do.

Comment 11

[Paul Adenot (:padenot)]

(In reply to Jan-Ivar Bruaroey [:jib] (needinfo? me) from comment #9)

Paul, do we have a document outlining the overall design here? Having Firefox's audio stack make low-level inferences about where audio should go based on microphone usage in the moment seems surprising: See "Alice can only hear Bob when she is self muted" from bug 1549467.

Yes, in a comment on a patch you're set to review (1549399), I've outlined the ideal behaviour (more or less a decision tree).

Setting aside how the Dell monitor in bug 1549467 got configured as a "default communication device" when it doesn't have audio inputs, even if Alice were to have an audible default communications device configured, AND even if she were aware of this, she would still experience Bob's audio as bouncing between coming out of that device vs. her regular speakers whenever she mutes/unmutes her own microphone (recall we relinquish mic on mute), which seems undesirable and surprising.

This is an implementation bug, fixed in 1549399.

Also, what happens if Alice opens 2 mics? Where should audio output go then? These feel like app decisions. The browser here seems to be assuming too much about the nature of the app, specifically that it is a calling app. The spec model so far has been to outsource decisions like these to the app, using setSinkId.

This cannot happen in Firefox. Even if it could happen, apps that use AudioContext for audio output, don't have setSinkId. Even if we had setSinkId on AudioContext, Firefox is not ready for the code and it requires a lot of work.

If it's not a calling app, then it does not want AEC. If this is the case, the normal device is used, not the communication device.

While there might be some room when element.sinkId == "" to say the user-agent default device refers to some smart browser default not in the enumerated list—Firefox recently got rid of its "Default (xxx)" choice in the list from enumerateDevices—I think most users and browser vendors will assume the default to be a dumb device and one of the ones listed. I therefore worry smart behaviors here, as useful as they may seem to the targeted calling use-cases, are actually web compat issues.

I'd like to have an entry that is "system default" and tracks the system default, even when it changes. That would be useful.

I think we need to work out APIs to limit these behaviors to apps that want them, and I think we need to restore behavior—at least on Windows,I haven't looked at android—until we do.

Yes, I'll do that, it's code freeze and it's simple enough. In terms of API, I've been telling this to spec people for years, but nobody cares it seems. Tagging audio input and output objects in terms of use case is essential.

Comment 12

[Paul Adenot (:padenot)]

Fixed by 1549699.

Comment 13

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1663917