1329976 - getUserMedia(audio, video) when already capturing audio fails

Status: RESOLVED FIXED

(Core :: WebRTC: Audio/Video, defect, P1)

Description

[Igor Sheko]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36

Steps to reproduce:

On Firefox for macOS run. Bug not reproduce at another platforms!
 
var stream, stream2; 
navigator.mediaDevices.getUserMedia({ audio: true, video: false })
	.then(function(s) { 
		stream = s; 
		console.log('audio success'); 
		navigator.mediaDevices.getUserMedia({ audio: true, video: true }).then(function(s) { stream2 = s; console.log('audio & video success'); }).catch(function(e) { console.log(e); });
	})
	.catch(function(e) { console.log(e); });


Actual results:

Got 
audio success 
MediaStreamError { name: "NotReadableError", message: "Failed to allocate videosource", constraint: "", stack: "" }


Expected results:

audio success 
audio & video success

Comment 1

[Randell Jesup [:jesup] (needinfo me)]

Try https://mozilla.github.io/webrtc-landing/gum_test.html  -- does it work for audio-only, video-only, audio+video?

Note that per the spec, if you ask for audio and video, and either is not available (such as having a mic/line-in, but not having a camera) the spec requires that the getUserMedia fail.  Chrome doesn't do this.

Does your test work for audio-only or for video-only?

Comment 2

[Randell Jesup [:jesup] (needinfo me)]

I see the audio-only worked in the test... Does video-only first work?  Does an approval doorhanger request appear for the second gUM?  I presume it doen't.

Comment 3

[Alex Chronopoulos [:achronop]]

I also get the reported error. 

Answering Randell's questions for my case:
When I change the first gUM to video-only or to audio+video the error does not happen.

The second permission prompt appears even when the error occurs.

In addition when I change the second gUM to audio-only or video-only the error does not happen.

Comment 4

[Randell Jesup [:jesup] (needinfo me)]

Does it happen if the second gum call isn't in the callback?  (do it from a setTimeout or some such)

The result seems similar to the problems with opening a new separate mic in order to switch mics, but that shouldn't be happening here unless they selected a different mic when the doorhanger appeared (and alex indicates the error is before that).

Comment 5

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

This reproduces 100% for me with this fiddle: http://jsfiddle.net/jt6kdcpe/

I even put in a 2 second delay to show it is not timing sensitive.

Comment 6

[Randell Jesup [:jesup] (needinfo me)]

ok, removing NI to reporter

Comment 7

[Randell Jesup [:jesup] (needinfo me)]

Oddly, this repro's with jib's fiddle on Linux m-c

Comment 8

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Workaround: Make sure to pick the same audio device both times.

The bug seems to have to do with the fact that the permission prompt picks a different audio device by default each time. Florian, is this some attempt at memory at play?

Comment 9

[Florian Quèze [:florian]]

(In reply to Jan-Ivar Bruaroey [:jib] from comment #8)
> Workaround: Make sure to pick the same audio device both times.
> 
> The bug seems to have to do with the fact that the permission prompt picks a
> different audio device by default each time. Florian, is this some attempt
> at memory at play?

Unless there's a bug in it, the prompt shows the list in the order it has been returned by mozGetUserMediaDevices, and selects the first one.

Comment 10

[Randell Jesup [:jesup] (needinfo me)]

(In reply to Florian Quèze [:florian] [:flo] from comment #9)
> Unless there's a bug in it, the prompt shows the list in the order it has
> been returned by mozGetUserMediaDevices, and selects the first one.

The first time (audio-only), it picks the first entry.  If you request audio-only again, it will give the first entry.  If you request audio-only, then request audio+video (I used two gum_test tabs) it will use the first entry for that too. If you then Stop both, and request audio-only again, the selected entry now is the *second* entry.  Note: until this point all I've done is hit the Audio button and Audio+video, and on the doorhanger just selected Allow.  No changing of entries anywhere.

Asking for video-only in the second tab doesn't cause the bug, only asking for audio+video.

Florian - what do you think?

Comment 11

[Igor Sheko]

Hello! I'm have some additional info about reproduce environment. This is new macbook pro late 2016 (it always have audio and video sources) macOS 10.12.2 .

For 100% reproduce, you need add MediaTracks to PeerConnection at another tab, create call, then close this PeerConnection and tab. From this point, at any new tab you will receive access error.

Looks like PC destructor locks and doesn't release MediaTracks.

Comment 12

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

We're already able to reproduce it in comment 5.

Comment 13

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

(In reply to Florian Quèze [:florian] [:flo] from comment #9)
> Unless there's a bug in it, the prompt shows the list in the order it has
> been returned by mozGetUserMediaDevices, and selects the first one.

Instrumented testing shows it does not:

+++ b/browser/modules/ContentWebRTC.jsm
@@ -156,18 +156,21 @@ function prompt(aContentWindow, aWindowI
 
   // MediaStreamConstraints defines video as 'boolean or MediaTrackConstraints'.
   let video = aConstraints.video || aConstraints.picture;
   let audio = aConstraints.audio;
   let sharingScreen = video && typeof(video) != "boolean" &&
                       video.mediaSource != "camera";
   let sharingAudio = audio && typeof(audio) != "boolean" &&
                      audio.mediaSource != "microphone";
+  let ii = 0;
   for (let device of aDevices) {
     device = device.QueryInterface(Ci.nsIMediaDevice);
+    let d = device;
+    dump("JIB: "+ (ii++) +": "+ d.type +" "+ d.mediaSource +" - "+ d.name +"\n");

JIB: 0: audio microphone - default: External Microphone
JIB: 1: audio microphone - External Microphone
JIB: 0: audio microphone - default: External Microphone
JIB: 1: audio microphone - External Microphone
JIB: 0: video camera - FaceTime HD Camera (Built-in)
JIB: 1: audio microphone - default: External Microphone
JIB: 2: audio microphone - External Microphone

Having deja vu, I think the problem is bug 989094 comment 3.

Comment 14

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

So this one is in Florian's court I think. Should I move it to Device Permissions, or do you want to open a bug blocking this one, so we can keep track of it in WebRTC?

Basically, desktop has an inadvertent "feature" that's sometimes desirable, and sometimes not, except nobody noticed the "wrong" device before, since it's in practice the same mic ("foo" vs "default (foo)"),  except now we really notice, because we now fail over multi-mic, because full duplex.

If we remove it, some may complain. If we don't, well people are already complaining. I should probably look at bug 1212501 again, but that'll take some time, so we probably need to disable this "memory" to fix the short-term problem.

Comment 15

[Florian Quèze [:florian]]

Attachment: Patch

It's not clear to me why we are failing to allocate the microphone in some cases, but there's indeed some inconsistent behavior of the microphone selector that needs fixing.
Let me know if this is all we need to fix the bug; if so it can be moved to Device Permissions.

Comment 16

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Since full duplex landed only one microphone can be allocated at a time. See bug 1329976.

Comment 17

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Comment on attachment 8826251 [details] [diff] [review]
Patch

Review of attachment 8826251 [details] [diff] [review]:
-----------------------------------------------------------------

Lgtm.

Comment 18

[Johann Hofmann [:johannh]]

Comment on attachment 8826251 [details] [diff] [review]
Patch

Review of attachment 8826251 [details] [diff] [review]:
-----------------------------------------------------------------

I think this solution is fine, the reset didn't really seem intrusive or annoying from trying it myself.

Adding a test for this would be really nice. You might run into problems with getting enough devices listed on CI machines, though. So r+ without tests, but please look into using media.navigator.permission.fake to make a small testcase for this.

::: browser/modules/webrtcUI.jsm
@@ +482,5 @@
>  
>        function listDevices(menupopup, devices) {
>          while (menupopup.lastChild)
>            menupopup.removeChild(menupopup.lastChild);
> +        menupopup.parentNode.removeAttribute("value");

Please leave a comment here saying _why_ this needs to be done. While you're at it maybe amend the comment below too, it's not really useful.

@@ +491,5 @@
>  
>        function listScreenShareDevices(menupopup, devices) {
>          while (menupopup.lastChild)
>            menupopup.removeChild(menupopup.lastChild);
> +        menupopup.parentNode.removeAttribute("value");

As mentioned on IRC, this is already done for screen sharing http://searchfox.org/mozilla-central/source/browser/modules/webrtcUI.jsm#547

Comment 19

[Florian Quèze [:florian]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/bc99b50404754b0e0d1637defcf2d3bebe0fe912
Bug 1329976 - make the microphone prompt select the same device by default all the time by clearing the leftover value from previous prompts, r=johannh.

Comment 20

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/bc99b5040475