getUserMedia(audio, video) when already capturing audio fails

RESOLVED FIXED in Firefox 53

Status

()

P1
normal
Rank:
13
RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: i, Assigned: florian)

Tracking

50 Branch
mozilla53
x86_64
All
Points:
---

Firefox Tracking Flags

(firefox53 fixed)

Details

Attachments

(1 attachment)

(Reporter)

Description

2 years ago
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36

Steps to reproduce:

On Firefox for macOS run. Bug not reproduce at another platforms!
 
var stream, stream2; 
navigator.mediaDevices.getUserMedia({ audio: true, video: false })
	.then(function(s) { 
		stream = s; 
		console.log('audio success'); 
		navigator.mediaDevices.getUserMedia({ audio: true, video: true }).then(function(s) { stream2 = s; console.log('audio & video success'); }).catch(function(e) { console.log(e); });
	})
	.catch(function(e) { console.log(e); });


Actual results:

Got 
audio success 
MediaStreamError { name: "NotReadableError", message: "Failed to allocate videosource", constraint: "", stack: "" }


Expected results:

audio success 
audio & video success
(Reporter)

Updated

2 years ago
OS: Unspecified → Mac OS X
Hardware: Unspecified → x86_64

Updated

2 years ago
Component: Untriaged → WebRTC: Audio/Video
Product: Firefox → Core
Try https://mozilla.github.io/webrtc-landing/gum_test.html  -- does it work for audio-only, video-only, audio+video?

Note that per the spec, if you ask for audio and video, and either is not available (such as having a mic/line-in, but not having a camera) the spec requires that the getUserMedia fail.  Chrome doesn't do this.

Does your test work for audio-only or for video-only?
Flags: needinfo?(i)
Whiteboard: [Needinfo 2017/1/11 to reporter]
I see the audio-only worked in the test... Does video-only first work?  Does an approval doorhanger request appear for the second gUM?  I presume it doen't.
I also get the reported error. 

Answering Randell's questions for my case:
When I change the first gUM to video-only or to audio+video the error does not happen.

The second permission prompt appears even when the error occurs.

In addition when I change the second gUM to audio-only or video-only the error does not happen.
Does it happen if the second gum call isn't in the callback?  (do it from a setTimeout or some such)

The result seems similar to the problems with opening a new separate mic in order to switch mics, but that shouldn't be happening here unless they selected a different mic when the doorhanger appeared (and alex indicates the error is before that).
Status: UNCONFIRMED → NEW
Rank: 15
Ever confirmed: true
Flags: needinfo?(mchiang)
Flags: needinfo?(jib)
Priority: -- → P1
This reproduces 100% for me with this fiddle: http://jsfiddle.net/jt6kdcpe/

I even put in a 2 second delay to show it is not timing sensitive.
Flags: needinfo?(jib)
ok, removing NI to reporter
Flags: needinfo?(i)
Whiteboard: [Needinfo 2017/1/11 to reporter]
Oddly, this repro's with jib's fiddle on Linux m-c
Flags: needinfo?(mchiang)

Updated

2 years ago
Rank: 15 → 13
OS: Mac OS X → All
Summary: getUserMedia not working on macOS → getUserMedia(audio, video) when already capturing audio fails
Workaround: Make sure to pick the same audio device both times.

The bug seems to have to do with the fact that the permission prompt picks a different audio device by default each time. Florian, is this some attempt at memory at play?
Flags: needinfo?(florian)
(Assignee)

Comment 9

2 years ago
(In reply to Jan-Ivar Bruaroey [:jib] from comment #8)
> Workaround: Make sure to pick the same audio device both times.
> 
> The bug seems to have to do with the fact that the permission prompt picks a
> different audio device by default each time. Florian, is this some attempt
> at memory at play?

Unless there's a bug in it, the prompt shows the list in the order it has been returned by mozGetUserMediaDevices, and selects the first one.
Flags: needinfo?(florian)
(In reply to Florian Quèze [:florian] [:flo] from comment #9)
> Unless there's a bug in it, the prompt shows the list in the order it has
> been returned by mozGetUserMediaDevices, and selects the first one.

The first time (audio-only), it picks the first entry.  If you request audio-only again, it will give the first entry.  If you request audio-only, then request audio+video (I used two gum_test tabs) it will use the first entry for that too. If you then Stop both, and request audio-only again, the selected entry now is the *second* entry.  Note: until this point all I've done is hit the Audio button and Audio+video, and on the doorhanger just selected Allow.  No changing of entries anywhere.

Asking for video-only in the second tab doesn't cause the bug, only asking for audio+video.

Florian - what do you think?
Flags: needinfo?(florian)
(Reporter)

Comment 11

2 years ago
Hello! I'm have some additional info about reproduce environment. This is new macbook pro late 2016 (it always have audio and video sources) macOS 10.12.2 .

For 100% reproduce, you need add MediaTracks to PeerConnection at another tab, create call, then close this PeerConnection and tab. From this point, at any new tab you will receive access error.

Looks like PC destructor locks and doesn't release MediaTracks.
We're already able to reproduce it in comment 5.
(In reply to Florian Quèze [:florian] [:flo] from comment #9)
> Unless there's a bug in it, the prompt shows the list in the order it has
> been returned by mozGetUserMediaDevices, and selects the first one.

Instrumented testing shows it does not:

+++ b/browser/modules/ContentWebRTC.jsm
@@ -156,18 +156,21 @@ function prompt(aContentWindow, aWindowI
 
   // MediaStreamConstraints defines video as 'boolean or MediaTrackConstraints'.
   let video = aConstraints.video || aConstraints.picture;
   let audio = aConstraints.audio;
   let sharingScreen = video && typeof(video) != "boolean" &&
                       video.mediaSource != "camera";
   let sharingAudio = audio && typeof(audio) != "boolean" &&
                      audio.mediaSource != "microphone";
+  let ii = 0;
   for (let device of aDevices) {
     device = device.QueryInterface(Ci.nsIMediaDevice);
+    let d = device;
+    dump("JIB: "+ (ii++) +": "+ d.type +" "+ d.mediaSource +" - "+ d.name +"\n");

JIB: 0: audio microphone - default: External Microphone
JIB: 1: audio microphone - External Microphone
JIB: 0: audio microphone - default: External Microphone
JIB: 1: audio microphone - External Microphone
JIB: 0: video camera - FaceTime HD Camera (Built-in)
JIB: 1: audio microphone - default: External Microphone
JIB: 2: audio microphone - External Microphone

Having deja vu, I think the problem is bug 989094 comment 3.
So this one is in Florian's court I think. Should I move it to Device Permissions, or do you want to open a bug blocking this one, so we can keep track of it in WebRTC?

Basically, desktop has an inadvertent "feature" that's sometimes desirable, and sometimes not, except nobody noticed the "wrong" device before, since it's in practice the same mic ("foo" vs "default (foo)"),  except now we really notice, because we now fail over multi-mic, because full duplex.

If we remove it, some may complain. If we don't, well people are already complaining. I should probably look at bug 1212501 again, but that'll take some time, so we probably need to disable this "memory" to fix the short-term problem.
(Assignee)

Comment 15

2 years ago
Created attachment 8826251 [details] [diff] [review]
Patch

It's not clear to me why we are failing to allocate the microphone in some cases, but there's indeed some inconsistent behavior of the microphone selector that needs fixing.
Let me know if this is all we need to fix the bug; if so it can be moved to Device Permissions.
Attachment #8826251 - Flags: feedback?(jib)
(Assignee)

Updated

2 years ago
Assignee: nobody → florian
Status: NEW → ASSIGNED
Since full duplex landed only one microphone can be allocated at a time. See bug 1329976.
Comment on attachment 8826251 [details] [diff] [review]
Patch

Review of attachment 8826251 [details] [diff] [review]:
-----------------------------------------------------------------

Lgtm.
Attachment #8826251 - Flags: feedback?(jib) → feedback+
(Assignee)

Updated

2 years ago
Flags: needinfo?(florian)
Attachment #8826251 - Flags: review?(jhofmann)
Comment on attachment 8826251 [details] [diff] [review]
Patch

Review of attachment 8826251 [details] [diff] [review]:
-----------------------------------------------------------------

I think this solution is fine, the reset didn't really seem intrusive or annoying from trying it myself.

Adding a test for this would be really nice. You might run into problems with getting enough devices listed on CI machines, though. So r+ without tests, but please look into using media.navigator.permission.fake to make a small testcase for this.

::: browser/modules/webrtcUI.jsm
@@ +482,5 @@
>  
>        function listDevices(menupopup, devices) {
>          while (menupopup.lastChild)
>            menupopup.removeChild(menupopup.lastChild);
> +        menupopup.parentNode.removeAttribute("value");

Please leave a comment here saying _why_ this needs to be done. While you're at it maybe amend the comment below too, it's not really useful.

@@ +491,5 @@
>  
>        function listScreenShareDevices(menupopup, devices) {
>          while (menupopup.lastChild)
>            menupopup.removeChild(menupopup.lastChild);
> +        menupopup.parentNode.removeAttribute("value");

As mentioned on IRC, this is already done for screen sharing http://searchfox.org/mozilla-central/source/browser/modules/webrtcUI.jsm#547
Attachment #8826251 - Flags: review?(jhofmann) → review+
(Assignee)

Comment 19

2 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/bc99b50404754b0e0d1637defcf2d3bebe0fe912
Bug 1329976 - make the microphone prompt select the same device by default all the time by clearing the leftover value from previous prompts, r=johannh.

Comment 20

2 years ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/bc99b5040475
Status: ASSIGNED → RESOLVED
Last Resolved: 2 years ago
status-firefox53: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla53
You need to log in before you can comment on or make changes to this bug.