Closed Bug 1299324 Opened 4 years ago Closed 3 years ago

Switching microphones in Firefox 49 hangs video feed, returns error

Categories

(Core :: WebRTC: Audio/Video, defect, P1)

49 Branch
defect

Tracking

()

VERIFIED FIXED
mozilla52
Tracking Status
firefox49 --- wontfix
firefox50 --- verified
firefox51 --- verified
firefox52 --- verified
firefox53 --- verified

People

(Reporter: jeremy.noring, Assigned: jesup, NeedInfo)

References

Details

(Keywords: regression)

Attachments

(1 file)

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36

Steps to reproduce:

1. Go to https://app.hirevue.com/interviews/Prsxq4u-8kghiu/manager/AqxvMX8jqVVg94fSVnk2ML/#/ on a system with multiple microphones.

2. Click "let's get started"

3. Once the connection test passes, try to change microphones using the drop-down box.


Actual results:

In Firefox 49, changing microphone results in: "ERROR: Failed to get access to local media. Error: SourceUnavailableError, message:Failed to allocate audiosource"; the "SourceUnavailableError" causes this page to go to an error page.

In Firefox Nightly, this simply hangs the video feed and provides no output at all.


Expected results:

The microphone should have been changed.  This code worked in previous versions of Firefox (48 and below) without issue.

I believe the problem may be that we don't officially shut down the previous media stream we got from getUserMedia() until we get a successful response from the second getUserMedia call?
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:51.0) Gecko/20100101 Firefox/51.0

I have tested this issue on Mac OS 10.10 with the latest Firefox release (48.0.2), Firefox (49.0b9-20160901141622) and the latest Nightly (51.0a1-20160901030202) and could not reproduce it.
Due to the fact that we couldn't test this issue on the website provided in step 1 from the description, we have tested this using https://opentokrtc.com/, here, after creating a meeting you are prompt with a dialog where a drop-down box is displayed with multiple available microphone options. After choosing the desired microphone and webcam, the meeting starts, without any sound or video issues.

Can you please retest this using the latest Firefox release and  the latest Nightly build (https://nightly.mozilla.org/) and report back the results? When doing this, please use a new clean Firefox profile, maybe even safe mode, to eliminate custom settings as a possible cause (https://goo.gl/PNe90E).

Also can you provide us with any testing credentials in order to log in to the website provided in the description and try to retest this issue?
Flags: needinfo?(jeremy.noring)
(In reply to Emil Pasca [:emilpasca] from comment #1)
> Can you please retest this using the latest Firefox release and  the latest
> Nightly build (https://nightly.mozilla.org/) and report back the results?
> When doing this, please use a new clean Firefox profile, maybe even safe
> mode, to eliminate custom settings as a possible cause
> (https://goo.gl/PNe90E).

Retested with 51.0a1 (2016-09-06); it still reproduces for me

> 
> Also can you provide us with any testing credentials in order to log in to
> the website provided in the description and try to retest this issue?


Try testing here instead: https://dev-licode1.hirevue.net:3004/ - try changing microphones repeatedly and you'll see your video freeze.  The same thing works flawlessly in Chrome.
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:51.0) Gecko/20100101 Firefox/51.0

I have retested this issue on Mac OS 10.11 and Windows 10 x64 with the latest Firefox release (49.0) and the latest Nightly (51.0a1-20160919030422) and managed to reproduce it.
After navigating to the link provided in Comment 2, when changing the microphones repeatedly the video freezes.

I have performed a regression on Windows 10 x64 and here are the results:

Last good revision: 90bd0f479ed71861ce2b9ebf9278f3416d3e537b
First bad revision: a81573e42b84cbf07aef10085aa0609201dfe3fa
Pushlog:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=90bd0f479ed71861ce2b9ebf9278f3416d3e537b&tochange=a81573e42b84cbf07aef10085aa0609201dfe3fa

Looks like the following bug has the changes which introduced the regression:
https://bugzilla.mozilla.org/show_bug.cgi?id=1243857

Because of a few technical issues, I haven't managed to perform regression on Mac OS, however, I will try later and come back with the results.
Status: UNCONFIRMED → NEW
Component: Untriaged → WebRTC: Audio/Video
Ever confirmed: true
Flags: needinfo?(jeremy.noring)
Product: Firefox → Core
P1 for a regression. We should get this boiled down in a fiddle or mochitest to ease debugging.
Rank: 15
Keywords: regression
Priority: -- → P1
I have performed a regression on Mac OS X 10.11 and received the same results as on Windows.
Alex-- Can you work with the reporter to track this down and identify a fix?  It looks like it's related to full duplex even though full duplex is pref'd off for Mac and Windows in Fx 49.  Thanks.
Assignee: nobody → achronop
I repro the issue even without multiply mics. Even with just one mic the gUM prompt show 2 options which correspond to the same device, the default and the direct choose of the mic. When I switch between those 2 options the video freeze and I have to refresh to make it work.
All the above happen with full duplex preff on. But the same behavior is reproducible with pref off.
I get the following assert on debug:

[Child 73605] ###!!! ASSERTION: Input from multiple mics not yet supported; bug 1238038: 'false', file <repo>/mozilla/firefox/dom/media/MediaStreamGraph.cpp, line 937
How did this work in 48? By sheer luck?

A workaround should be to stop() the old track first.
That isn't a work-around.  During a live webrtc session, the user is completely absent while they select another microphone through the gUM prompt.  It's a terrible user experience that I've complained about elsewhere, and literally the suggestion I was given on another mozilla defect was to do it this way.

It's extremely frustrating.  I wish we could either A) get Chrome's gUM prompt strategy (which, I'm sorry to say, is dramatically better) or B) get this bug fixed and stable.
For switching (stopping to use A, starting to use B) microphones it's a workaround. If you need two microphones to be active then it's not. But then you should update the title.
I do not agree.  I don't see how it's a workaround if someone is absent from a WebRTC session for seconds (if not longer if they somehow botch the gUM prompt).  Again: terrible user experience, and literally this is the "workaround" I was previously advised to do to deal with Firefox's bizarre gUM behavior.
Agree or not, this is the only workaround I can think of. gUM behavior is another issue that we can discuss at length somewhere else.
Agree or not, it isn't a workaround.  Would you expect skype or any other video conference software to show a user absent for seconds while they change their microphone?  It simply isn't an experience that equates to a "workaround."
I have been able to repro this. Video freezes because audio input stops and video uses audio to sync.

This happens when you do gUM with a constrained specific audio device, then do gUM with a second constrained specific audio device (different from the first), then stop the first gUM track.

Here's a fiddle that makes it easier to test: https://jsfiddle.net/pehrsons/cev96go6/

Here's another fiddle that uses the proposed workaround in comment 10: https://jsfiddle.net/pehrsons/cev96go6/10/
With it, audio input doesn't stop.


(thanks jib for your original enumrateDevices fiddle)
So this happens with full duplex enabled (which on linux goes for 49). With it disabled the gUM request is rejected with SourceUnavailableError.

To get the same behavior we could make MSGImpl::OpenAudioInput return a Pledge<nsresult> or so. Then we'd have to move the SourceMediaStream::OpenAudioInput() call from MediaEngine::Start() to MediaEngine::Allocate() and make said Allocate() block until the Pledge is resolved or rejected and return that.

Randell, do you think this is doable (moving the OpenAudioInput() from Start() to Allocate()) and upliftable? We don't have Allocate() for the audio engine as it looks. Might have to consult jib on this.

Also, how should we let this affect the full duplex rollout? We don't regress on functionality but we regress on notifying the user.
Flags: needinfo?(rjesup)
Assignee: achronop → rjesup
Status: NEW → ASSIGNED
Attachment #8798293 - Flags: review?(jib) → review+
What is multi-mic?

I don't want multiple microphones--I just want one, and I want the ability not to give up the old one until I'm positive I have the new one.  Is an error message the only quick option here?
(In reply to Jeremy Noring from comment #19)
> What is multi-mic?
> 
> I don't want multiple microphones--I just want one, and I want the ability
> not to give up the old one until I'm positive I have the new one.  Is an
> error message the only quick option here?

To not give up the old one until the new one is active requires two active at once.

The only really quick options are to emulate 48 behavior (like this), or to cause a new channel opening to redirect all old channels to the new content (i.e. still only one mic open, but instead of failing the new and leaving the old, let the new succeed and feed all the old streams from it).  That isn't super-simple to do (though much easier than real multi-mic support), and is unlikely to be upliftable to beta.  This fix should be upliftable
Flags: needinfo?(rjesup)
> To not give up the old one until the new one is active requires two active at once.

Correct.  

But surely there's a difference between having them active at the same time for a few milliseconds verses constantly?  I'm assuming the problem with having two active at the same time is it messes with echo cancellation?

To clarify, I don't care about "real multi-mic support", but it seems like the fix you're proposing isn't a fix so much as a message that makes it clear what I'm trying to do is broken?

Another option: gUM prompt doesn't show repeatedly within the same page. 

Surely there must be some other fix here than "sorry, it doesn't work" error message?
It's not an issue of a few ms... if you get a second stream, we have no idea if it's replacing the first or not (and if it is, when it will).  With full_duplex, additional input streams are complex compared to a pair (duplex), and it gets more complex when you have to rejuggle after one goes away (duplex pair.  Add another input (simplex+duplex).  Input side of duplex is closed - now you have to take the simplex input stream and somehow merge it with the output-only simplex stream (which likely means glitching and closing the simplex, and re-opening as duplex).  

We have some of the support for this in the code, but not all of it, and we'd need to feed the AEC output data to multiple streams.  And any change to implement multiple active mics will have to ride the trains; we likely can't uplift it (and certainly not to beta).
Pushed by rjesup@wgate.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/6a4107d258b0
return error if an audio channel is already open until multi-mic support is done r=jib
Comment on attachment 8798293 [details] [diff] [review]
return error if an audio channel is already open until multi-mic support is done

Approval Request Comment
[Feature/regressing bug #]: full_duplex

[User impact if declined]: Attempts to switch audio devices fail without returning an error

[Describe test coverage new/current, TreeHerder]: Manual - we have limited ability to test real media devices in automation.

[Risks and why]: Extremely low risk - the only risk would be if it returned errors when we didn't want it to.  Restores behavior of Fx48/non-full-duplex in this case.

[String/UUID change made/needed]: none
Attachment #8798293 - Flags: approval-mozilla-beta?
Attachment #8798293 - Flags: approval-mozilla-aurora?
https://hg.mozilla.org/mozilla-central/rev/6a4107d258b0
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla52
Hi Jeremy, could you please verify this issue is fixed as expected on a latest Nightly build? Thanks!
Flags: needinfo?(jeremy.noring)
Comment on attachment 8798293 [details] [diff] [review]
return error if an audio channel is already open until multi-mic support is done

Fixes a regression, Aurora51+, Beta50+
Attachment #8798293 - Flags: approval-mozilla-beta?
Attachment #8798293 - Flags: approval-mozilla-beta+
Attachment #8798293 - Flags: approval-mozilla-aurora?
Attachment #8798293 - Flags: approval-mozilla-aurora+
Flags: qe-verify+
I managed to reproduce this issue on Firefox 49.0b6, under Windows 10x64, using STR from Comment 2.
I've tested this fix on Firefox 50.0b11, Firefox 51.0a2 (2016-11-02) and on Firefox 52.0a1 (2016-11-02) and noticed that the video works correctly after changing the microphone multiple times, but the "ERROR: Failed to get access to local media. Error: NotReadableError, message:Failed to allocate audiosource" error is still thrown in the Browser Console.

Is this the expected behavior?

The tests were performed under Windows 10x64, Mac OS X 10.11.6 and under Ubuntu 16.04x64.
Flags: needinfo?(rjesup)
The hang is no longer reproducible on Firefox 50.1.0, Firefox 51.0b10, Firefox 52.0a2 (2017-01-03), Firefox 53.0a1 (2017-01-02).
I'm marking this issue Verified Fixed since Bug 1328267 was logged for the issue described in Comment 29. 
Note that the tests were performed under Mac OS X 10.12.1, Windows 10x64 and under Ubuntu 14.04x64 OSs.
Status: RESOLVED → VERIFIED
Flags: qe-verify+
Flags: needinfo?(rjesup)
>  but the "ERROR: Failed to get access to local media. Error: NotReadableError, message:Failed to allocate audiosource" error is still thrown in the Browser Console.
> 
> Is this the expected behavior?

Yes, this is expected behavior
You need to log in before you can comment on or make changes to this bug.