getUserMedia({audio}) right after audioTrack.stop() fails with AbortError

RESOLVED FIXED in Firefox 68

Status

()

defect
P3
normal
RESOLVED FIXED
2 months ago
19 days ago

People

(Reporter: shaker.haibo, Assigned: jib)

Tracking

66 Branch
mozilla68
Points:
---

Firefox Tracking Flags

(firefox68 fixed)

Details

Attachments

(1 attachment)

Reporter

Description

2 months ago

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36

Steps to reproduce:

(1) using navigator.mediaDevices.getUserMedia({audio: true}) to get the audio stream. Add the stream to a WebRTC peer connection.
(2) After a while, we want to switch the microphone. Enumerate the devices and get another available microphone device id.
(3) Stop the previous old stream.
oldStream.getAudioTracks().forEach(function (track) {
track.enabled = false;
track.stop();
});
(4) Capture the new audio stream by getUserMedia() with specific device

Actual results:

The getUserMedia promise rejects with "Abort Error"

The Firefox internal logs show "GetUserMediaStreamRunnable::Run: starting failure callback following InitializeAsync()"

Expected results:

The getUserMedia promise resolves and returns a valid stream

B.T.W, it works well on older Firefox 60

Component: Untriaged → WebRTC: Audio/Video
Product: Firefox → Core

Thanks for the report!

I have a hunch where this is failing, so I'd like to confirm.

Does the abort error come with a message with more information (on the js object)? Does the Web Console (in Developer Tools) show anything related?

I don't understand why it would be failing like this though, given that the second microphone otherwise works normally. Do you have a simplified testcase page I can reproduce this on myself?

Flags: needinfo?(shaker.haibo)
Reporter

Comment 2

2 months ago

navigator.mediaDevices.getUserMedia({
audio: _audioConstraints
}).then(function (stream) {
}, function (error) {
// Catch the error here.
});

The error is:

MediaStreamError

constraint: ""

message: "Starting audio failed"

name: "AbortError"

stack: ""

About the demo page, I have no such page now. I will post one if available.

Flags: needinfo?(shaker.haibo)

This is probably the old "a single input device per document" limitation. We need to try to understand why stopping the track is not cleaning things up fast enough. Andreas, an idea about this?

Flags: needinfo?(apehrson)
Status: UNCONFIRMED → NEW
Ever confirmed: true
Priority: -- → P3

Ah, I managed to reproduce this eventually. There are missing bits in comment 0 needed to reproduce this.

Steps to reproduce:
1 Ensure you have two audio input devices on your system (this tested on Mac)
2 Go to https://jsfiddle.net/pehrsons/7hurneaz/
3 Click on "Ask for Devices"
4 Accept the permission prompt, and be sure to check the "Remember this decision" checkbox

Expected:
No errors

Actual:
AbortError shown on page, console shows it has the message "Starting audio failed", i.e., from [1], thus [2], like padenot speculated in.

[1] https://searchfox.org/mozilla-central/rev/75294521381b331f821aad3d6b60636844080ee2/dom/media/MediaManager.cpp#4144
[2] https://searchfox.org/mozilla-central/rev/75294521381b331f821aad3d6b60636844080ee2/dom/media/webrtc/MediaEngineWebRTCAudio.cpp#556-558

Flags: needinfo?(apehrson)

Waiting a bit after the audioTrack.stop() seems to work, like in https://jsfiddle.net/pehrsons/xeojsw0p/

[1] shows it depends on MSGImpl::InputDeviceID() which we call from the MediaManager thread. However this member is set on the graph thread at [2]. There's a race here that is not trivial to solve unless we move the MediaEngineWebRTCMicrophoneSource::Start() error to be an async MediaStreamTrack::OnEnded instead. Perhaps we need that as the fallback in MediaStreamGraph, and a way of tracking which input device is used for the document in MediaManager so we don't get into the situation that triggers that "ended" event later.

Or remove the limitation of one input device per doc, but how much work is that?

Either way we should fix the race here first.

I won't have time to work on this anytime soon. Paul or jib, perhaps either of you could take a look at what's acceptable here?

[1] https://searchfox.org/mozilla-central/rev/75294521381b331f821aad3d6b60636844080ee2/dom/media/webrtc/MediaEngineWebRTCAudio.cpp#554
[2] https://searchfox.org/mozilla-central/rev/75294521381b331f821aad3d6b60636844080ee2/dom/media/MediaStreamGraph.cpp#812

Flags: needinfo?(padenot)
Flags: needinfo?(jib)

If this is only an issue with audio—which seems to be the case—then I vote we hack it at the primary call site for Start() in InitializeAsync. The other place we call it, for mute/unmute shouldn't need changing.

Basically: pick a specific error code, and check for it here and if it matches, wait 200 ms and try one more time before giving up.

If we do that, we should use a timer—maybe make a nice media::Wait(200) function that returns a MozPromise? And not jank.

Flags: needinfo?(jib)

jib, you seem to have the solution, can you write the patch?

Flags: needinfo?(padenot) → needinfo?(jib)
Assignee: nobody → jib
Flags: needinfo?(jib)
Summary: getUserMedia always return "Abort Error" → getUserMedia({audio}) right after audioTrack.stop() fails with AbortError

Comment 9

Last month
Pushed by jbruaroey@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f798e09dc294
Retry gUM on mic process limit collision after 200 ms. r=padenot

Comment 10

Last month
bugherder
Status: NEW → RESOLVED
Closed: Last month
Resolution: --- → FIXED
Target Milestone: --- → mozilla68
QA Whiteboard: [qa-68b-p2]
You need to log in before you can comment on or make changes to this bug.