Allow capture from more than one mic at a time in getUserMedia
Categories
(Core :: WebRTC: Audio/Video, defect, P2)
Tracking
()
Tracking | Status | |
---|---|---|
firefox45 | --- | unaffected |
firefox46 | --- | unaffected |
firefox47 | --- | unaffected |
firefox48 | --- | unaffected |
firefox-esr45 | --- | unaffected |
relnote-firefox | --- | 101+ |
firefox-esr91 | --- | wontfix |
firefox99 | --- | wontfix |
firefox100 | --- | wontfix |
firefox101 | --- | fixed |
People
(Reporter: jesup, Assigned: chunmin)
References
(Depends on 1 open bug, Blocks 1 open bug)
Details
(Keywords: regression)
Attachments
(12 files, 17 obsolete files)
48 bytes,
text/x-phabricator-request
|
Details | Review | |
48 bytes,
text/x-phabricator-request
|
Details | Review | |
48 bytes,
text/x-phabricator-request
|
Details | Review | |
48 bytes,
text/x-phabricator-request
|
Details | Review | |
48 bytes,
text/x-phabricator-request
|
Details | Review | |
48 bytes,
text/x-phabricator-request
|
Details | Review | |
48 bytes,
text/x-phabricator-request
|
Details | Review | |
48 bytes,
text/x-phabricator-request
|
Details | Review | |
48 bytes,
text/x-phabricator-request
|
Details | Review | |
48 bytes,
text/x-phabricator-request
|
Details | Review | |
48 bytes,
text/x-phabricator-request
|
Details | Review | |
48 bytes,
text/x-phabricator-request
|
Details | Review |
Reporter | ||
Updated•9 years ago
|
Reporter | ||
Comment 1•9 years ago
|
||
Updated•8 years ago
|
Comment 2•8 years ago
|
||
Comment 3•8 years ago
|
||
Comment 5•8 years ago
|
||
Comment 7•7 years ago
|
||
Comment 8•7 years ago
|
||
Comment 9•7 years ago
|
||
Comment 11•5 years ago
|
||
Jitsi issue https://github.com/jitsi/jitsi-meet/issues/2835 depends on this one
Comment 12•4 years ago
|
||
I'm interested in finding out when this might get fixed.
Comment 13•4 years ago
|
||
This here is required for Jitsi Meet to work properly.
As such, as you had some whiteboard for [jitsi-meet], please assign that to this. (It is not, currently.)
Assignee | ||
Comment 14•4 years ago
|
||
I've involved in the discussion about how to implement it. I am taking the bug for now. Perhaps we can split this goal into several smaller tasks so it will be clearer what we are going to do.
Comment 15•4 years ago
|
||
(In reply to C.M.Chang[:chunmin] from comment #14)
I've involved in the discussion about how to implement it. I am taking the bug for now. Perhaps we can split this goal into several smaller tasks so it will be clearer what we are going to do.
Thank you!
Assignee | ||
Comment 16•4 years ago
|
||
Brief update from the discussion: Now our architecture (simplified version) is something like this (for microphone device). The goal is to implement this design.
Comment 18•3 years ago
|
||
Why many local audio streams can't be merged into a single stream ? Maybe this way could spare the bandwidth ? Even if you keep 2 channels (for stereo) into the single stream, would it allow the codec to achieve a few optimizations ?
Comment 19•3 years ago
|
||
(In reply to Jérôme from comment #18)
Why many local audio streams can't be merged into a single stream ?
They might not come from the same physical device, and so will drift with respect to each other. This is the main complexity in this bug, keep the low-latency, but reclock audio input devices.
Assignee | ||
Comment 20•3 years ago
|
||
Assignee | ||
Comment 21•3 years ago
|
||
To add a non-native input source in the MediaTrackGraph, we need a
input-only cubeb stream.
Depends on D116534
Assignee | ||
Comment 22•3 years ago
|
||
Add a AudioInputStream class that will generate audio input data from
one thread and it allows its user to read the data from another thread.
Updated•3 years ago
|
Assignee | ||
Comment 23•3 years ago
|
||
The following patches will add an abstract MediaTrack for the audio
input devices. The NativeInputTrack will be one implementation of it.
These MediaTracks holding the audio input data should be put together so
this patch moves the NativeInputTrack into individual .h and .cpp files,
which will also be the places implementing this new type of MediaTrack.
Assignee | ||
Comment 24•3 years ago
|
||
This patch addes a new DeviceInputTrack class, which is also the base
class for our NativeInputTrack class. The AudioInputTrack's
data-producer should always be DeviceInputTrack. We will implement
another subclass of DeviceInputTrack for non-native input device in the
following patches. As a result, the AudioInputTrack has no need to care
whether its requested device source is native or non-native.
Assignee | ||
Comment 25•3 years ago
|
||
This patch creates a new class, DeviceInputTrackManager, to manage all
the DeviceInputTrack. It helps to lower the complexity of
MediaTrackGraph and it is the only way to get access to the
DeviceInputTrack.
Assignee | ||
Comment 26•3 years ago
|
||
Depends on D118552
Assignee | ||
Comment 27•3 years ago
|
||
By the comment in AudioDriftCorrection
: "The construction can happen
in any thread", the AudioDriftCorrection
class is ought to be created
in any thread, but it's not true actually. The fact that ctor calls
Preferences::GetInt
makes it can only be called on main-thread, which
is the thread Preferences::GetInt
should be.
To make AudioDriftCorrection
work as what it's expected, the
preference getter should be moved to the AudioDriftCorrection
's caller
itself.
Updated•3 years ago
|
Assignee | ||
Comment 28•3 years ago
|
||
InitDataHolderIfNeeded
is used to initialize an AudioDataBuffers
but
there is no memory needed to be allocated and there is no need to wrap
the AudioDataBuffers
by a Maybe
.
Depends on D118552
Assignee | ||
Comment 29•3 years ago
|
||
AudioBufferInfo::Scope::All
is a shortcut version of
static_cast<AudioBufferInfo::Scope>(AudioBufferInfo::Scope::Input | AudioBufferInfo::Scope::Output)
.
Depends on D122513
Assignee | ||
Comment 30•3 years ago
|
||
D116534 is probably solved by D122338 in bug 1725137
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Assignee | ||
Comment 31•3 years ago
|
||
Add more debugging log showing device information
Depends on D118862
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Assignee | ||
Comment 32•3 years ago
|
||
The patches can be tested here: https://chunminchang.github.io/playground/webrtc/multi_microphones.html.
The test page can open multiple AudioInputTrack
with different devices, and the channel count can be reset anytime. The audio-processing including autoGainControl, echoCancellation and noiseSuppression can be set when the track are opened. Those audio-processing settings are used to set the AudioInputTrack
to passthrough
mode. It will lead to a different path when ProcessInput
is called.
Assignee | ||
Comment 33•3 years ago
|
||
This comment should be deleted in Bug 1725810.
Updated•3 years ago
|
Updated•3 years ago
|
Assignee | ||
Comment 34•3 years ago
|
||
The NativeInputTrack should only care about input data. Other
non-data related members should be moved out of the class. This makes
NativeInputTrack become a simple class focusing on input data storing.
Depends on D122513
Assignee | ||
Comment 35•3 years ago
|
||
The NativeInputTrack should be the only track that can open or close the
audio stream.
Depends on D130142
Assignee | ||
Comment 36•3 years ago
|
||
Add a simple test demonstrating the usage of NativeInputTrack
Depends on D118242
Assignee | ||
Comment 37•3 years ago
|
||
This patch implements a NonNativeInputTrack class that inherits the
DeviceInputTrack class. The audio data in this class instance is
produced by a non-graph audio stream via NotifyInputData(), and is
consumed by the graph thread via ProcessInput().
Depends on D122297
Updated•3 years ago
|
Updated•3 years ago
|
Comment 38•3 years ago
|
||
Comment on attachment 9244448 [details]
Bug 1238038 - Remove useless comment
Revision D127605 was moved to bug 1741959. Setting attachment 9244448 [details] to obsolete.
Comment 39•3 years ago
|
||
Comment on attachment 9241255 [details]
Bug 1238038 - Add debugging logs
Revision D125626 was moved to bug 1741959. Setting attachment 9241255 [details] to obsolete.
Comment 40•3 years ago
|
||
Comment on attachment 9227313 [details]
Bug 1238038 - Move NativeInputTrack to individual files
Revision D117965 was moved to bug 1741959. Setting attachment 9227313 [details] to obsolete.
Comment 41•3 years ago
|
||
Comment on attachment 9236045 [details]
Bug 1238038 - Move audio data processing to ProcessInput
Revision D122513 was moved to bug 1741959. Setting attachment 9236045 [details] to obsolete.
Comment 42•3 years ago
|
||
Comment on attachment 9248980 [details]
Bug 1238038 - Add test for NativeInputTrack
Revision D130232 was moved to bug 1741959. Setting attachment 9248980 [details] to obsolete.
Comment 43•3 years ago
|
||
Comment on attachment 9235703 [details]
Bug 1238038 - Relax AudioDriftCorrection's thread limitation
Revision D122297 was moved to bug 1746893. Setting attachment 9235703 [details] to obsolete.
Updated•3 years ago
|
Updated•3 years ago
|
Assignee | ||
Comment 44•3 years ago
|
||
Assignee | ||
Comment 45•3 years ago
|
||
Separate common interfaces that will be used for NonNativeInputTrack
class, from NativeInputTrack, into a base class: DeviceInputTrack, where
the NonNativeInputTrack will be implemented in the next patch.
Depends on D137781
Updated•3 years ago
|
Updated•3 years ago
|
Assignee | ||
Comment 46•3 years ago
|
||
Assignee | ||
Comment 47•3 years ago
|
||
Depends on D130233
Updated•3 years ago
|
Assignee | ||
Comment 48•3 years ago
|
||
Depends on D116535
Comment 49•3 years ago
|
||
Comment on attachment 9262249 [details]
WIP: Bug 1238038 - Fix comment for include guard
Revision D137817 was moved to bug 1754250. Setting attachment 9262249 [details] to obsolete.
Comment 50•3 years ago
|
||
Comment on attachment 9262163 [details]
WIP: Bug 1238038 - Narrower down the thread limits
Revision D137781 was moved to bug 1754254. Setting attachment 9262163 [details] to obsolete.
Assignee | ||
Comment 51•3 years ago
|
||
Depends on D137911
Assignee | ||
Comment 52•3 years ago
|
||
Depends on D116535
Assignee | ||
Comment 53•3 years ago
|
||
Depends on D138189
Updated•3 years ago
|
Assignee | ||
Comment 54•3 years ago
|
||
This patch creates a class to manage all DeviceInputTracks in one place,
which simplify the code in the MediaTrackGraph.
Depends on D138726
Assignee | ||
Comment 55•3 years ago
|
||
The mInputDeviceID can be removed now since we can get the input-device
id via the DeviceInputTrackManager on the graph-thread side now.
Depends on D139444
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Assignee | ||
Comment 56•3 years ago
|
||
Depends on D137910
Assignee | ||
Comment 57•3 years ago
|
||
Add a waitable event that will be fired when the state is changed, so
the user can wait for certain state of the cubeb stream.
Depends on D143100
Assignee | ||
Comment 58•3 years ago
|
||
Add a test to verify we can open multiple microphones
Depends on D139445
Comment 59•3 years ago
|
||
Updated•3 years ago
|
Comment 60•3 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/ed855828ee19
https://hg.mozilla.org/mozilla-central/rev/cee7aa3686a2
https://hg.mozilla.org/mozilla-central/rev/3e81a5bebba2
https://hg.mozilla.org/mozilla-central/rev/f2c23215c842
https://hg.mozilla.org/mozilla-central/rev/28f666109dfa
https://hg.mozilla.org/mozilla-central/rev/eff158481883
https://hg.mozilla.org/mozilla-central/rev/b66220158398
https://hg.mozilla.org/mozilla-central/rev/80575371d79f
https://hg.mozilla.org/mozilla-central/rev/8f7feefeda0c
https://hg.mozilla.org/mozilla-central/rev/a1e8aebd2e88
https://hg.mozilla.org/mozilla-central/rev/eb898ad02abd
https://hg.mozilla.org/mozilla-central/rev/2b2de9f71e7f
Updated•3 years ago
|
Updated•3 years ago
|
Assignee | ||
Comment 63•3 years ago
|
||
This should be for bug 1731141. The patches here remove the code that causes the crash.
Comment 64•2 years ago
|
||
Release Note Request (optional, but appreciated)
[Why is this notable]:
Audio input support improvements for web conferencing.
[Affects Firefox for Android]:
Yes
[Suggested wording]:
Firefox now allows users to use as many microphones as you want at the same time during video conferencing! The most important benefit is that you can easily switch your microphones at any time, if your conferencing service provider enables this flexibility.
[Links (documentation, blog post, etc)]:
Assignee | ||
Comment 65•2 years ago
|
||
jitsi enables this in PR 1988 🎉
Description
•