1914046 - Be stricter on what combinations of audio processing settings are allowed

Status: NEW

(Core :: WebRTC: Audio/Video, task)

Description

[Andreas Pehrson [:pehrsons]]

Historically we have used libwebrtc's audio processing in software, and been able to turn on/off every setting (echo cancellation (AEC), automatic gain control (AGC), and noise suppression (NS)) at will.

We have been switching to using VoiceProcessingIO (VPIO) audio units to provide audio processing at the platform level on macOS. This lets the echo canceller do system-wide echo cancellation, and can better use Apple hardware for processing where needed. On the downside, some combinations of processing settings are not useful for some input devices. On VPIO there are only two knobs: bypass and agc. We have mapped the processing contraints AEC and NS to bypass-off and AGC to agc-on. For some more recent Macbook Pro builtin microphones, bypass-on seems to render a low volume on the audio from the device, making it hard to use, even with the libwebrtc AGC applied downstream, see bug 1896938. Testing has also shown that with bypass-on, agc-on doesn't seem to have the expected effect, rendering that combination useless.

For Android the situation is similar. There is a java API to create and turn individual processing effects on/off at will, but my testing for bug 1839885 has shown that devices behave differently when using that API, with some not allowing the effects to be created although they are reported as present, unless the input stream has a preset that enables the effect already set. Just using presets for the input stream is more straight forward, but doesn't map to all combinations of processing settings.

Also worth considering is that we share devices for multiple device requests, and having the shared requests use the same settings means we can rely entirely on platform processing and don't have to either 1) disable platform processing and do sw processing downstream, or 2) do processing in two stages where the intersection of the sets of processing settings is applied in the platform and the rest downstream. Users having different processing behavior on different sites could cause confusion, it seems better to keep it simple (fewer combinations allowed) and opinionated (we don't have to strictly follow constraints).

Comment 1

[Andreas Pehrson [:pehrsons]]

This may still help with some corner cases leading to bug 1896938.