1881981 - echo cancellation on Jitsi not working after diconnecting AirPods

Status: ASSIGNED

(Core :: WebRTC: Audio/Video, defect)

Description

[Karl Tomlinson (:karlt)]

Steps to reproduce:

1. Start a WebRTC call on meet.jit.si from Laptop A within builtin microphone and speakers.
2. Join the call from Laptop B, a Mac with system default audio output to AirPods.
   The audio menu in Jitsi shows AirPods and MacBook Pro Speakers but neither are checked for Speakers. Audio on the call is good.
3. Put the AirPods away in their box. The system default audio output switches to the MacBook Pro Speakers and this becomes checked in Jitsi's audio menu.
4. Speak into the microphone of Laptop A.

Expected: Hear little or nothing in speakers of Laptop A.

Actual: Hear clear echo in Laptop A, reproducible for at least 30 seconds.
This eventually resolves within several minutes.

Laptop B was using Firefox version 123.

This did not reproduce with media.setsinkid.enabled switched to false, in which case there is no option to display or change Speakers in Jitsi's audio menu.

I was not able to reproduce on Windows 11: on removal of a wired device, Jitsi does not place a check mark beside any speakers in its audio menu.

If step 3 is replaced with explicitly "Select MacBook Pro Speakers in Jitsi's audio menu", then a similar echo is heard but this resolves after a few seconds.

Comment 1

[Michael Froman [:mjf]]

jib: Any chance this is a slightly more involved version of Bug 1879204?

Comment 2

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

I doubt it. Bug 1879204 is about letting apps detect the OS switching the system default, unrelated to setSinkId and echo. If we assume Jitsi isn't turning off echo cancellation — which seems fair to assume, though can be hard ruled out I suppose with repro steps of production sites — the lack of echo cancellation here seems like a bug in Firefox closer to bug 1879180 or bug 1880997.

This seems hard to repro in a fiddle without a second machine or headset to keep the expectation of echo low.

Karl, since you have a setup to repro this, any idea if this is a regression?

Comment 3

[Karl Tomlinson (:karlt)]

I observed this on a call with Dan, but I don't have a Mac or AirPods myself to reproduce.

The behavior change with "media.setsinkid.enabled" implies that setSinkId() is involved here. There was a general AEC regression due to setSinkId() becoming available in version 116 for bug 1498512. The general case was largely addressed in version 123 for bug 1849108. The lack of AEC with setSinkId() over that period would obscure any other influencing changes.

I'll add some logging so we can check whether Jitsi is setting echoCancellation off through constraints.

Comment 4

[Karl Tomlinson (:karlt)]

Attachment: Bug 1881981 restore echoCancellation constraint and Mic source logging r?ng

This restores logging of a constraint removed in
https://hg.mozilla.org/mozilla-central/rev/4c8284f52c8be9ab5d382a79067567374613cb4d#l3.12

It was still possible to determine whether echoCancellation was on from
the presence or absence of "AudioProcessing statistics" lines with
MediaManager:4 logging if more than half a second of audio was processed, but
this "Audio config" line is more explicit and collects associated info
together.

Comment 5

[Karl Tomlinson (:karlt)]

Attachment: Bug 1881981 log MediaEnginePrefs after changes r?ng

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/43612b73c6e8
restore echoCancellation constraint and Mic source logging r=ng

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/88e37f77f460
log MediaEnginePrefs after changes r=ng

Comment 8

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/43612b73c6e8

Comment 9

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/88e37f77f460

Comment 10

[Karl Tomlinson (:karlt)]

Attachment: Bug 1881981 log AEC output device changes r?padenot

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3a4dd7f0ebef
log AEC output device changes r=padenot

Comment 12

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/3a4dd7f0ebef

Comment 13

[Karl Tomlinson (:karlt)]

Attachment: mac-speakers-trimmed.log

Dan and I reproduced this again with some logging.

• Jitsi doesn't always mark MacBook Pro Speakers with a check in its menu after step 3.
• When it does not mark MacBook Pro Speakers, the drift controller does not start and the problem does not occur.
• When it does mark MacBook Pro Speakers, AEC is configured on in the new "Mic source", and the MediaTrackGraph is using the output to the secondary device for AEC reverse stream, as expected.

When switching to MacBook Pro Speakers manually, the same symptoms do sometimes occur. When that happens, the drift controller is making some significant adjustments to the audio rate, so this may be related to bug 1880997 comment 11. The attached file contains logging from this case. The drift controller's buffering target increases to 720ms due to a few underruns.

• When buffering was increased from 90 to 180 ms, it quickly dropped to 117.8ms after 1 s and 41.6ms after 5s, even though the drift controller was slowing the consumption of data.
• Buffering increased to 87.57ms, but then underran.
• The 360ms of buffering then added was consumed within 8 seconds. The controller was slowing the consumption rate by 48000/48866.66 ~ 1.8% when the underrun occurred.
• When the desired 720ms of buffering was eventually reached (about 3.5 minutes after the start of the call, the controller increased the consumption rate by 48000/47736.03 ~ 0.55% to reduce the buffering.

Comment 14

[Andreas Pehrson [:pehrsons]]

Those underruns seem unexpected during normal operation. Were they triggered by some user action?

Comment 15

[Karl Tomlinson (:karlt)]

We don't have an explanation for the underruns. There were no further user changes after switching to the MacBook Pro Speakers, which triggered creation of the drift controller in the log. Underruns were still happening 1 minute after the switch, which was well after audio was successfully output to the speakers, and so would seem a long period for start-up issues. The system was an M1 Mac and wasn't doing any other heavy processing at the time.

Comment 16

[BugBot [:suhaib / :marco/ :calixte]]

The leave-open keyword is there and there is no activity for 6 months.
:karlt, maybe it's time to close this bug?
For more information, please visit BugBot documentation.