1880244 - With duplex streams, stream_init and stream_destroy are slow on Mac

Status: RESOLVED FIXED

(Core :: Audio/Video: cubeb, defect, P2)

Description

[Andreas Pehrson [:pehrsons]]

Because of the VoiceProcessingIO audio unit we started using in bug 1670633, creating (mainly) and destroying (somewhat) a duplex stream is very slow because creating (mainly) and destroying (somewhat) a VoiceProcessingIO audio unit is very slow. This caused bug 1878380 and while that bug fixes the video freeze during device reconfiguration in Meet, audio input and output are still frozen for the time the reconfiguration takes, while UX is not informing the user of this.

We can speed this up with a couple of tricks:

• Have gecko tell the cubeb backend it is planning to use input audio, the backend then, async, creates a VoiceProcessingIO audio unit and caches it.
  • Creating and immediately destroying a duplex stream without the VOICE pref will be quick and can be used by the backend as the signal.
• Optionally as part of getUserMedia wait for creating and immediately destroying a duplex stream with the VOICE pref such that this will wait until the cached VoiceProcessingIO audio unit is ready, to ensure that getUserMedia does not resolve seconds before audio flows, possibly leading to unexpected UX.
• In the backend only allow one duplex stream per context to use VoiceProcessingIO.
  • One document will only ever request one duplex stream, however
  • ...there is up to one context per process, i.e. all documents within a process share the context, so
  • ...in a scenario where two tabs are sitting in a room, only the first to request audio input would get VoiceProcessingIO. Fortunately, this is not a common real-world use case.
• Re-use the VoiceProcessingIO audio unit by returning it to the cache on stream_destroy.

Comment 1

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1880244 - Update cubeb-coreaudio-rs to 7b10217149. r?#cubeb-reviewers

Comment 2

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1880244 - Prime voice processing when asking user for gUM permissions. r?padenot!

Comment 3

[Andreas Pehrson [:pehrsons]]

I rewrote the priming patch after the latest cubeb backend updates. The priming is typically user-observable though, and I currently don't have a way to avoid priming when there is a persistent block on gUM audio for a given site. For now I'm deferring priming as future work. If I figure out a decent way to avoid priming when the user has persistently blocked gUM audio I'll file a followup.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/17662e43cc85
Update cubeb-coreaudio-rs to 7b10217149. r=cubeb-reviewers,padenot

Comment 5

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/17662e43cc85

Comment 6

[Jim Mathies [:jimm]]

Hey Andreas, do you still think this work will address Bug 1879093?

Comment 7

[Andreas Pehrson [:pehrsons]]

I think it will affect the numbers as a mitigation but it might not address the crash signatures fully. The root cause seems to be an OS library issue after all. Should fully adress the reported use case though.