1938570 - Tune buffering levels in AAudio backend

Status: RESOLVED FIXED

(Core :: Audio/Video: cubeb, defect, P2)

Description

[Andreas Pehrson [:pehrsons]]

See analysis in bug 1934566 comment 7:

The profile shows this is playback, so latency and callback size is 100ms (4800 frames).
If I read AAudio and libaudioclient code right, then:

Before bug 1905636:

• burst size (hw callback) unclear but typically 5ms (240 frames)
• buffer size set to 3x the burst size, so 15ms (720 frames)
• buffer capacity is cubeb's requested latency, so 100ms (4800 frames)
• callback size (user callback) is cubeb's requested latency so 100ms (4800 frames) (its own buffer)

Note the comment on setting the callback size:

 * If you do call this function then the requested size should be less than
 * half the buffer capacity, to allow double buffering.

After bug 1905636:

• burst size (hw callback) unclear but typically 5ms (240 frames)
• buffer size not set, defaults to buffer capacity, so 300ms (14400 frames)
• buffer capacity is 3x cubeb's requested latency, so 300ms (14400 frames)
• callback size (user callback) is cubeb's requested latency so 100ms (4800 frames) (its own buffer)

From trying the STR on my own device I hear a gap of silence in the sound early on, then a premature end. It seems to me as if we receive more user callbacks right away in order to fill the larger buffer, but underrun in our buffer of decoded data, so insert silence into the stream (supported by the profile). The premature end probably means we don't drain the stream properly.

This seems like a bit much buffering, especially if AAudio/Android tries to fill the entire buffer right away. My finding suggested that the callback buffer allocates its own buffer, despite the comment about having a sufficiently large buffer capacity when setting the callback buffer size. Maybe we should revert the behavior from bug 1905636 for output, but we might have to do something for input still, though bug 1905636 may have helped there for low-latency cases.

Comment 1

[Andreas Pehrson [:pehrsons]]

I've been sketching how I think the buffers work and, for realtime, burst size (device callback) 256 (5.3ms):

• Pre-1905636:
  • The user callback could hold 5.3ms.
  • The AAudio buffer we thought could hold 16ms (3 bursts), but the capacity was set to the user callback size, i.e. 5.3ms. If that was accepted the size couldn't be bigger.
  • Total buffer size 10.7ms.
  • For input it sends a user callback on the first burst. Then has 5.3ms (1 burst) to buffer while waiting for the user callback to return.
  • For output it issues 1 user callback, moves that to the AAudio buffer and issues another user callback. When there is room to move a pending user callback buffer into the AAudio buffer (meaning it's empty), it can issue another user callback.
  • Or, if the tiny buffer capacity wasn't accepted (I haven't double checked) we'd have 2 more bursts of AAudio buffer size.
• Post-1905636, pre-1930825:
  • The user callback could hold 5.3ms.
  • The AAudio buffer could hold 16ms (3 user callbacks).
  • Total buffer size 21.3ms.
  • We noticed improvement with this setup, suggesting that pre-1905636 we really had just 1 burst of buffer size.
• Post-1930825:
  • The user callback could hold 21.3ms.
  • The AAudio buffer could hold 64ms (3 user callbacks).
  • Total buffer size 85.3ms.
• Post-1930825, if we revert 1905636 and increase the buffer capacity to at least 3 bursts:
  • The user callback could hold 5.3ms.
  • The AAudio buffer could hold 16ms.
  • Total buffer size 21.3ms.
  • This is an improvement over pre-1905636, but identical to pre-1930825. Perhaps this could work when bug 1931080 is sorted. It would fix the playback case in bug 1934566.

To summarize, I plan to revert bug 1905636, then add another patch to make sure the AAudio buffer capacity is large enough to hold the 3 bursts we try to set the AAudio buffer size to.

Comment 2

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1938570 - Update libcubeb to 6043d35a. r?#cubeb-reviewers

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/98427b42feb4
Update libcubeb to 6043d35a. r=cubeb-reviewers,padenot

Comment 4

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/98427b42feb4

Comment 5

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1938570 - Update libcubeb to 6043d35a. r?#cubeb-reviewers

Original Revision: https://phabricator.services.mozilla.com/D233659

Comment 6

[Phabricator Automation]

beta Uplift Approval Request

• User impact if declined: Audio glitches when starting playback of (decoded) audio on android
• Code covered by automated testing: no
• Fix verified in Nightly: no
• Needs manual QE test: yes
• Steps to reproduce for manual QE testing: See bug 1934566 comment 0. This fix is for the gap that often occurs early in the audio.
• Risk associated with taking this patch: Low
• Explanation of risk level: Trivial; Slight adjustment of how audio is buffered on Android
• String changes made/needed: None
• Is Android affected?: yes

Comment 7

[Mira Lobontiu (Android QA)]

Verified as fixed on the latest Fenix Nightly 136.0a1 from 1/12 with a Samsung Galaxy S24 (Andorid 14).
Verified on this website: https://www.merriam-webster.com/dictionary/matriculate, and the stutter is noticeable on Beta 135.0b3, but not on the above mention Nightly build.

Comment 8

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-beta/rev/204d8fc3da71