978649 - Request buffer size is not equal written buffer size

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: AudioChannel, defect)

Description

[Star Cheng [:scheng]]

This is a meta bug to trace the topic relate to buffer size.

device: Hamachi
SW: MC master

In AudioTrack.cpp::processAudioBuffer(), the reqSize is 4800 but the writtenSize is not more than 4412. So we have to pull data from "data call back function" more than once to satisfy 4800 bytes. For example, I try to modify the buffer length from (stm->queuebuf_len / stm->framesize) to 4800 in data_callback function. And get the 2 different result. In the initialization situation, it need more than 2 pull data from data call back function to satisfy audioflinger minFramecount. I might decrease the initialization CPU time. And it might decrease the times of the call pull data callback function.

case 1: (normal case)
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 4800 writtenSize 4412
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 388 writtenSize 388
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 4800 writtenSize 4024
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 776 writtenSize 776
E/AudioFlinger(  143): [star] AudioFlinger mixer threadloop mixBufferSize 4800
 
case 2: (modify the callback function buffer size as 4800)
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioFlinger(  140): [star] AudioFlinger mixer threadloop mixBufferSize 4800


------------------------------------------------------------------------------------
(1)There are 3 buffers are maintained in AudioStream/Decode thread, Cubeb_OpenSL/OpenSL ndk, and AudioTrack/AudioFlinger. 
<AudioStream/Decode thread> // size is aRate* mBytesPerFrame (44100 x 4 = 176400)
  void SetCapacity(uint32_t aCapacity) {
    mBuffer = new uint8_t[mCapacity];
  }
<Cubeb_OpenSL/OpenSL ndk> // size is  (stm->bytespersec * latency) / (1000 * NBUFS) -> (44100 x 4) x 100 / 1000 x 4 = 4410
  for (i = 0; i < NBUFS; i++) {
    stm->queuebuf[i] = malloc(stm->queuebuf_len);  // 4 x 4410
  }
<AudioTrack/AudioFlinger> // 2 x 1200 = 2400 (framecount) size :  2 x 1200 x mBytesPerFrame = 2 x (1200 x 4) = 2 x 4800 
AudioTrack::createTrack_l() {

        uint32_t minBufCount = afLatency / ((1000 * afFrameCount)/afSampleRate);
        if (minBufCount < 2) minBufCount = 2;

        if (sharedBuffer == 0) {
            if (frameCount == 0) {
                frameCount = minFrameCount;
            }
        }
}
------------------------------------------------------------------------------------
(2) data call back code flow
1.register audiostream data callback function to get decoded PCM data.
[AudioStream.cpp::Init()]
    if (cubeb_stream_init(cubebContext, &stream, "AudioStream", params,
                          latency, DataCallback_S, StateCallback_S, this) == CUBEB_OK)
[Cubeb_OpenSL.c opensl_stream_init()]
  stm->data_callback = data_callback;

2.new android::AudioTrack object
[cubeb_opensl.c]
res = (*stm->playerObj)->Realize(stm->playerObj, SL_BOOLEAN_FALSE);

[android_AudioPlayer.cpp @@opensl ndk]
SLresult android_audioPlayer_realize(CAudioPlayer *pAudioPlayer, SLboolean async) {

        pAudioPlayer->mAudioTrack = new android::AudioTrack(
                pAudioPlayer->mStreamType,                           // streamType
                sampleRate,                                          // sampleRate
                sles_to_android_sampleFormat(df_pcm->bitsPerSample), // format
                sles_to_android_channelMaskOut(df_pcm->numChannels, df_pcm->channelMask),
                                                                     //channel mask
                0,                                                   // frameCount (here min)
                0,                                                   // flags
                audioTrack_callBack_pullFromBuffQueue,               // callback
                (void *) pAudioPlayer,                               // user
                0      // FIXME find appropriate frame count         // notificationFrame
#ifndef USE_BACKPORT
                , pAudioPlayer->mSessionId
#endif
                );

}

3.register data call back function to get lib cube buffer from OpenSL ndk layer
[IBufferQueue.c @opensl ndk]
SLresult IBufferQueue_RegisterCallback(SLBufferQueueItf self,
    slBufferQueueCallback callback, void *pContext)
{
 this->mCallback = callback;
}

[cubeb_opensl.c]
res = (*stm->bufq)->RegisterCallback(stm->bufq, bufferqueue_callback, stm);

4.pull data from data call back function
[AudioTrack.cpp::processAudioBuffer() @libmedia.so]
mCbf(EVENT_MORE_DATA, mUserData, &audioBuffer);

[android_AudioPlayer.cpp @opensl ndk]
static void audioTrack_callBack_pullFromBuffQueue(int event, void* user, void *info) {
callback = ap->mBufferQueue.mCallback;
(*callback)(&ap->mBufferQueue.mItf, callbackPContext);
}

[cubeb_opensl.c]
bufferqueue_callback(SLBufferQueueItf caller, void * user_ptr)
{
    long written = stm->data_callback(stm, stm->user_ptr,
                                      buf, stm->queuebuf_len / stm->framesize);
}

[AudioStream.cpp]
AudioStream::DataCallback(void* aBuffer, long aFrames) {  // aFrames is not mapping 4800
  servicedFrames = GetUnprocessed(output, aFrames, insertTime);
}

AudioStream::GetUnprocessed(void* aBuffer, long aFrames, int64_t &aTimeMs)
{
  mBuffer.PopElements(available, &input[0], &input_size[0], &input[1], &input_size[1]);
}
------------------------------------------------------------------------------------
(3) write pcm data from audioflinger to HAL
[AudioFlinger.cpp] // mixBufferSize = 4800
bool AudioFlinger::MixerThread::threadLoop()
{
int bytesWritten = (int)mOutput->stream->write(mOutput->stream, mMixBuffer, mixBufferSize);
}

[AudioHardware.cpp] // bytes = count = 4800
ssize_t AudioHardware::AudioStreamOutMSM72xx::write(const void* buffer, size_t bytes)
{
    while (count) {
        ssize_t written = ::write(mFd, p, count);
    }

    if (mStartCount) {
        if (--mStartCount == 0) {
            ioctl(mFd, AUDIO_START, 0);
        }
    }
}

[AudioHardware.h] // if we don't overwrite this virtual function, the buffersize is 4800
virtual size_t      bufferSize() const { return 4800; }

Comment 1

[Star Cheng [:scheng]]

It should be :
(advantage)
1.To decrease the cpu times (invoke data callback function) of audio stream int.
2.To decrease the cpu times (invoke data callback function) while playing music in foreground/background.

(disadvantage)
1.To increase the moved mounts of memory in AudioStream -> Cubeb layer and Cubeb -> AT/AF layer.

Comment 2

[Star Cheng [:scheng]]

(1)launch music app.
(2)playing one mp3 file

Invoking once CallBack function to get 4800 bytes in case 2, instead twice in case 1.

case 1: (normal case)
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 4800 writtenSize 4412
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 388 writtenSize 388
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 4800 writtenSize 4024
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 776 writtenSize 776
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 4800 writtenSize 3636
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 1164 writtenSize 1164
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 4800 writtenSize 3248
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 1552 writtenSize 1552
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 4800 writtenSize 2860
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 1940 writtenSize 1940
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 4800 writtenSize 2472
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 2328 writtenSize 2328
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 4800 writtenSize 2084
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 2716 writtenSize 2716
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 4800 writtenSize 1696
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 3104 writtenSize 3104
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 4800 writtenSize 1308
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 3492 writtenSize 3492
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 4800 writtenSize 920
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 3880 writtenSize 3880
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 4800 writtenSize 532
E/AudioTrack(  628): [star] AT processAudioBuffer reqSize 4268 writtenSize 4268

case 2: (modified buffer size)
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800
E/AudioTrack(  479): [star] AT processAudioBuffer reqSize 4800 writtenSize 4800

Comment 3

[Star Cheng [:scheng]]

Should we modify the buffer_len in opensl_stream_init() of cubeb_opensl.c

opensl_stream_init(cubeb * ctx, cubeb_stream ** stream, char const * stream_name,
                  cubeb_stream_params stream_params, unsigned int latency,
                  cubeb_data_callback data_callback, cubeb_state_callback state_callback,
                  void * user_ptr)
{
 stm->queuebuf_len = (stm->bytespersec * latency) / (1000 * NBUFS);

}

And how to decide the queuebuf_len value instead of using fixed latency. (the latency value always equals 100)

Comment 4

[Star Cheng [:scheng]]

In opensl init(), I did 2 testing of changing buffer length of queue. The state of data moving is following log:


[1] stm->queuebuf_len = 3840;
I/Cubeb_OpenSL(  874): [star] stm->queuebuf_len 3840 
D/AudioTrack(  874): [star] processAudioBuffer frames 960
D/AudioTrack(  874): [star] processAudioBuffer reqSize 3840 writtenSize 3840
D/AudioTrack(  874): [star] processAudioBuffer frames 960
D/AudioTrack(  874): [star] processAudioBuffer reqSize 3840 writtenSize 3840

[2] stm->queuebuf_len = (stm->bytespersec * latency) / (1000 * NBUFS);
I/Cubeb_OpenSL(  901): [star] stm->queuebuf_len 4410 
D/AudioTrack(  865): [star] processAudioBuffer frames 960
D/AudioTrack(  865): [star] processAudioBuffer reqSize 3840 writtenSize 3840
D/AudioTrack(  865): [star] processAudioBuffer frames 960
D/AudioTrack(  865): [star] processAudioBuffer reqSize 3840 writtenSize 572
D/AudioTrack(  865): [star] processAudioBuffer frames 817
D/AudioTrack(  865): [star] processAudioBuffer reqSize 3268 writtenSize 3268


According to the log, [1] have fewer data moving than [2] while filling the first 2 frames (min frame count).

If we want to get that. The value of "stm->queuebuf_len" should be calculated from Audio back-end (AudioTrack/AudioFlinger). We can get the frame_count by invoking dlsym(libmedia, "_ZN7android11AudioSystem19getOutputFrameCountEPj19audio_stream_type_t"); 

So the value formula: queue buffer length = frame_count * stm->framesize;

Comment 5

[Star Cheng [:scheng]]

I/Cubeb_OpenSL(  874): [star] stm->queuebuf_len 3840 

D/AudioTrack(  874): [star] processAudioBuffer frames 960
D/AudioTrack(  874): [star] processAudioBuffer reqSize 3528 writtenSize 3528
D/AudioTrack(  874): [star] processAudioBuffer frames 78
D/AudioTrack(  874): [star] processAudioBuffer reqSize 312 writtenSize 312


On Nexus4 device (Android JB), It's not 1-1 mapping between reqSize & writtenSize, because the sample rate of source file (44.1k) is different from sample rate of output deivce(48k). So AudioTrack write 3840 but AudioFlinger read 3528 (3840 x 44.1 / 48). We can't see the 1-1 mapping of case 2 in comment 2 (On Hamachi (Android ICS)).

Comment 6

[Star Cheng [:scheng]]

Hi, Matthew

About comment 4 and comment 5, do you think weather it can decrease the times of data moving or not. And the the stm->queuebuf_len value should be *not* base on fixed latency value (always is 100). Could you give some suggestions about those?

Thanks

Comment 7

[Ivan Tsay (:ITsay)]

Hi Paul,

please take a look this one and see how it can be tested.

Comment 8

[Paul Yang [:pyang] (away)]

Need to have web page to demo, discussing with Scar

Comment 9

[Ivan Tsay (:ITsay)]

Move this feature/enhancement work to 2.1

Comment 10

[Matthew Gregan [:kinetik]]

(In reply to Star Cheng [:scheng] from comment #6)
> About comment 4 and comment 5, do you think weather it can decrease the
> times of data moving or not. And the the stm->queuebuf_len value should be
> *not* base on fixed latency value (always is 100). Could you give some
> suggestions about those?

It's only 100ms for normal playback, audio originating from the MediaStreamGraph (e.g. WebRTC/WebAudio) tries to use a lower value (based on what's returned from get_min_latency, not sure what that typically returns on Android-based systems - hopefully not also 100ms!).

(In reply to Star Cheng [:scheng] from comment #4)

> So the value formula: queue buffer length = frame_count * stm->framesize;

It still needs to be adjusted for the value passed in to stream_init via the latency parameter.

It does seem worthwhile to adjust the queue buffer length calculation based on whatever we can determine about the underlying buffer sizing, so that in any cases possible there's a 1:1 mapping.

Comment 11

[Star Cheng [:scheng]]

(In reply to Matthew Gregan [:kinetik] from comment #10)
> It does seem worthwhile to adjust the queue buffer length calculation based
> on whatever we can determine about the underlying buffer sizing, so that in
> any cases possible there's a 1:1 mapping.

In terms of Nexux4, The output device sample rate is defined as 48K in hardware\qcom\audio\hal\audio_hw.h (#define DEFAULT_OUTPUT_SAMPLING_RATE 48000). I try to change the sample rate to 44.1K, and It can 1:1 maping.


D/AudioTrack(  822): [star] processAudioBuffer reqSize 960 writtenSize 960
D/AudioTrack(  822): [star] write 3840 data need time - 21.94 ms
D/AudioTrack(  822): [star] processAudioBuffer reqSize 960 writtenSize 960
D/AudioTrack(  822): [star] processAudioBuffer reqSize 960 writtenSize 960
D/AudioTrack(  822): [star] processAudioBuffer reqSize 960 writtenSize 960
D/AudioTrack(  822): [star] processAudioBuffer reqSize 960 writtenSize 960
D/AudioTrack(  822): [star] write 3840 data need time - 22.04 ms
D/AudioTrack(  822): [star] processAudioBuffer reqSize 960 writtenSize 960
D/AudioTrack(  822): [star] processAudioBuffer reqSize 960 writtenSize 960
...

Comment 12

[BMO Automation]

Firefox OS is not being worked on