Closed Bug 1530234 Opened 1 year ago Closed 1 year ago

Android data decoder takes over 2s to create audio decoder

Categories

(Core :: Audio/Video: Playback, defect, P3)

defect

Tracking

()

RESOLVED FIXED
mozilla67
Tracking Status
firefox67 --- fixed

People

(Reporter: jya, Assigned: jya)

References

()

Details

Attachments

(2 files, 1 obsolete file)

In bug 1222851 we implemented support for audio gapless.

However, when you're playing test file https://jyavenard.github.io/htmltests/tests/mse_mp4/gapless/1222851.html you can see the audio stopping every 1s for close to 2s.

Adding some logs we see:

02-25 01:06:24.210 10381 22893 I RemoteDataDecoderModule:
02-25 01:06:24.236 10381 22843 I AndroidDecoderModule: CreateAudioFormat with mimeType=audio/mp4a-latm, mRate=44100, channels=2
02-25 01:06:26.376 10381 22878 I TrimmerDecoderModule: sample[5976779,5999999] has trimming info ([6000000,6000001]

It takes here 2.14s to create the AAC decoder. This makes for a very audible audio silence.

What's surprising however is this is intermittent. Sometimes it's super quick and the test page behaves properly.

Not this issue will be seen with normal playback too like if the audio content change. A 2s gap will be heard

Summary: Android data decoder takes over 1s to create audio decoder → Android data decoder takes over 2s to create audio decoder

:jhlin

In this particular case I can check if only the AudioInfo has changed between video and avoid recreating the audio decoder, but still this is particularly limiting.

We don't want an audio gap of 2s if you're changing from stereo to 5.1 for example.

Assignee: nobody → jyavenard
Flags: needinfo?(jolin)

This prevents re-creating a new audio decoder which on Android can take an awful long time.

This prevents re-creating a new audio decoder which on Android can take an awful long time.

It looks like the remote decoding service/process gets restarted because of audio decoder recreation so it takes much longer to create a new decoder. Although we can just add a delay for the service unbinding action to avoid these kinds of use cases, I think we should try integrating Android remote decoder into the RDD mechanism introduced in bug 1471535.

Flags: needinfo?(jolin)
Rank: 25
Priority: -- → P3
Attachment #9046263 - Attachment description: Bug 1530234 - Ignore init segment for audio only buffer if audio content didn't change. r!bryce → Bug 1530234 - P1. Ignore init segment for audio only buffer if audio content didn't change. r=bryce

The WMF audio decoder recalculated the timestamp of each audio sample according to the number of frames decoded so far.
This is incompatible with the trimming mechanism that rely on the timestamps of the audio to be matching what is found in the container.

All the other audio decoders do it that way already.

Depends on D20969

Pushed by jyavenard@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/0bd5d8684436
P1. Ignore init segment for audio only buffer if audio content didn't change. r=bryce
https://hg.mozilla.org/integration/autoland/rev/4e5ac450f2b6
P2. Don't recalculate audio sample start time. r=bryce
Status: NEW → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla67
Depends on: 1531500
Attachment #9046261 - Attachment is obsolete: true
You need to log in before you can comment on or make changes to this bug.