Android data decoder takes over 2s to create audio decoder

RESOLVED FIXED in Firefox 67

Status

()

defect
P3
normal
Rank:
25
RESOLVED FIXED
4 months ago
4 months ago

People

(Reporter: jya, Assigned: jya)

Tracking

unspecified
mozilla67
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox67 fixed)

Details

()

Attachments

(2 attachments, 1 obsolete attachment)

Assignee

Description

4 months ago

In bug 1222851 we implemented support for audio gapless.

However, when you're playing test file https://jyavenard.github.io/htmltests/tests/mse_mp4/gapless/1222851.html you can see the audio stopping every 1s for close to 2s.

Adding some logs we see:

02-25 01:06:24.210 10381 22893 I RemoteDataDecoderModule:
02-25 01:06:24.236 10381 22843 I AndroidDecoderModule: CreateAudioFormat with mimeType=audio/mp4a-latm, mRate=44100, channels=2
02-25 01:06:26.376 10381 22878 I TrimmerDecoderModule: sample[5976779,5999999] has trimming info ([6000000,6000001]

It takes here 2.14s to create the AAC decoder. This makes for a very audible audio silence.

What's surprising however is this is intermittent. Sometimes it's super quick and the test page behaves properly.

Assignee

Comment 1

4 months ago

Not this issue will be seen with normal playback too like if the audio content change. A 2s gap will be heard

Assignee

Updated

4 months ago
Summary: Android data decoder takes over 1s to create audio decoder → Android data decoder takes over 2s to create audio decoder
Assignee

Comment 2

4 months ago

:jhlin

In this particular case I can check if only the AudioInfo has changed between video and avoid recreating the audio decoder, but still this is particularly limiting.

We don't want an audio gap of 2s if you're changing from stereo to 5.1 for example.

Assignee: nobody → jyavenard
Flags: needinfo?(jolin)
Assignee

Comment 3

4 months ago

This prevents re-creating a new audio decoder which on Android can take an awful long time.

Assignee

Comment 4

4 months ago

This prevents re-creating a new audio decoder which on Android can take an awful long time.

It looks like the remote decoding service/process gets restarted because of audio decoder recreation so it takes much longer to create a new decoder. Although we can just add a delay for the service unbinding action to avoid these kinds of use cases, I think we should try integrating Android remote decoder into the RDD mechanism introduced in bug 1471535.

Flags: needinfo?(jolin)

Updated

4 months ago
Rank: 25
Priority: -- → P3
Attachment #9046263 - Attachment description: Bug 1530234 - Ignore init segment for audio only buffer if audio content didn't change. r!bryce → Bug 1530234 - P1. Ignore init segment for audio only buffer if audio content didn't change. r=bryce
Assignee

Comment 6

4 months ago

The WMF audio decoder recalculated the timestamp of each audio sample according to the number of frames decoded so far.
This is incompatible with the trimming mechanism that rely on the timestamps of the audio to be matching what is found in the container.

All the other audio decoders do it that way already.

Depends on D20969

Comment 7

4 months ago
Pushed by jyavenard@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/0bd5d8684436
P1. Ignore init segment for audio only buffer if audio content didn't change. r=bryce
https://hg.mozilla.org/integration/autoland/rev/4e5ac450f2b6
P2. Don't recalculate audio sample start time. r=bryce

Comment 8

4 months ago
bugherder
Status: NEW → RESOLVED
Closed: 4 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla67

Updated

4 months ago
Depends on: 1531500
Attachment #9046261 - Attachment is obsolete: true
You need to log in before you can comment on or make changes to this bug.