1828967 - Consistently reproducible crackling and popping of audio in specific sample video clip across OSes

Status: ASSIGNED

(Core :: Audio/Video: Playback, defect)

Description

[alex]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/112.0

Steps to reproduce:

1. Download the following test case, open index.html: https://drive.google.com/file/d/156aIuUO-9VWDVI_vpE25nWJiFzIIVBfC/view?usp=sharing
2. Playback audio, observe cracks and pops in audio

This test case can be observed in the wild at https://urbanists.video/w/5FgLJv6aRz3YHSkgv6WJa3

Actual results:

Crackling and popping audio on both MacOS and Windows (linux not tested).

Expected results:

Playback with non-crackly audio, like when played back in any other application (Chrome, Safari, Quicktime, Microsoft built-in video player, VLC, or mpv).

Comment 1

[BugBot [:suhaib / :marco]]

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[C.M.Chang[:chunmin]]

Yeah, the popping sound is obvious when using a headset. I can reproduce this on all my platforms, Windows, MacOS, and Ubuntu Linux, which suggests it's not a cubeb issue but a playback issue. Besides, disabling cubeb remote doesn't help.

I've got a profile, https://share.firefox.dev/43XS0oi, but I don't see anything abnormal. Crackling and popping can happen when gecko doesn't feed enough data to the audio buffer, but I am not sure if this is the case here.

Paul, is it possible that AudioSink's frames underrun?

Comment 3

[Paul Adenot (:padenot)]

This is what happens: https://searchfox.org/mozilla-central/source/dom/media/mediasink/AudioSink.cpp#488, see the markers "Pushing silence" in the zoomed profile.

This is caused by the fact that the file is badly muxed: it reports audio packet size of 1025 frames sometimes, and 1024 frames some other times (aac packets are always 1024 frames, this is a rounding error while muxing most probably), here's an excerpt from the log, with all my patches that rework TimeUnit to have time based on a number of frames / sample-rate:

::GetNextSample: Audio packet demuxed (track id: 2): [{150440,44100},{151465,44100}], duration: {1025,44100} (original time: [{invalid},{invalid}])
::GetNextSample: Audio packet demuxed (track id: 2): [{151465,44100},{152489,44100}], duration: {1024,44100} (original time: [{invalid},{invalid}])
::GetNextSample: Audio packet demuxed (track id: 2): [{152489,44100},{153514,44100}], duration: {1025,44100} (original time: [{invalid},{invalid}])
pushing silence: 2 (pts: 62494, parserd: 62492)
::GetNextSample: Audio packet demuxed (track id: 2): [{153514,44100},{154538,44100}], duration: {1024,44100} (original time: [{invalid},{invalid}])
::GetNextSample: Audio packet demuxed (track id: 2): [{154538,44100},{155563,44100}], duration: {1025,44100} (original time: [{invalid},{invalid}])
::GetNextSample: Audio packet demuxed (track id: 2): [{155563,44100},{156587,44100}], duration: {1024,44100} (original time: [{invalid},{invalid}])
::GetNextSample: Audio packet demuxed (track id: 2): [{156587,44100},{157612,44100}], duration: {1025,44100} (original time: [{invalid},{invalid}])
pushing silence: 2 (pts: 66592, parserd: 66590)
::GetNextSample: Audio packet demuxed (track id: 2): [{157612,44100},{158636,44100}], duration: {1024,44100} (original time: [{invalid},{invalid}])
::GetNextSample: Audio packet demuxed (track id: 2): [{158636,44100},{159661,44100}], duration: {1025,44100} (original time: [{invalid},{invalid}])
::GetNextSample: Audio packet demuxed (track id: 2): [{159661,44100},{160685,44100}], duration: {1024,44100} (original time: [{invalid},{invalid}])
::GetNextSample: Audio packet demuxed (track id: 2): [{160685,44100},{161710,44100}], duration: {1025,44100} (original time: [{invalid},{invalid}])
pushing silence: 2 (pts: 70690, parserd: 70688)
::GetNextSample: Audio packet demuxed (track id: 2): [{161710,44100},{162734,44100}], duration: {1024,44100} (original time: [{invalid},{invalid}])
::GetNextSample: Audio packet demuxed (track id: 2): [{162734,44100},{163759,44100}], duration: {1025,44100} (original time: [{invalid},{invalid}])
::GetNextSample: Audio packet demuxed (track id: 2): [{163759,44100},{164783,44100}], duration: {1024,44100} (original time: [{invalid},{invalid}])
::GetNextSample: Audio packet demuxed (track id: 2): [{164783,44100},{165808,44100}], duration: {1025,44100} (original time: [{invalid},{invalid}])
::GetNextSample: Audio packet demuxed (track id: 2): [{165808,44100},{166832,44100}], duration: {1024,44100} (original time: [{invalid},{invalid}])
pushing silence: 2 (pts: 74788, parserd: 74786)
::GetNextSample: Audio packet demuxed (track id: 2): [{166832,44100},{167857,44100}], duration: {1025,44100} (original time: [{invalid},{invalid}])
::GetNextSample: Audio packet demuxed (track id: 2): [{167857,44100},{168881,44100}], duration: {1024,44100} (original time: [{invalid},{invalid}])
::GetNextSample: Audio packet demuxed (track id: 2): [{168881,44100},{169906,44100}], duration: {1025,44100} (original time: [{invalid},{invalid}])
::GetNextSample: Audio packet demuxed (track id: 2): [{169906,44100},{170930,44100}], duration: {1024,44100} (original time: [{invalid},{invalid}])

what we can do here is check the actual number of frames in the packet and use that instead of the duration written on the packet. But the file is badly muxed.

Comment 4

[Paul Adenot (:padenot)]

(I have recorded this, I'll post the pernosco link when I have it).

Comment 5

[Paul Adenot (:padenot)]

https://pernos.co/debug/sMGEYmTOX-FcwQZxowww5g/index.html

Comment 6

[Paul Adenot (:padenot)]

https://pernos.co/debug/sMGEYmTOX-FcwQZxowww5g/index.html#f{m[BJdO,AkPC_,t[ATM,KpAa_,f{e[BJdO,5vs_,s{afwkp4AAA,bAY8,uDK3XCA,oDLP2bQ___/

This link shows the problem explained in comment 3.

Comment 7

[Paul Adenot (:padenot)]

Attachment: Bug 1828967 - Rebase pts of decoded packets when inconsistencies are detected during decoding. r?alwu

Comment 8

[BugBot [:suhaib / :marco]]

There is an r+ patch which didn't land and no activity in this bug for 2 weeks.
:padenot, could you have a look please?
If you still have some work to do, you can add an action "Plan Changes" in Phabricator.
For more information, please visit BugBot documentation.