1838756 - Delayed audio after muting and unmuting looped media

Status: VERIFIED FIXED

(Core :: Audio/Video: Playback, defect)

Description

[Karl Tomlinson (back Dec 8 :karlt)]

1. Load data:text/html,<audio%20controls%20loop%20src="https://philips.s3.eu-central-1.amazonaws.com/Rudolf/VfE_html5.mp4">
2. Play audio.
3. Listen to the acorn hitting the bird just before 0:21.
4. Wait (or seek to near the end then wait) for audio to loop back to beginning.
5. Mute for a few seconds.
6. Unmute before 0:21.
7. Listen for acorn again.

Expect: acorn sound at 0:21
Actual: acorn sound is delayed by something like the duration for which the audio was muted.

Seeking back to before 0:21 plays the acorn sound at the correct time.

This does not reproduce if steps 4 and 5 are interchanged.

Comment 1

[Karl Tomlinson (back Dec 8 :karlt)]

Regression from https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=4dfcb6e877c9ec1a9560db06777ee984891908d4&tochange=1d0656c6185de116bb9cbffbcbb92319b1c07e28

Toggling media.seamless-looping-video has no effect on the audio element.
Setting media.seamless-looping to false produces expected behavior.

Reproduces also with video: data:text/html,<video%20controls%20loop%20src="https://philips.s3.eu-central-1.amazonaws.com/Rudolf/VfE_html5.mp4">

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1262276

:alwu, since you are the author of the regressor, bug 1262276, could you take a look? Also, could you set the severity field?

For more information, please visit BugBot documentation.

Comment 3

[Karl Tomlinson (back Dec 8 :karlt)]

The old revisions in the regression range can be built with https://hg.mozilla.org/mozilla-central/rev/53badec13c4cf6a638e8bd5ee360d8f316aba917 applied (but not on try). This identifies the changeset triggering the regression as https://hg.mozilla.org/integration/autoland/rev/e43f99440e29

Comment 4

[Alastor Wu [:alwu]]

Sorry for late reply, I am going to take a look at this today.

Comment 5

[Alastor Wu [:alwu]]

Attachment: Bug 1838756 - do not adjust timestamp again for samples, which has been adjusted, when pushing them back to the audio queue.

When seamless looping is on, sample's timestamp would only be adjusted
once when they fist get pushed into the media queue.

When re-queuing sample back to the audio queue from the
mProcessedSPSCQueue, we don't need to apply timestamp adjustment
again. Otherwise, it would result in incorrect audio time so that the
AudioSink would never discard samples [1] during muting, causing audio
can't be rendered in the correct moment.

[1]
https://searchfox.org/mozilla-central/rev/d307d4d9f06dab6d16e963a4318e5e8ff4899141/dom/media/mediasink/AudioSinkWrapper.cpp#159

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/25833be56610
do not adjust timestamp again for samples, which has been adjusted, when pushing them back to the audio queue. r=padenot

Comment 7

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/25833be56610

Comment 8

[Ryan VanderMeulen [:RyanVM]]

Can we land a test for this?

Comment 9

[Alastor Wu [:alwu]]

We're not able to write a test for this, see this.

Comment 10

[Ryan VanderMeulen [:RyanVM]]

Did you want to nominate this for ESR115 uplift? It grafts cleanly.

Comment 11

[Alastor Wu [:alwu]]

Comment on attachment 9340586 [details]
Bug 1838756 - do not adjust timestamp again for samples, which has been adjusted, when pushing them back to the audio queue.

ESR Uplift Approval Request

• If this is not a sec:{high,crit} bug, please state case for ESR consideration: This fixes the problem where looping audio sometime can't play sound correctly
• User impact if declined: audio won't be able to play sound, only silence
• Fix Landed on Version: 115
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This didn't introduce a new functionality or a big architecture change, we only disable the incorrect time adjustment on a specific case where looping audio gets muted.

Comment 12

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9340586 [details]
Bug 1838756 - do not adjust timestamp again for samples, which has been adjusted, when pushing them back to the audio queue.

Approved for 115.1esr.

Comment 13

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-esr115/rev/d4c67edf18f6

Comment 14

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-esr115/rev/d4c67edf18f6

Comment 15

[Negritas Sergiu, Desktop QA]

Hello I have managed to reproduce the issue with Firefox 116.0a1(2023-06-10) on Ubuntu 22.04. I can confirm that the issue is fixed with firefox 117.0a1(2023-07-11), 116.0b3, 115.1.0esr(treeherder build from comment 14) on Ubuntu 22.04, Mac12 and Windows 10. I will update the corresponding flags.

I would like to mention that firefox 115.0.2esr is still affected by this issue.

Have a nice day!