1443511 - A small delay occurs when the volume is changed on Soundcloud

Status: VERIFIED FIXED

(Core :: Audio/Video: MediaStreamGraph, defect, P3)

Description

[Emil Pasca [:emilpasca]]

Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0
OS: Linux 4.15.2-2-ARCH

[Affected versions]:
- Firefox 58.0.2
- Nightly 60.0a1

[Affected Platforms]:
- All Linux

[Steps to reproduce]:
1. Navigate to "soundcloud.com" and play any song.
2. Change the volume up/down using the player volume handle.

[Expected result]:
- The volume is adjusted instantaneously according to the volume handle position.

[Actual result]:
- A small delay occurs until the volume is adjusted to the volume handle position.

[Notes]:
- The issue is reproducible with Jack/USB audio output device.
- The issue is reproducible for mute/unmute tab icon option.
- The issue is reproducible for mute/unmute context menu option.

Comment 1

[Ralph Giles (:rillian)]

Matthew, any idea what would cause this?

Comment 2

[Matthew Gregan [:kinetik]]

I can reproduce this on macOS and Windows as well.  I don't think it's an audio latency issue, it seems far too long for that.  I tried older Firefox versions going back to 45 where the delay is much lower (and can be attributed to audio latency, which might be 100-250ms); there's a step change between 45 and 46 where it starts taking well over a second to apply a mute/volume change.

Comment 3

[Matthew Gregan [:kinetik]]

Seems to be within the range https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=e790bba372f14241addda469a4bdb7ab00786ab3&tochange=ad1f85f172b7302bef0fa9780df8e2b962780ac6 according to mozregression.

Bug 948267 touches the audio code in that range, but it doesn't look like it would affect the behaviour of volume adjustment (and if it did, why not all sites/media?), so I don't think it's that.  Nothing else seems like an obvious candidate.

Comment 4

[Matthew Gregan [:kinetik]]

SoundCloud is using AudioContext.createMediaElementSource to capture the media element's audio, which is why we're seeing the volume delay regression here but not on other sites.

From some quick logging, it's taking 500ms(!) between when HTMLMediaElement::SetVolume is called and that value arriving at DecodedStream::SetVolume.  The volume is reflected via the state watching machinery, but it shouldn't take that long unless there's something slow or many events ahead of it in the event queue.

The volume adjustment is then applied (asynchronously) slightly later when DecodedStream::SendData runs.  There are still several chunks of audio already queued with the old volume still applied, adding additional delay.

Comment 5

[Matthew Gregan [:kinetik]]

Attachment: simplified test case

Simplified test case that demonstrates the delay is induced by using AudioContext.createMediaElementSource.

Comment 6

[[e-mail]]

I was just about to create a ticket for this. I work on the playback library at SoundCloud :)

Correct we started using web audio recently, and the issue only happens when the media element is routed into that.

I also created a small test case: https://jsbin.com/dejokop/edit?html,js,output

Would be great for this to be fixed!

Comment 7

[Paul Adenot (:padenot)]

It's probably a dupe of bug 1447273, that we're about to land.

Comment 8

[Paul Adenot (:padenot)]

It's not. pehrsons, bryce, it's probably related in some capacity.

Comment 9

[Andreas Pehrson [:pehrsons]]

From playing with this it seems to me like the delay is the amount of data that has been pushed into the stream already, but I haven't verified this yet. It's one of the reasons kinetik mentions in comment 4.

A pull based DecodedStream would in that case take care of it. Or a way to set the volume for already-pushed AudioSegments in a MediaStream.

I can take this tentatively, but if anyone else has time feel free to steal. I can always advise on how to fix it once the cause is verified.

Comment 11

[Bogdan Maris, Desktop Test Engineering]

This is still affecting Fx 60.3.0esr and newer 64.0a1. Marking flags accordingly. Is there a fix planned for this one?

Comment 12

[Liz Henry (:lizzard) (relman/hg->git project)]

Too late for a fix in 63 but we could still take a patch for 65/64.

Comment 13

[Paul Adenot (:padenot)]

(In reply to Andreas Pehrson [:pehrsons] from comment #9)
> From playing with this it seems to me like the delay is the amount of data
> that has been pushed into the stream already, but I haven't verified this
> yet. It's one of the reasons kinetik mentions in comment 4.
> 
> A pull based DecodedStream would in that case take care of it. Or a way to
> set the volume for already-pushed AudioSegments in a MediaStream.
> 
> I can take this tentatively, but if anyone else has time feel free to steal.
> I can always advise on how to fix it once the cause is verified.

pehrsons, do you think this will be solved by the things you're doing around streams and tracks?

Comment 14

[Andreas Pehrson [:pehrsons]]

There's a small chance of course, but until we know the root cause I'll doubt it.

Comment 15

[Timea Cernea [:tbabos][inactive]]

Reproduced on latest Nightly 66.0a1 (2018-12-18) on Mac OS 10.14

Comment 16

[Liz Henry (:lizzard) (relman/hg->git project)]

Marking fix-optional for 65 and 66 so that these already triaged issues don't show up repeatedly in weekly regression triage. Happy to take a patch in nightly.

Comment 17

[u554753]

Reproducible on latest Nightly for Win 7 x64 using 32-bit build.
Version 67.0a1
Build ID 20190220040540
User Agent Mozilla/5.0 (Windows NT 6.1; WOW64; rv:67.0) Gecko/20100101 Firefox/67.0

Comment 18

[Marcela (Please NI request to Brindusa Tot)]

Reproducible in last Nightly for Windows10 68.0a1 (2019-05-13) 64 bit
Build ID 20190513215004
User Agent Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0

Comment 19

[Patricia Lawless]

Bulk change of P3 carryover bugs to wontfix for 68.

Comment 20

[Virginia (please send need info request to Brindusa Tot)]

Reproducible in last Nightly for Windows10 69.0a1 (2019-06-10) (64-bit)
Build ID 20190610214846
User Agent Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:69.0) Gecko/20100101 Firefox/69.0

Comment 21

[Alin Ilea, Desktop Test Engineering [pto until 01.09]]

Reproducible on Mac OS 10.12 in latest Nightly 69.0a1 (2019-06-10)
Build ID 20190610214846
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:69.0) Gecko/20100101 Firefox/69.0

Comment 22

[Liz Henry (:lizzard) (relman/hg->git project)]

Happy to take a patch for 70 but since this is triaged and set to P3 priority I'm setting it as fix-optional.
That will remove the bug from weekly regression triage.

Comment 23

[Andrei Purice]

Reproduced on latest beta build 70.0b5 and Nightly build 71.0a1 (2019-09-12) on Windows 7 x64.

Comment 24

[Liz Henry (:lizzard) (relman/hg->git project)]

Too late for 70 but we could still take a patch for 71/72.

Comment 25

[Jon Bauman [:jbauman:]]

Attachment: Bug 1443511 - Apply HTMLMediaElement volume to currently playing audio segments. r=pehrsons,padenot

Rather than applying the volume change to the DecodedStream, which won't be
played until the buffer catches up, instead apply the volume to the segments
as they are consumed by the MediaTrackGraph.

This is a work-in-progress, but I wanted to get feedback on the overall approach.

Comment 26

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/cd6eeee3e04a
Apply HTMLMediaElement volume to currently playing audio segments. r=pehrsons

Comment 27

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/cd6eeee3e04a

Comment 28

[BugBot [:suhaib / :marco/ :calixte]]

Since the status are different for nightly and release, what's the status for beta?
For more information, please visit auto_nag documentation.

Comment 29

[Alin Ilea, Desktop Test Engineering [pto until 01.09]]

Verified - Fixed in latest Nightly build 73.0a1 (Build id: 20191215214948) using Windows 10, Ubuntu 18.04 and Mac OS 10.14