Last Comment Bug 487504 - Audible changes to volume (and mute state) lag behind setting value
: Audible changes to volume (and mute state) lag behind setting value
Product: Core
Classification: Components
Component: Audio/Video (show other bugs)
: Trunk
: x86 All
-- normal with 2 votes (vote)
: ---
Assigned To: Matthew Gregan [:kinetik]
: Maire Reavy [:mreavy] Please needinfo me
: 602835 604409 726538 (view as bug list)
Depends on: 585851 586535 cubeb
  Show dependency treegraph
Reported: 2009-04-08 14:22 PDT by Justin Dolske [:Dolske]
Modified: 2013-09-23 01:35 PDT (History)
20 users (show)
roc: wanted1.9.1+
See Also:
Crash Signature:
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---

Patch: Win32 fix (9.02 KB, patch)
2010-10-21 21:30 PDT, Chris Pearce (:cpearce)
kinetik: review-
Details | Diff | Splinter Review
Chrome/Firefox audio fade in/out comparison (357.96 KB, application/octet-stream)
2013-07-07 12:59 PDT, tamfs887xb
no flags Details

Description User image Justin Dolske [:Dolske] 2009-04-08 14:22:59 PDT
Bug 475317 adds a volume-level slider, and I noticed that audio level I hear lags behind the value set by the controls. I'd guess it's a few hundred milliseconds. Fast enough that it's < 1 second, but slow enough that it feels wrong. Clicking mute/unmute exhibits the same behavior.

The mute case is interesting, because the icon changes immediately. That's driven by the volumechange event (not the actual click). So the video backend is immediately sending out the event, even when the level heard by the user hasn't yet changed.

[Thought this was already filed, but I was thinking of bug 469266.]
Comment 1 User image cajbir (:cajbir) 2009-04-08 16:03:33 PDT
I suspect it's the already buffered audio data that has been queued at the old volume.
Comment 2 User image Robert O'Callahan (:roc) (email my personal email if necessary) 2009-04-11 03:01:27 PDT
I'm reluctant to mark this blocking because I'm not sure how we can fix it.
Comment 3 User image cajbir (:cajbir) 2009-04-11 03:04:01 PDT
The way to fix it is to use platform API's to control the volume rather than adjusting the sample values directly.
Comment 4 User image Robert O'Callahan (:roc) (email my personal email if necessary) 2009-04-11 03:15:20 PDT
Hmm, for some reason I thought we had to adjust the sample values directly...
Comment 5 User image Robert O'Callahan (:roc) (email my personal email if necessary) 2009-04-11 03:15:55 PDT
Because we didn't have independent control over the volume of each stream, or something?
Comment 6 User image cajbir (:cajbir) 2009-04-11 03:17:38 PDT
We did it this way because libsydneyaudio didn't expose the api's fully for volume control across all platforms (It still doesn't iirc). And on Linux it controlled the master volume which was problematic as you mention in comment 5. I don't know the capabilities of all platforms as to whether per stream control is possible in all cases. So we took the route of modifying the sample values.
Comment 7 User image Robert O'Callahan (:roc) (email my personal email if necessary) 2009-04-11 03:21:42 PDT
OK, I'm still reluctant to mark this blocking if it depends on the existence of platform APIs that may not exist :-)
Comment 8 User image cajbir (:cajbir) 2009-04-11 03:23:43 PDT
I fully agree :-)
Comment 9 User image Robert O'Callahan (:roc) (email my personal email if necessary) 2009-04-11 03:26:04 PDT
I'll make it wanted. It would be nice if we used platform volume control APIs where they exist, and/or turned down the audio buffer if we think we can without skipping, but this doesn't really need to block IMHO.
Comment 10 User image Tony Chung [:tchung] 2009-04-13 15:48:50 PDT
i can repro this too on Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b4pre) Gecko/20090413 Shiretoko/3.5b4pre.   The mute/unmute lags about .5 seconds off
Comment 11 User image Matthew Gregan [:kinetik] 2009-05-21 23:01:22 PDT
This is trivial to fix on Mac.  The appropriate code already exists in sa_stream_set_volume_abs, which works per-stream.  All we'd need to do is call it from nsAudioStream::SetVolume and leave mVolume set at 1.0.

On Windows, some code exists in sa_stream_change_write_volume which could be used to implement sa_stream_set_volume_abs.  Unfortunately (despite being passed a per-stream handle) it affects the master "Wave Out" volume on pre-Vista, and the per-app "Wave Out" on Vista and later systems.  There's no way to control per-stream volume using the waveOut API.  We'd need to switch to using a more modern sound API to solve this.

sa_stream_set_volume_abs is implemented on Linux (in all of the backends), but it has similar limitations to the existing Windows code, except there's no obvious "modern API" to switch to to solve this, either (though I suppose that might eventually be PulseAudio once it's widespread enough).

I don't think we can afford to buffer less (unless we can make it adaptive), since we already have bug reports about audio glitches on slow or heavily loaded machines.

A more complicated way to fix this would be to immediately suspend audio playback when a volume change occurs, get the current playback position (ideally this would be sample accurate to be glitch-free, but maybe a "fairly" accurate value is fine if we can bear small glitches), purge the buffered data, remix it at the new volume, begin resubmitting the data, and resume the stream.  I don't think the existing sydneyaudio APIs are sufficient to permit this, though.
Comment 12 User image Matthew Gregan [:kinetik] 2009-05-21 23:32:14 PDT
Or we could push the software volume control down into sydneyaudio, which would then perform volume adjustment on the samples just before passing them to the OS.  This would be easy to do in the existing Mac and Windows code, but the ALSA code would need some restructuring.
Comment 13 User image nemo 2009-06-02 11:32:23 PDT
I noticed this in a test page I had open for Shiretoko on OSX.

I'd hit play on the <video> - the sound blasted out rather loudly.  I quickly paused, then toggled mute.  On hitting play again, the sound blasted out once more for a moment.
Comment 14 User image Luis Villa [Outside Counsel; for non-law use] 2010-04-12 09:02:59 PDT
The delay is at least a full second here on a Mac build from 04/11, FWIW.
Comment 15 User image Benjamin Smedberg [:bsmedberg] 2010-04-26 11:18:42 PDT
When viewing air mozilla on Windows the delay can be 3+ seconds hitting the Mute button. It's really annoying!
Comment 16 User image Matthew Gregan [:kinetik] 2010-05-30 20:52:11 PDT
Per comment 11, it's not easy to fix this properly on all platforms, so it's unlikely to be fixed for beta1.
Comment 17 User image Benjamin Smedberg [:bsmedberg] 2010-06-01 06:52:20 PDT
Assuming this affects WebM video as well as ogg, this is going to be a pretty major black eye for us: having audio continue for seconds after you close a youtube tab or hit the mute button is very noticeable. What can we accomplish in the beta timeframe (and for 3.6.x, if we decide to backport WebM)?
Comment 18 User image Robert O'Callahan (:roc) (email my personal email if necessary) 2010-06-01 11:42:24 PDT
Does audio actually continue after you close a tab? If so, that's a different bug.
Comment 19 User image Benjamin Smedberg [:bsmedberg] 2010-06-01 11:47:46 PDT
Yes, at least in air mozilla it has. I can file that separately.
Comment 20 User image Matthew Gregan [:kinetik] 2010-06-01 14:37:46 PDT
Audio continuing after a tab closes is a different bug--we can stop audio immediately by destroying the audio stream.
Comment 21 User image Damon Sicore (:damons) 2010-06-18 14:25:58 PDT
Bumping this to beta2 per beta re-triage with dbaron.  If this should indeed block beta1, please re-nom.  Is this still looking like it's not going to make beta1, regardless?
Comment 22 User image cajbir (:cajbir) 2010-06-19 03:08:05 PDT
Was the bug for the 'close tab' issue in comment 19 filed?
Comment 23 User image Mike Beltzner [:beltzner, not reading bugmail] 2010-07-19 19:21:25 PDT
Moving this to betaN to eventually be triaged into a specific beta milestone.
Comment 24 User image cajbir (:cajbir) 2010-10-08 05:08:38 PDT
*** Bug 602835 has been marked as a duplicate of this bug. ***
Comment 25 User image JK 2010-10-08 05:37:34 PDT
The bug exists in Windows 7 too.

Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b8pre) Gecko/20101007 Firefox/4.0b8pre
Comment 26 User image Benjamin Smedberg [:bsmedberg] 2010-10-14 09:58:40 PDT
*** Bug 604409 has been marked as a duplicate of this bug. ***
Comment 27 User image Chris Pearce (:cpearce) 2010-10-21 21:29:05 PDT
This has been fixed on Mac, still needs fixing on Linux and Windows.
Comment 28 User image Chris Pearce (:cpearce) 2010-10-21 21:30:25 PDT
Created attachment 485225 [details] [diff] [review]
Patch: Win32 fix

Uses WaveAPI to set volume on Windows. This is immediately setting the volume for me on native Windows 7 and in my WinXP VM.

Linux still needs to be fixed separately.
Comment 29 User image Matthew Gregan [:kinetik] 2010-10-21 22:00:50 PDT
Comment on attachment 485225 [details] [diff] [review]
Patch: Win32 fix

This isn't per-stream volume control, it's either global (pre-Vista) or per-application (Vista onwards).  See comment 11 for a few more details.
Comment 30 User image Chris Pearce (:cpearce) 2010-10-25 16:50:23 PDT
This is already fixed on Mac, but to fix on other platforms we'd need to rewrite the libsydney audio backend, so we shouldn't block 2.0 on this; it's a large and risky change to be making this late in the cycle.
Comment 31 User image Robert O'Callahan (:roc) (email my personal email if necessary) 2010-10-26 19:06:13 PDT
Sucks, but I guess you're right.
Comment 32 User image Ginn Chen 2012-01-17 01:14:58 PST
With Firefox 9, the lag of mute/unmute/volume change is about 1 second on Solaris.
With Firefox 10, the lag is about 9 seconds on Solaris.
There is a regression.

Similarly, pause is almost immediately with Firefox 9.
With Firefox 10, there is a 1-2 seconds lag with pause.
Comment 33 User image Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-01-17 15:57:15 PST
Thanks Ginn. Does this happen for you on Nightly (Firefox 12)? If so, would you be willing to track down a specific regression range using the Mozregression tool?
Comment 34 User image Ginn Chen 2012-01-17 21:18:39 PST
Thanks, the regression doesn't happen on Linux.
I found it is caused by Bug 669556, apply the same patch to sydney_audio_sunaudio.c will solve it.
Comment 35 User image Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-01-18 11:51:35 PST
Marking this as a regression as per comment 34.
Comment 36 User image Ginn Chen 2012-01-18 20:58:15 PST
Anthony, I think I should file a separate bug for the regression on Solaris.
The lag is not regressed on tier 1 platforms.
Comment 37 User image Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-01-18 22:12:50 PST
(In reply to Ginn Chen from comment #36)
> Anthony, I think I should file a separate bug for the regression on Solaris.
> The lag is not regressed on tier 1 platforms.

I think I agree. Please do so and link it back here. Thanks.
Comment 38 User image Matthew Gregan [:kinetik] 2012-02-13 11:15:06 PST
*** Bug 726538 has been marked as a duplicate of this bug. ***
Comment 39 User image bogas04 2012-03-09 11:46:20 PST
What's the status of this bug? Now that youtube is rolling out html5 videos for users not having flash installed , i think this regression should be fixed...
Comment 40 User image Matthew Gregan [:kinetik] 2012-03-09 12:31:31 PST
It's going to be fixed by bug 623444.
Comment 41 User image bogas04 2012-04-30 04:10:18 PDT
Seems like its fixed now with latest nightly. Any confirms ?
Comment 42 User image Matthew Gregan [:kinetik] 2012-04-30 04:16:11 PDT
Yes, this has been fixed on Windows by the landing of patches for bug 623444.  It's not yet fixed on Linux, so I'll leave the bug open until those patches land.
Comment 43 User image Matthew Gregan [:kinetik] 2012-06-04 18:45:29 PDT
Fixed by bug 623444.
Comment 44 User image tamfs887xb 2013-07-07 12:59:15 PDT
Created attachment 771870 [details]
Chrome/Firefox audio fade in/out comparison

This is still buggy for me. Please see this fiddle: It belongs to the following question on Stack Overflow:

While the audio fades very smooth in for example Chrome, it's still very choppy in Firefox, not as bad as in previous versions, but still anything but smooth. I'm testing this on Windows 7 x64 with an EMU-0404 PCIe, using Chrome 27.0.1453.116m and Firefox 22.0 as well as 25.0a1.

The attachment demonstrates exactly what it sounds for me. The first sample is using Chrome, the second one is using Firefox, where you should clearly hear the choppy transition.
Comment 45 User image Paul Adenot (:padenot) 2013-07-08 02:34:13 PDT
I can't repro on Linux/Mac, but I can repro on Windows, so this is backend specific.

I'll try with the new WASAPI backend, because I have the fear that the winmm buffer size is simply too bad to be able to have a smooth fade-in/fade-out effect using the volume attribute the way we do it (i.e. having a single possible value for the callback).
Comment 46 User image tamfs887xb 2013-08-01 05:41:17 PDT
So, did you have any luck using WASAPI?
Comment 47 User image Paul Adenot (:padenot) 2013-08-01 05:54:08 PDT
I haven't tried and I don't have a windows machine at the moment. I'll NEEDINFO myself as a reminder.

Note You need to log in before you can comment on or make changes to this bug.