484814 - <audio> is choppy on WindowsXP (32bit) / affected by screen drawing.

Status: RESOLVED FIXED

(Core :: Audio/Video, defect)

Description

[John-Paul Harold]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b4pre) Gecko/20090322 Shiretoko/3.5b4pre
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b4pre) Gecko/20090322 Shiretoko/3.5b4pre

Hello

while playing back media embedded within an <audio> tag, the playback performance on the WinXP machines I've tested on is choppy, and unusable as is. My own experiments have shown that the performance is affected more when screen activity is busiest, scrolling down a page for example, and that some performance can be increased if the browser is minimised, or a new, blank tab is created, and selected. Even then, performance is frequently interrupted by disk activity.

This ticket is a continuation from this, https://bugzilla.mozilla.org/show_bug.cgi?id=462667#c13

For developers interested in this, I can supply a URL to test against. It's not something I can make public, for various reasons.



Reproducible: Always

Steps to Reproduce:
1. Load up Firefox 3.{1|5}beta in a WindowsXP(32bit) environment
2. Navigate to a page using the <audio> element and commence audio playback.
3. Try scrolling down the page, moving the browser window. Notice choppiness.
Actual Results:  
audio playback is directly affected by screen draw actions. Even without any heavy screen drawing activity, playback is prone to odd jumps, interruptions. Other actions will also cause choppiness, such as disk i/o

Expected Results:  
I expect the audio to playback, without interruptions.

like I said, I can provide a link to an <audio> using page, but it's not a link I can make public at this point.

Comment 1

[Ria Klaassen (not reading all bugmail)]

https://bugzilla.mozilla.org/attachment.cgi?id=360209
I hear at least 3 hesitations. But also when I don't scroll and do nothing with my mouse. Audio/video rarely sound/look right on a simple laptop.

Comment 2

[Chris Pearce [:cpearce (Not reading bugmail)]]

(In reply to comment #1)
> https://bugzilla.mozilla.org/attachment.cgi?id=360209

I hear scratchiness near the start on Vista SP1. I'm running a build in the background, but CPU usage doesn't go over 64%, so I don't think lack of CPU time is causing this.

It plays fine on my Mac Mini.

Comment 3

[Matthew Gregan [:kinetik]]

Are you seeing this with any audio file, or just Ogg Vorbis?  <audio> also
supports Wave files, and uses a slightly different buffering strategy for them.
 Is Wave playback using <audio> also choppy on your machine?

On Ria's testcase (which uses Vorbis) I hear a couple of dropouts at the start
of playback on my idle MBP.  Sounds like buffer underruns.

Comment 4

[Chris Pearce [:cpearce (Not reading bugmail)]]

Attachment: Testcase - wav playback

(In reply to comment #3)
>  Is Wave playback using <audio> also choppy on your machine?

No, Wave playback is smooth and clear for me (see testcase). This is only a problem with Ogg-Vorbis audio.

Comment 5

[John-Paul Harold]

(In reply to comment #3)
> Are you seeing this with any audio file, or just Ogg Vorbis?  <audio> also
> supports Wave files, and uses a slightly different buffering strategy for them.
>  Is Wave playback using <audio> also choppy on your machine?

A simple test with the following code

<audio src="test.wav" autoplay="true">no audio</audio>

brought about interruptions, just like with vorbis. I started resizing the browser window with the grippy in the bottom right. That brings about a lot of stuttering. 

Please note, I get perfectly fine audio playback on both OSX and Linux, just not Windows. XP in my case, I don't have Vista, 7, or 2K to hand.

Comment 6

[John-Paul Harold]

hi, is this bug recognised as valid, or does it require more testing? If there's more I can do, please say. I'm keen to get <audio> working flawlessly for me on Firefox (WinXP 32bit)

Comment 7

[Matthew Gregan [:kinetik]]

It's marked as NEW, so yes it is recognized as valid.

Comment 8

[Matthew Gregan [:kinetik]]

I've made some improvements to the Windows libsydneyaudio backend to avoid underrunning the audio buffer.  Taking bug, will post a patch once I've done some more testing.

Comment 10

[John-Paul Harold]

I should point out, I have exactly the same issue with <video> on Firefox 3.5. If this is unrelated to this specific bug and requires a new ticket creating, please say and I'll do so.

Comment 11

[Matthew Gregan [:kinetik]]

It's probably the same bug.  When I get a bit of time, I'll push my changes to the try server so there's a build available for you to test.

Comment 12

[Matthew Gregan [:kinetik]]

I've push a test patch to the try server.  The binaries should appear here in a couple of hours:

https://build.mozilla.org/tryserver-builds/[e-mail]-try-f154ed0c631d/

Please test on Windows machines where you're hearing audio glitches and let me know how it goes.

Comment 13

[John-Paul Harold]

Initial tests show that the skipping/glitches are greatly reduced. You can still get glitches by loading some JS heavy site in another tab. Gmail, that new Slashdot page seemed to cause me issues.

Slowish scrolling of pages (in the <audio> using tab or not) generally doesn't cause glitches, however, more frenzied scrolling, such as quickly flicking the scrollwheel to return to the top of a page does. 

The resize grip in the bottom right seems to still cause glitches.

I also sent the audio stream into a spin by going to an (windows) explorer window and right clicking on a file to rename it. It didn't like that at all.

However, it is *a lot* better and improved. Thanks a lot! I can do some more testing with the <video> elem tomorrow. I've also got a DELL '3rd degree inducing burns' laptop at home I can test on.

Comment 14

[Matthew Gregan [:kinetik]]

I forgot to adjust the buffering in the Wave backend, so you might still notice (more) audio glitches playing Waves than Oggs.  It'll still benefit from the improvements to the Win32-specific audio code, but won't show the same improvement as the Ogg backend.

Thanks for your testing so far.  I'm a bit surprised that you can still hear glitches fairly easily, I have to load up my VM (running on a faster machine, admittedly) quite heavily to cause an underrun now.  I'll have to find a slower machine to test on, and also try using XP--Vista has a bunch of "glitch free" improvements in the OS code that may have some effect on our behaviour.

Comment 15

[Matthew Gregan [:kinetik]]

Attachment: changes so far (not for review)

Comment 16

[IU]

Too bad I'm away right now.  I run a P3 933 MHz Win XP machine.  If there's glitching, my system is a magnet for it. :-).  Unfortunately, I won't be able to test on my system till July 28th.

I might try through RDP (Remote Desktop) to my system, but I'm not sure the results will be all that great.

Comment 17

[John-Paul Harold]

I've tested this off and on for about a week now. I've written a web jukebox type of app, using the <audio> element, so for testing I've left that playing for hours at a time. Mostly unattended.

To clarify on the previous note I made, it seems subject to audio issues when the CPU use (which I'm measuring in task manager) spikes or stays high (upper 90 %) I last heard the audio buckle for a prolonged amount of time when svchost.exe suddenly decided to use 90+% of CPU. Loading gmail or slashdot, with their high JS use I suppose, causes a similar spike in CPU consumption.

The same is true for <video> in my tests. 

I stress again, it's a whole heap betterthan what's in 3.5 currently, but it still has some issues, at least in my use case. I guess it depends on how much you can improve what's in the Mozilla code, and how much you are at the mercy of the OS audio system.

Let me know if I can help/test in some other way.

Comment 18

[IU]

Any chance I can get my mitts on that try server build?  I'm back and ready to test, but the build is no longer on the server.

Thanks.

Comment 19

[Steven Rowat]

I have the same audio choppiness on Windows XP (ThinkPad T22), and since I'm developing a page with multiple audio controllers on it, I ran a small test:

--running 1 ogg vorbis file, plays fine, CPU about 40%, unless I scroll (and get the choppiness described in the current bug.)
--running 4 ogg vorbis files simultaneously, all of them play fine, all at once, even though the CPU meter shows 100% by the time I've enabled the 4th one. Then if I try to scroll, they all chop up.

My conclusion is that it doesn't seem to be just lack of CPU that's causing the chopping up; it's an interaction with the scrolling.

Comment 20

[IU]

Is there any way I can get a copy of the try server build?  Or can we expect a reviewable patch anytime soon?

Thanks

Comment 21

[IU]

OK I finally got an opportunity to test the July 8 try server build.  Thanks, John-Paul.

There is some improvement, but I notice some video jumpiness, on lower resolution videos, I don't think I've seen before.

Audio does go choppy if things are being done with the browser.  On higher resolution videos, simply moving the mouse causes choppiness.

Comment 22

[John-Paul Harold]

so, is it decided that WinXP is unlikely to get any improved <audio> performance? Chrome browser is playing audio now, and while it too is subject to jitters and interruptions, it seems a lot more forgiving than Firefox, with or without these patches.

Comment 23

[cajbir (:cajbir)]

I don't see any comments on the bug indicating that it's not going to be fixed so I think you can safely assume that improved audio is a goal.

Comment 24

[Matthew Gregan [:kinetik]]

The new Ogg backend landed recently, which has changed behaviour here.  For some people, it seems to have made problems worse.  The good news is that the new backend is much easier to modify to do what we want, which is to decode more audio data ahead of the video and queue it for playback at the audio backend.  I made some more specific comments here: https://bugzilla.mozilla.org/show_bug.cgi?id=531340#c95

Comment 25

[Matthew Gregan [:kinetik]]

Attachment: patch v0

Increase the Windows backend buffer size from 32kB (180ms) to 160kB (900ms) (times for 48kHz stereo).  This brings it closer to the other backends (Mac's ~5000ms, Linux depends on the ALSA/PulseAudio config, but is usually 250-2000ms).

Comment 26

[Matthew Gregan [:kinetik]]

http://hg.mozilla.org/mozilla-central/rev/54bac6389504

This should help, but it may not be a complete fix.

Comment 27

[IU]

HUUUUUGE improvement!  Thanks.  I can now playback video with pretty much no more audio stuttering (only a tiny bit at the very beginning, if video starts too soon before much has been buffered).

I'm now able to switch tabs, open various browser dialogs, refresh pages, etc, without any audio stuttering, while video is playing.

But, I must ask, why only 900 ms?  Why not 1500 or 2000 ms?  Is there some sort of hard limit imposed by the Windows OS?

http://hg.mozilla.org/mozilla-central/rev/72c7b128abaa

Tested against:
http://www.dailymotion.com/openvideodemo
http://www.spreadfirefox.com/5years/en-US/
http://lachy.id.au/dev/markup/tests/html5/video/003.html

Comment 28

[Matthew Gregan [:kinetik]]

> But, I must ask, why only 900 ms?  Why not 1500 or 2000 ms?  Is there some sort
of hard limit imposed by the Windows OS?

No, it's pretty arbitrary.  But we need to balance the buffer size with other concerns (such as memory use).  The bigger the buffer is, the more memory is used by every playing video/audio element.  There's already buffering going on at a higher layer--as the decoder runs it queues decoded audio samples ready for playback.  The decoder is trying to refill the backend audio buffers as quickly as possible in a dedicated audio thread, so the backend buffer only needs to be large enough to cover any delays the audio thread might suffer when attempting to refill.

It's possible to increase the buffer size further if necessary, but let's see how we go with the current size for now.  The glitch at the start of playback is more likely to be caused by a different bug, such as the decoder not pre-filling the audio buffers sufficiently before beginning playback.

Comment 29

[IU]

OK then.

Wonder what's going on with the following OGG radio stream, though: http://www.radiopanik.org/spip/Ecoutez-radio-Panik

This one stutters every no-and-then (especially on session saver writes, but also when doing anything with the browser).  Seems like it's under-running.

Comment 30

[Matthew Gregan [:kinetik]]

I'm marking this as fixed on trunk.  I'm going to post a similar patch for the 1.9.2 branch for approval that will improve matters there as well (but probably not quite as much in all cases--see bug 526411 comment 76 for rationale).

IU, would you mind filing a new bug for the radiopanik issue?  It's likely to be a different issue and needs separate investigation.

Comment 31

[Matthew Gregan [:kinetik]]

Attachment: similar patch for 1.9.2 branch

Increase Windows backend buffers from 32kB to 160kB.  Change Ogg and Wave decoders to make them prepare more audio for the backend.

Comment 32

[Matthew Gregan [:kinetik]]

Comment on attachment 437154 [details] [diff] [review]
similar patch for 1.9.2 branch

Fairly safe change that has already been partially made on trunk (the sydneyaudio part) and only increases the sizes of existing buffers.  Should improve audio drop-out behaviour on lower spec machines considerably.

Comment 33

[IU]

(In reply to comment #30)
> IU, would you mind filing a new bug for the radiopanik issue?  It's likely to
> be a different issue and needs separate investigation.

OK.  I'll file something later in the week.

Comment 34

[IU]

No longer experiencing the radiopanik issue with recent builds.  Unable to reproduce with several different streams, so issue is resolved for me and no new bug to be file.  Thanks.

Comment 35

[LegNeato]

Comment on attachment 437154 [details] [diff] [review]
similar patch for 1.9.2 branch

a=LegNeato for 1.9.2.5. Please ONLY land this on mozilla-1.9.2 default, as we
are still working on 1.9.2.4 on the relbranch

Comment 36

[Tony Chung [:tchung]]

Using attached testcase, Verified fix on Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.7pre) Gecko/20100630 Namoroka/3.6.7pre (.NET CLR 3.5.30729)