849264 - Concurrent playback of lots of MP3 <audio> blob URLs broken on Windows 7

Status: RESOLVED DUPLICATE of bug 1353629

(Core :: Audio/Video: Playback, defect)

Description

[Katelyn Gadd (:kael)]

Recent builds of Aurora appear to have broken <audio> playback from Blob URLs. JSIL games use Blob URLs as a way to ensure that sound effects are fully preloaded and individual playback does not result in network traffic (mostly as a workaround for bugs in other browsers), which means that every time a sound effect is played, it's played from a Blob URL. This used to work perfectly in FF and still works fine in most other browsers.

As of recently, though, over time sound effect playback stops working in FF Aurora and Nightly, with errors like this appearing in the web console:

[08:36:04.003] Media resource blob:01e4b05b-1588-487c-9c43-6b0905e7632e could not be decoded. @ http://uploads.ungrounded.net/alternate/623000/623433_alternate_372.zip/15/index.html
[08:36:04.022] Media resource blob:5d48d083-a60b-4398-aadf-9e188ce3c8cd could not be decoded. @ http://uploads.ungrounded.net/alternate/623000/623433_alternate_372.zip/15/index.html
[08:36:04.022] Media resource blob:07821957-71ac-4945-9a4f-45c2769de91e could not be decoded. @ http://uploads.ungrounded.net/alternate/623000/623433_alternate_372.zip/15/index.html
[08:36:04.074] Media resource blob:8a97dc50-6f38-4c04-9666-e7409a82530e could not be decoded. @ http://uploads.ungrounded.net/alternate/623000/623433_alternate_372.zip/15/index.html
[08:36:04.121] Media resource blob:11f523fd-d122-4595-9695-526d223df2db could not be decoded. @ http://uploads.ungrounded.net/alternate/623000/623433_alternate_372.zip/15/index.html
[08:36:04.186] Media resource blob:8a97dc50-6f38-4c04-9666-e7409a82530e could not be decoded. @ http://uploads.ungrounded.net/alternate/623000/623433_alternate_372.zip/15/index.html

Comment 1

[Katelyn Gadd (:kael)]

Oh, sorry. Observed in Aurora 21.0a2 (2013-03-07)

Furthermore, if I open one of the blobs that could not be decoded (by pasting the URI into the address bar) it plays the sound just fine in another tab. So the blobs are still there.

Comment 2

[Katelyn Gadd (:kael)]

Also reproduces in Nightly 22.0a1 (2013-03-08) after letting the game run and play creature sound effects for maybe 1-2 minutes. Same symptoms and log errors.

Comment 3

[Katelyn Gadd (:kael)]

padenot on IRC suggested disabling media.windows-media-foundation.enabled. Setting this to false causes Firefox to load the ogg versions of the sounds instead of the MP3s, and everything seems to work fine.

For reference, here is the logic used to select a mime type for audio to play:
https://github.com/kevingadd/JSIL/blob/master/Libraries/JSIL.Browser.Audio.js#L373

It looks like the problem is that the support for MP3 playback landing causes me to pick MP3 over OGG (since you now support both formats) and there's something wrong with MP3 playback. This is probably Windows specific.

Comment 4

[Katelyn Gadd (:kael)]

I just remembered that this may have also broken looping (via the 'loop' attribute) for MP3s as well. I've seen looping music stop looping in FF recently and that is probably caused by this same phenomenon.

Comment 5

[Katelyn Gadd (:kael)]

One other point of comparison:

With media foundation turned on (using MP3), the number of Win32 threads in firefox.exe reaches around 180 by the time playback stops working. I think the playback may be failing because firefox.exe has run out of address space and can't create threads anymore.

With media foundation turned off (using OGG) the number of threads stays relatively steady between 50 and 60.

Comment 6

[Katelyn Gadd (:kael)]

Bah, adding a comment with an old tab changed the hardware field. Sorry paul.

Comment 7

[Chris Pearce [:cpearce (Not reading bugmail)]]

Attachment: Minimized testcase

I can reproduce the bug with the attached testcase, also available at:
http://people.mozilla.org/~cpearce/audio-blob.html

If "Num elements played" is not the same as "Num elements created" after running the test, you've reproduced the bug.

This seems to be a problem with Windows Media Foundation on Windows 7.

On my Windows 8 machine I can create up to 10000 audio elements with blob src's in Firefox Nightly using the testcase above, but on my Windows 7 machine Nightly can't handle 70.

By comparison, on Win8 IE10 can handle about 870 audio elements with blob src, and in Chrome the tab crashes when it reaches about 90 created.

On Windows 7, IE9 handles 2 (!!) and Chrome Canary handles about 250 before crashing (so Chrome is getting better at least).

Comment 8

[Katelyn Gadd (:kael)]

IE9's behavior isn't a surprise; they have an internal limit of something like 6 total <audio> instances after which they silently ignore any attempts to play new audio. Cool to hear that this is apparently fixed in IE10 on Win8, though - I wonder if that old instance limit was caused by this Media Foundation problem?

Is it intended for the thread count to skyrocket the way it does with media foundation enabled? That indicates a bug to me because the count never goes down.

Thanks for creating a reduced test case, I hadn't realized this applied to data URIs as well. When I run it in Aurora on my Win7 box it conks out after 61 plays.

Hearing that this causes Chrome tab crashes is unfortunate. I guess I'll have to keep using my stupid Chrome only Web Audio backend for them :-(

Comment 9

[Katelyn Gadd (:kael)]

Digging into the Chrome tab crash, it appears to be choking for the same reason FF Nightly does for me: It looks like it spawns a thread (if not multiple threads) per audio element and the tab crashes once it runs out of address space. I wonder if they're also using Media Foundation to decode?

Comment 10

[Chris Pearce [:cpearce (Not reading bugmail)]]

Actually I do sometimes see this in Nightly on Windows 8 too, but at around 5000 elements created in the above testcase.

Kevin: If you give your audio elements a preload="none" attribute, do you still see the bug in your game? You'd need to set the preload attribute before setting the src, i.e.:

var a = new Audio();
a.preload="none";
a.src = someDataUri;

That way we won't initialize the Windows Media Foundation decoders until you call play() on the elements.

Though I suppose that would just be delaying hitting the bug; once you've played enough elements and initialized enough WMF decoders, presumably you'd hit the same limit... 

I guess what you'd really need to do to work around this issue would be to only create the Audio elements right before you need or play them, and delete them after they finish playing.

Comment 11

[Katelyn Gadd (:kael)]

I create them right before I play them and I don't leak them (at least not on purpose), see here:

https://github.com/kevingadd/JSIL/blob/master/Libraries/JSIL.Browser.Audio.js#L288

And here:

https://github.com/kevingadd/JSIL/blob/master/Libraries/JSIL.Browser.Audio.js#L67

Is there some special magic DOM incantation I can use to ensure that an <audio> element is actually freed?

Comment 12

[Chris Pearce [:cpearce (Not reading bugmail)]]

(In reply to Kevin Gadd (:kael) from comment #11)
> Is there some special magic DOM incantation I can use to ensure that an
> <audio> element is actually freed?

There's a hack you can do which works in Firefox; set the src="" when you want to dispose of the audio element. This will force a new load of an invalid resource, which internally cleans up the old decoder. This does however cause "emptied", "loadstart" and "error" events to be dispatched to the media element, so make sure you remove any such listeners before doing this.


(In reply to Kevin Gadd (:kael) from comment #9)
> Digging into the Chrome tab crash [...] I wonder if they're also using Media Foundation to decode?

They were using ffmpeg last time I checked.


(In reply to Kevin Gadd (:kael) from comment #8)
> Is it intended for the thread count to skyrocket the way it does with media
> foundation enabled? That indicates a bug to me because the count never goes
> down.

It's not unsurprising that we create a lot of threads. Firefox allocates one thread to manage the decoding, and another to play the audio. Windows Media Foundation will allocate threads too. Our threads only reserve 1MB virtual address space for their stacks, but I don't know how much memory the threads WMF creates are reserving for their address space. We shut our threads down when we're not using them though, i.e. when the audio isn't playing.

That being said, if you're ensuring that audio elements are being released properly, then the thread count *should* go down. If the thread count isn't going down, that would indicate that there's some reference to the audio element keeping it alive... There couldn't be event listeners still attached could there? Or it would indicate a bug in Firefox of course. ;)

Comment 13

[Katelyn Gadd (:kael)]

I never detach my "ended" event listener explicitly; I thought event listeners causing elements to leak was fixed a long time ago in modern browsers? I'll try detaching them manually and setting src to "" and see if the problem goes away.

Comment 14

[Katelyn Gadd (:kael)]

OK, so, I made the two changes you described:

I already had a listener for 'ended' that I was using to mark a sound as no longer playing, and 'dispose' it. I updated the handler for the 'dispose' event so that it now unregisters the event listener and sets the node's src to "".

This seems to fix the problem where Nightly leaks threads and audio playback stops working.

The downside is that I get a huge spam of errors in the developer console:

"Invalid URI. Load of media resource  failed."

It seems like if the src="" hack is what I'm supposed to use, it shouldn't produce an error.

Comment 15

[Paul Adenot (:padenot)]

To avoid the error spam in the console, just do a |removeAttribute("src");| and then a |Load()| instead of the |src = ""|.

Comment 16

[Katelyn Gadd (:kael)]

That seems like it works, Paul. Thanks!