478299 - Crash in [@ winmm.dll@0x490d][@ ValidateHandle - waveOutReset - closeAudio] while pausing video on tinyvid.tv and no audio output device enabled

Status: VERIFIED FIXED

(Core :: Audio/Video, defect, P1)

Description

[Marcia Knous [:marcia]]

Seen while running Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1b3pre) Gecko/20090212 Shiretoko/3.1b3pre.

STR:
1. Visit http://tinyvid.tv/show/22axtzbrwetr4 and while the video is still loading, hit the pause button.
2. I crash, 100% reproducible. I also crash trying the same steps on other videos on that same page.

Machine is 32 bit Windows Vista Ultimate with SP 1, 4 GB RAM.

Comment 1

[Marcia Knous [:marcia]]

Sorry, forgot to include the breakpad IDs of the first crash:

http://crash-stats.mozilla.com/report/index/b8dd5c2f-c3e9-47a9-b76f-213cc2090212

Comment 2

[Chris Pearce [:cpearce (Not reading bugmail)]]

WFM on Vista Business SP1. Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1b3pre) Gecko/20090212 Shiretoko/3.1b3pre (.NET CLR 3.5.30729) Ubiquity/0.1.4

Mozilla central nightly 20090212 WFM too.

Comment 3

[Marcia Knous [:marcia]]

So if the video is loaded, it will not crash for me, but if it has not fully loaded I can crash at will. Otherwise maybe an issue of the version of Windows Vista? Not sure if it matters but I also have the Aero stuff on, not Windows Classic.

Comment 4

[Marcia Knous [:marcia]]

Tried on my Windows Vista Ultimate laptop and could not reproduce the issue. This machine is running Win Vista Ultimate SP1 as well so I am not sure why the Win Vista machine in the lab crashes so easily.

Comment 5

[Marcia Knous [:marcia]]

One difference is the machine in the lab has a NVIDIA Quadro FX 3450/4000 SDI display adapter while the machine in my office does not.

Comment 6

[Marcia Knous [:marcia]]

The laptop has an ATI Mobility Radeon X1400 if that helps.

Comment 7

[Marcia Knous [:marcia]]

Attachment: Screenshot of WinDbg

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Chris, as what we have seen in the lab is that this crash only happens if you don't have any output device enabled. Just go to the tray area and disable your device. That should be enough.

As you can see from the screenshot we call a Windows API function with an illegal pointer. After the crash the driver is somewhat broken which doesn't let you re-enable the output device. I had to remove the audio device in the device manager and search for it to get it back.

The crash happens here:
http://mxr.mozilla.org/mozilla-central/source/media/libsydneyaudio/src/sydney_audio_waveapi.c#455

I think that we should block on this.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+1]]

As what I've seen, disabled output devices only gets hidden by windows. You can show them again within the same dialog.

Comment 10

[Chris Pearce [:cpearce (Not reading bugmail)]]

Muting my sound doesn't allow me to reproduce this unfortunately. I even tried disabling my sound driver in Device Manager, but I still can't reproduce the crash.

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Please do not disable your sound driver! In the following are the steps we were using and which gives us the chance to reproduce it on three different machines:

1. Right click the speaker icon in the tray area and select playback devices
2. In the dialog right click your speaker and disable it
3. Click ok
4. Notice you have a red cross over your speaker icon
5. Open http://tinyvid.tv/show/22axtzbrwetr4
6. Click on play/pause several times after the video has been loaded

With step 6 you should be able to crash the browser each time.

Comment 12

[Chris Pearce [:cpearce (Not reading bugmail)]]

Sorry, I still can't reproduce the bug even when exactly following your STR. :(

Comment 13

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Chris, do you have any idea what's going on when looking at the attachment? How can we help to investigate this problem? Timeless, is there a way to save the whole stack information from within WinDBG? We haven't found a way yet.

Comment 14

[Chris Pearce [:cpearce (Not reading bugmail)]]

I've asked Matthew Gregan to take a look, he worked on the Wav backend, so is more familiar with libsyndeyaudio.

Comment 15

[timeless]

personally, i'd save a dump and share it, for that, you need bandwidth

.dump /maipwd /u /ba /c "ValidateHandle - waveOutReset - closeAudio" c:\478299

for stack copying, your choices are:

!analyze -v -f
(As suggested in How to get a stack trace with windbg)

or some variant in the k family (I typically use kp, but):
kP

your other bits are (get locals from some frame):
.frame /r 3; dv

and (.hh !handle - get info about handles)
!handle

Offhand, it sounds like you might have closed a handle twice. is this possible?
(if so, breakpoints would be helpful.)

Comment 16

[Chris Pearce [:cpearce (Not reading bugmail)]]

Ok, I was able to reproduce this in nightly builds, and in local opt builds. It's a bug in libsydneyaudio. In sydney_audio_waveapi.c, sa_stream_create_pcm(), they don't zero the memory for a new sa_stream_t. Then when the sa_stream_open()/openAudio() fails, it bails out, and we call sa_stream_destroy(), which calls in and tries to call waveOutReset() on the handle which is stored in the sa_stream_t. This will have a garbage value. Normally waveOutReset() is smart enough to recognize that the handle is invalid, but in some cases it isn't, and we get the crash. The fix is simple, just zero the memory in sa_stream_create_pcm(). We'll need to upstream the fix as well.

Comment 17

[Chris Pearce [:cpearce (Not reading bugmail)]]

Henrik, I've made a patched build on TryServer, can you download and test to see if the bug is fixed for you with this build:

https://build.mozilla.org/tryserver-builds/2009-02-16_15:[e-mail]-1234827857/

Comment 18

[Chris Pearce [:cpearce (Not reading bugmail)]]

Attachment: libsydneyaudio patch v1 - zero new sa_stream_t

Patch required to fix libsydneyaudio. Will follow up with another patch which applies this patch during build, like we already do for bug 469698.

Comment 19

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Yes, that works fine. No crash anymore. (Ok, I get a crash when the awesome bar autocomplete opens but thats another issue for current trunk builds perhaps - will look at this).

Comment 20

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Timeless, I'll note this great hints for the next debug session. Thanks!

Comment 21

[Chris Pearce [:cpearce (Not reading bugmail)]]

Attachment: Patch v1 - calloc instead of malloc, change update.sh [Checkin: Comment 26 & 28]

Patch which adds patch to media/libsydneyaudio and changes update.sh, so we can track the divergence from straight libsydneyaudio. I haven't managed to file a bug with trac.annodex, it's not emailed me back with account confirmation yet.

Comment 22

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Pushed http://hg.mozilla.org/mozilla-central/rev/fd60f1dd1299

Comment 23

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Patch looks great. I cannot reproduce the crash with Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2a1pre) Gecko/20090219 Minefield/3.2a1pre ID:20090219032045

Comment 24

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Which priority has this bug? Will it be included in beta 3?

Comment 25

[Serge Gautherie (:sgautherie)]

(In reply to comment #22)
> Pushed http://hg.mozilla.org/mozilla-central/rev/fd60f1dd1299

"Bug 478684 - test_play.html fails intermittently"
This doesn't look like to be the checkin for this bug...

Comment 26

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Sorry, it was http://hg.mozilla.org/mozilla-central/rev/4470cdc3409d

This should make beta3, yes.

Comment 27

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to comment #26) 
> This should make beta3, yes.

Setting to P1 accordingly.

Comment 28

[Serge Gautherie (:sgautherie)]

Comment on attachment 362770 [details] [diff] [review]
Patch v1 - calloc instead of malloc, change update.sh
[Checkin: Comment 26 & 28]


http://hg.mozilla.org/releases/mozilla-1.9.1/rev/5bd71b6ef46d

Comment 29

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Verified fixed on 1.9.1 with Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1b3pre) Gecko/20090225 Shiretoko/3.1b3pre ID:20090225031913

Comment 30

[Matthew Gregan [:kinetik]]

Filed Annodex Trac ticket #452 for this issue: https://trac.annodex.net/ticket/452