Last Comment Bug 688997 - Pointer truncation in waveOutProc callback (x64)
: Pointer truncation in waveOutProc callback (x64)
: crash
Product: Core
Classification: Components
Component: Audio/Video (show other bugs)
: Trunk
: x86_64 Windows 7
-- normal (vote)
: mozilla9
Assigned To: Matthew Gregan [:kinetik]
: Maire Reavy [:mreavy] Please needinfo me
Depends on:
  Show dependency treegraph
Reported: 2011-09-24 13:20 PDT by bugzilla.10.animefan
Modified: 2011-09-26 07:44 PDT (History)
3 users (show)
See Also:
Crash Signature:
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---

patch v0 (2.02 KB, patch)
2011-09-24 17:47 PDT, Matthew Gregan [:kinetik]
cpearce: review+
Details | Diff | Splinter Review

Description User image bugzilla.10.animefan 2011-09-24 13:20:14 PDT
User Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)

Steps to reproduce:

Trying to play the video at crashes Firefox 9.0 x64 nightly on a system with more than 4 GB RAM and AllocationPreference set to 0x100000 in the registry.

Actual results:

Firefox 9.0 x64 crashed in

xul!waveOutProc+0x1f [e:\builds\moz2_slave\m-cen-w64-ntly\build\media\libsydneyaudio\src\sydney_audio_waveapi.c @ 639]

Firefox uses this signature for the callback:

void CALLBACK waveOutProc(
     HWAVEOUT hWaveOut, 
     UINT uMsg, 
     DWORD dwInstance,  
     DWORD dwParam1,    
     DWORD dwParam2     
This is not correct. According to MSDN this is the correct signature for this callback:
void CALLBACK waveOutProc(
  UINT uMsg,
  DWORD_PTR dwInstance,
  DWORD_PTR dwParam1,
  DWORD_PTR dwParam2

( reference : )

By assigning the dwInstance to a DWORD the pointer to the handle can get truncated on 64 bit systems.

Expected results:

Firefox should not crash.

Use the correct calling convention for the callback.
Comment 1 User image Matthew Gregan [:kinetik] 2011-09-24 17:47:18 PDT
Created attachment 562269 [details] [diff] [review]
patch v0
Comment 2 User image Matthew Gregan [:kinetik] 2011-09-25 18:53:28 PDT

Note You need to log in before you can comment on or make changes to this bug.