942819 - crash in mozilla::layers::DeviceManagerD3D9::Init()

Status: VERIFIED FIXED

(Core :: Graphics: Layers, defect)

Description

[Tracy Walker [:tracy]]

This bug was filed from the Socorro interface and is 
report bp-424b3d3c-7a32-4d20-a092-ccbb22131124.
=============================================================

This is a very explosive top crasher that spiked to #1 on Nightly builds beginning 2013112303.  

Crash Reason 	EXCEPTION_ACCESS_VIOLATION_WRITE

Frame 	Module 	Signature 	Source
0 	xul.dll 	mozilla::layers::DeviceManagerD3D9::Init() 	gfx/layers/d3d9/DeviceManagerD3D9.cpp
1 	xul.dll 	mozilla::layers::LayerManagerD3D9::Initialize(bool) 	gfx/layers/d3d9/LayerManagerD3D9.cpp
2 	xul.dll 	nsWindow::GetLayerManager(mozilla::layers::PLayerTransactionChild *,mozilla::layers::LayersBackend,nsIWidget::LayerManagerPersistence,bool *) 	widget/windows/nsWindow.cpp
3 	xul.dll 	nsWindow::AllowD3D9WithReinitializeCallback(nsWindow *) 	widget/windows/nsWindow.cpp
4 	xul.dll 	nsWindow::EnumAllThreadWindowProc(HWND__ *,long) 	widget/windows/nsWindow.cpp
5 	user32.dll 	InternalEnumWindows 	
6 	user32.dll 	EnumThreadWindows 	
7 	xul.dll 	nsWindow::EnumAllWindows(void (*)(nsWindow *)) 	widget/windows/nsWindow.cpp
8 	xul.dll 	nsWindow::StartAllowingD3D9(bool) 	widget/windows/nsWindow.cpp
9 	xul.dll 	StartAllowingD3D9 	widget/windows/nsToolkit.cpp
10 	xul.dll 	nsTimerImpl::Fire() 	xpcom/threads/nsTimerImpl.cpp
11 	xul.dll 	nsThread::ProcessNextEvent(bool,bool *) 	xpcom/threads/nsThread.cpp
12 	xul.dll 	NS_ProcessNextEvent(nsIThread *,bool) 	xpcom/glue/nsThreadUtils.cpp
13 	xul.dll 	mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate *) 	ipc/glue/MessagePump.cpp
14 	xul.dll 	MessageLoop::RunHandler() 	ipc/chromium/src/base/message_loop.cc
15 	xul.dll 	MessageLoop::Run() 	ipc/chromium/src/base/message_loop.cc
16 	xul.dll 	nsBaseAppShell::Run() 	widget/xpwidgets/nsBaseAppShell.cpp
17 	xul.dll 	nsAppShell::Run() 	widget/windows/nsAppShell.cpp
18 	xul.dll 	nsAppStartup::Run() 	toolkit/components/startup/nsAppStartup.cpp
19 	xul.dll 	XREMain::XRE_mainRun() 	toolkit/xre/nsAppRunner.cpp
20 	xul.dll 	XREMain::XRE_main(int,char * * const,nsXREAppData const *) 	toolkit/xre/nsAppRunner.cpp
21 	xul.dll 	XRE_main 	toolkit/xre/nsAppRunner.cpp
22 	firefox.exe 	do_main 	browser/app/nsBrowserApp.cpp
23 	firefox.exe 	NS_internal_main(int,char * *) 	browser/app/nsBrowserApp.cpp
24 	firefox.exe 	wmain 	toolkit/xre/nsWindowsWMain.cpp
25 	firefox.exe 	__tmainCRTStartup 	f:/dd/vctools/crt_bld/self_x86/crt/src/crtexe.c
26 	kernel32.dll 	BaseThreadInitThunk 	
27 	ntdll.dll 	__RtlUserThreadStart 	
28 	ntdll.dll 	_RtlUserThreadStart

Comment 1

[Robert Kaiser]

Bug 900248 landed on the 22nd and changed the assignment of the variable that frame #1 is accessing a code line later. CCing nrc and Bas because of that.
I'm not setting this blocking the other one as I'm not sure that it was the cause but it looks very likely.

Comment 2

[Robert Kaiser]

Also, this is by far the topcrasher on 28 right now, requesting tracking for that.

Comment 3

[Nick Cameron [:nrc]]

(In reply to Robert Kaiser (:[e-mail]) from comment #1)
> Bug 900248 landed on the 22nd and changed the assignment of the variable
> that frame #1 is accessing a code line later. CCing nrc and Bas because of
> that.
> I'm not setting this blocking the other one as I'm not sure that it was the
> cause but it looks very likely.

I'm not sure if I'm doing it right, but from following the link to the crash report and then to hg, I get code from before bug 900248 landed. Does that imply 900248 is not to blame? Or are there many other crash reports after it landed?

Comment 4

[Nick Cameron [:nrc]]

Ah, I see what is wrong. Patch coming up...

Comment 5

[Nick Cameron [:nrc]]

The error happens because in frame 0 |this| is null, thus we get a write error when storing into mFocusWnd. That happens because GetD3D9DeviceManager can return null. In fact the initialisation is done in GetD3D9DeviceManager, so we don't need to do it in the LayerManager too.

(In reply to Nick Cameron [:nrc] from comment #3)
> I'm not sure if I'm doing it right, but from following the link to the crash

And, no, I was not doing it right at all :-|

Comment 6

[Nick Cameron [:nrc]]

Attachment: fix (and fix a similar issue in TextureD3D9)

Comment 7

[Nick Cameron [:nrc]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/4fa706cb38bb

Comment 8

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/4fa706cb38bb

Comment 9

[Tracy Walker [:tracy]]

no longer crashing on builds beyond 20131126