Closed Bug 823077 Opened 12 years ago Closed 9 years ago

crash in gfxWindowsPlatform::UpdateRenderMode @ nvwgf2um.dll@0xf5ddf (Updating Nvidia driver 310.70)

Categories

(Core :: Graphics, defect)

Product:
Core
Component:
Graphics
Version:
17 Branch
Platform:
x86
Windows 7
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

()

Status:
RESOLVED INCOMPLETE

People

(Reporter: marcia, Unassigned)

References

Details

(Keywords: crash, Whiteboard: [startupcrash][gfx-noted])

Crash Data

This bug was filed from the Socorro interface and is report bp-1bf3e70d-46b5-4272-b335-0e2eb2121218 . ============================================================= Seen while looking at crash stats and the most recent explosive report. Comments indicating at the time of the crash people were updating there Nvidia graphics driver 310.70 and that they are using hardware acceleration. Link to crashes: https://crash-stats.mozilla.com/report/list?signature=nvwgf2um.dll@0xf5ddf Frame Module Signature Source 0 nvwgf2um.dll nvwgf2um.dll@0xf5ddf 1 nvwgf2um.dll nvwgf2um.dll@0xf4194 2 nvwgf2um.dll nvwgf2um.dll@0x1422aa 3 nvwgf2um.dll nvwgf2um.dll@0x1417dc 4 nvwgf2um.dll nvwgf2um.dll@0x16dbff 5 nvwgf2um.dll nvwgf2um.dll@0x16e0d2 6 nvwgf2um.dll nvwgf2um.dll@0x141964 7 nvwgf2um.dll nvwgf2um.dll@0x16d69a 8 nvwgf2um.dll nvwgf2um.dll@0x141a7d 9 nvwgf2um.dll nvwgf2um.dll@0x16d69a 10 nvwgf2um.dll nvwgf2um.dll@0x16cd8b 11 nvwgf2um.dll nvwgf2um.dll@0x16d120 12 nvwgf2um.dll nvwgf2um.dll@0x16d680 13 nvwgf2um.dll nvwgf2um.dll@0x16cd8b 14 nvwgf2um.dll nvwgf2um.dll@0x16d120 15 nvwgf2um.dll nvwgf2um.dll@0x16d680 16 nvwgf2um.dll nvwgf2um.dll@0x16cd8b 17 nvwgf2um.dll nvwgf2um.dll@0x16d120 18 nvwgf2um.dll nvwgf2um.dll@0x16d680 19 nvwgf2um.dll nvwgf2um.dll@0xf3f00 20 nvwgf2um.dll nvwgf2um.dll@0x14007d 21 nvwgf2um.dll nvwgf2um.dll@0x140304 22 nvwgf2um.dll nvwgf2um.dll@0x1404ad 23 nvwgf2um.dll nvwgf2um.dll@0x13e6d6 24 nvwgf2um.dll nvwgf2um.dll@0x70f2 25 nvwgf2um.dll nvwgf2um.dll@0xc68b0 26 dlumd32.dll dlumd32.dll@0x26f00 27 dlumd32.dll dlumd32.dll@0x41e83 28 dxgi.dll CD3D10Device::DestroyDriverInstance 29 d3d10_1core.dll CDevice::LLOBeginLayerDestruction 30 d3d10_1core.dll CBridgeImpl<ILayeredLockOwner,IDXGILayeredDevice,CLayeredObject<CDevice> >::LLOB 31 dxgi.dll CD3D10Device::LLOBeginLayerDestruction 32 dxgi.dll CBridgeImpl<ILayeredLockOwner,IDXGILayeredDevice,CLayeredObject<CD3D10Device> >: 33 d3d10_1core.dll NMultithread::CDevice::LLOBeginLayerDestruction 34 d3d10_1core.dll CBridgeImpl<ILayeredLockOwner,IDXGILayeredDevice,CLayeredObject<NMultithread::CD 35 dxgi.dll NOutermost::CDevice::LLOBeginLayerDestruction 36 dxgi.dll NOutermost::CDevice::FinalRelease 37 dxgi.dll TComObject<NOutermost::CDevice>::~TComObject<NOutermost::CDevice> 38 dxgi.dll TComObject<NOutermost::CDevice>::`scalar deleting destructor' 39 dxgi.dll NOutermost::CDevice::FinalRelease 40 d3d10_1core.dll CLayeredObject<NMultithread::CDevice>::CContainedObject::Release 41 xul.dll nsRefPtr<nsCSSStyleSheet>::assign_with_AddRef obj-firefox/dist/include/nsAutoPtr.h:846 42 xul.dll gfxWindowsPlatform::UpdateRenderMode gfx/thebes/gfxWindowsPlatform.cpp:474 43 xul.dll gfxWindowsPlatform::gfxWindowsPlatform gfx/thebes/gfxWindowsPlatform.cpp:365 44 xul.dll gfxPlatform::Init gfx/thebes/gfxPlatform.cpp:290 45 xul.dll gfxPlatform::GetPlatform gfx/thebes/gfxPlatform.cpp:237 46 xul.dll nsDOMClassInfo::Init dom/base/nsDOMClassInfo.cpp:4606 47 xul.dll NS_GetDOMClassInfoInstance dom/base/nsDOMClassInfo.cpp:5185 48 xul.dll nsFrameMessageManager::QueryInterface content/base/src/nsFrameMessageManager.cpp:107 49 xul.dll nsCOMPtr_base::assign_from_qi obj-firefox/xpcom/build/nsCOMPtr.cpp:56 50 xul.dll xpcObjectHelper::GetClassInfo js/xpconnect/src/xpcObjectHelper.h:71 51 xul.dll XPCWrappedNative::GetNewOrUsed js/xpconnect/src/XPCWrappedNative.cpp:493 52 xul.dll XPCConvert::NativeInterface2JSObject js/xpconnect/src/XPCConvert.cpp:920 53 xul.dll NativeInterface2JSObject js/xpconnect/src/nsXPConnect.cpp:1240 54 xul.dll nsXPConnect::WrapNative js/xpconnect/src/nsXPConnect.cpp:1272 55 xul.dll nsJSCID::GetService js/xpconnect/src/XPCJSID.cpp:788 56 xul.dll NS_InvokeByIndex_P xpcom/reflect/xptcall/src/md/win32/xptcinvoke.cpp:70 57 xul.dll XPCWrappedNative::CallMethod js/xpconnect/src/XPCWrappedNative.cpp:2406 58 xul.dll XPC_WN_CallMethod js/xpconnect/src/XPCWrappedNativeJSOps.cpp:1478 59 mozjs.dll js::InvokeKernel js/src/jsinterp.cpp:352 60 mozjs.dll js::Interpret js/src/jsinterp.cpp:2414 61 mozjs.dll js::InvokeKernel js/src/jsinterp.cpp:363 62 mozjs.dll js::Invoke js/src/jsinterp.h:119 63 mozjs.dll js_fun_call js/src/jsfun.cpp:830 64 mozjs.dll js_fun_apply js/src/jsfun.cpp:848 65 mozjs.dll js::InvokeKernel js/src/jsinterp.cpp:352 66 mozjs.dll js::Interpret js/src/jsinterp.cpp:2414 67 mozjs.dll js::InvokeKernel js/src/jsinterp.cpp:363 68 mozjs.dll js::Invoke js/src/jsinterp.cpp:396 69 mozjs.dll js::CrossCompartmentWrapper::call js/src/jswrapper.cpp:736 70 mozjs.dll proxy_Call js/src/jsproxy.cpp:1888 71 mozjs.dll js::InvokeKernel js/src/jsinterp.cpp:345 72 mozjs.dll js::Invoke js/src/jsinterp.cpp:396 73 mozjs.dll js::Shape::get js/src/jsscopeinlines.h:296 74 mozjs.dll js::NameOperation js/src/jsinterpinlines.h:417 75 mozjs.dll js::Interpret js/src/jsinterp.cpp:2517 76 mozjs.dll js::InvokeKernel js/src/jsinterp.cpp:363 77 mozjs.dll array_readonlyCommon<ArrayForEachBehavior> js/src/jsarray.cpp:3189 78 mozjs.dll array_forEach js/src/jsarray.cpp:3226 79 mozjs.dll js::InvokeKernel js/src/jsinterp.cpp:352 80 mozjs.dll js::Interpret js/src/jsinterp.cpp:2414 81 mozjs.dll js::RunScript js/src/jsinterp.cpp:301 82 mozjs.dll js::ExecuteKernel js/src/jsinterp.cpp:494 83 mozjs.dll js::XDRState<1>::codeScript js/src/vm/Xdr.cpp:147 84 mozjs.dll js::Execute
(In reply to Marcia Knous [:marcia] from comment #0) > people were updating there Nvidia graphics driver 310.70 It's confirmed by correlations: 100% (184/184) vs. 5% (7349/145097) nvwgf2um.dll 100% (184/184) vs. 1% (734/145097) 9.18.13.1070
Summary: crash in nvwgf2um (Updating Nvidia driver 310.70) → crash in gfxWindowsPlatform::UpdateRenderMode @ nvwgf2um.dll@0xf5ddf (Updating Nvidia driver 310.70)
bp-1b86bc5d-78d4-4c5c-bebe-de9a22121218 User Comments: After installing Nvidia graphics driver 310.70, firefox keeps crashing when starting up because "hardware acceleration when possible". When I turn it off, Firefox works normally. We should contact NVidia about this to make sure they know their update causes problems.
Benoit, what info should we provide to NVidia when contacting them, to make sure what we tell them is helpful?
Flags: needinfo?(bjacob)
Does the bug reproduce consistently under well-undestood conditions (e.g. with a particular driver version)? If yes, giving Nvidia steps to reproduce is the most important thing. They do care to run our testcases, at least if you ask the right person. CC'ing Piers Daniell. In any case, do give them this information: - precise NVIDIA driver version - precise Windows version (e.g. Win7 SP1) - Firefox version - 32bit or 64bit for each of the above - anything you know about what NVIDIA devices reproduce. Precise Nvidia device IDs if applicable. Check a few crash reports to see if the AdapterDeviceID's always fall in the same device family (you can use google search to map device ID's to actual device names). And do also give them a couple of good Mozilla crash links (the original one in comment 0 looks good, good stack etc). Say that it's a crash in nvwgf2um.dll under CD3D10Device::DestroyDriverInstance (the first resolved symbol in the stack). You can also say it looks like a null pointer dereference (see how the crash address 0x64 is very close to 0, so the 0x64 here is likely just an offset).
Flags: needinfo?(bjacob)
(In reply to Benoit Jacob [:bjacob] from comment #4) > In any case, do give them this information: I'll be putting everything I know in here as well. > - precise NVIDIA driver version nvwgf2um.dll 9.18.13.1070 - to whatever driver version that corresponds > - precise Windows version (e.g. Win7 SP1) The crashes all seem to be on Win7 SP1 (Windows NT 6.1.7601 Service Pack 1), judging from the "Reports" tab in https://crash-stats.mozilla.com/report/list?signature=nvwgf2um.dll@0xf5ddf > - Firefox version As the "Signature Summary" tab in the link above points out, this happens across multiple Firefox versions, on top are the current 17.0.1 release and 18.0b4 beta. > - 32bit or 64bit for each of the above As we only ship 32bit versions of Firefox on beta and release, this has only been seen in 32bit Firefox (so far). We don't see in crash reports if the OS is 32bit or 64bit. > - anything you know about what NVIDIA devices reproduce. Precise Nvidia > device IDs if applicable. Check a few crash reports to see if the > AdapterDeviceID's always fall in the same device family (you can use google > search to map device ID's to actual device names). Adapter Vendor ID: 0x10de Adapter Device ID: 0x06c4, 0x0a20, 0x0a66, 0x0dc4, 0x0de1, 0x0df0, 0x0df4, 0x0e22, 0x0e3a, 0x0f00, 0x1040, 0x1056, 0x1080, 0x1183, 0x11c0, 0x1200, 0x1201, 0x1244, 0x1251 That's in the reports I checked manually (by looking through links in the "Reports" tab). There are probably more. > And do also give them a couple of good Mozilla crash links (the original one > in comment 0 looks good, good stack etc). Say that it's a crash in > nvwgf2um.dll under CD3D10Device::DestroyDriverInstance (the first resolved > symbol in the stack). You can also say it looks like a null pointer > dereference (see how the crash address 0x64 is very close to 0, so the 0x64 > here is likely just an offset). Looking through a number of reports, I saw three different stacks: 1bf3e70d-46b5-4272-b335-0e2eb2121218 (28 frames of nvwgf2um.dll) 282cd30c-0efa-49ba-89eb-99dfd2121220 (34 frames of nvwgf2um.dll) f0107b3d-f778-4c50-8a3b-deeb92121220 (10 frames of nvwgf2um.dll) All others are repeats of those stacks - and all those call from CD3D10Device::DestroyDriverInstance into that driver library.
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #5) > Looking through a number of reports, I saw three different stacks: > 1bf3e70d-46b5-4272-b335-0e2eb2121218 (28 frames of nvwgf2um.dll) > 282cd30c-0efa-49ba-89eb-99dfd2121220 (34 frames of nvwgf2um.dll) > f0107b3d-f778-4c50-8a3b-deeb92121220 (10 frames of nvwgf2um.dll) Actually, let's put it into a form that is getting linked here in Bugzilla: bp-1bf3e70d-46b5-4272-b335-0e2eb2121218 bp-282cd30c-0efa-49ba-89eb-99dfd2121220 bp-f0107b3d-f778-4c50-8a3b-deeb92121220
Whiteboard: [startupcrash]
Those crash reports we have been seeing are mostly in the first two seconds of starting Firefox, those that crash later might be slow computers that just take longer to launch it. We don't get any URLs reported, which tell me that we crash even before we can load the home page. Note that I've now even seen a crash report from Windows 8, see bp-5fc8428b-0725-445b-99a0-1b1272121221 and we're also seeing some on Win7 without SP1, e.g. bp-ace5c4c9-2e81-4dc2-92d4-74a832121221 What I find interesting is that so early in the startup, the last call from Windows libraries before entering nvwgf2um.dll is ::DestroyDriverInstance which sounds like the crash occurs when actually shutting down or deleting a driver instance. The last frames in Mozilla code before entering Windows libraries are those: 41 xul.dll nsRefPtr<nsCSSStyleSheet>::assign_with_AddRef obj-firefox/dist/include/nsAutoPtr.h:846 42 xul.dll gfxWindowsPlatform::UpdateRenderMode gfx/thebes/gfxWindowsPlatform.cpp:474 43 xul.dll gfxWindowsPlatform::gfxWindowsPlatform gfx/thebes/gfxWindowsPlatform.cpp:365 44 xul.dll gfxPlatform::Init gfx/thebes/gfxPlatform.cpp:290 That line in gfxWindowsPlatform::UpdateRenderMode is a call to VerifyD2DDevice() so it looks to me like we crash when trying to verify that D2D works. Benoit, it looks like the folks at NVidia are unable to reproduce and might have a hard time finding out what's going on there. Does that info from the call stacks or something else we have in the reports give us some info we can forward to them to help them here?
Flags: needinfo?(bjacob)
I'm not a D3D specialist, but Bas is: CC'ing him. Maybe there exists a tracing tool that would allow us to record a trace of what's happening here, and give it to Nvidia. Looking more closely at the crash reports: An interesting bit is that the AppNotes don't have any of our graphics features annotations (like "D2D? D2D+"...). That means that we crash _before_ hitting any ScopedGfxFeatureReporter. Now looking at the stack, we are crashing inside of VerifyD2DDevice that's been inlined into UpdateRenderMode at line 474. This VerifyD2DDevice starts as follows: void gfxWindowsPlatform::VerifyD2DDevice(bool aAttemptForce) { #ifdef CAIRO_HAS_D2D_SURFACE if (mD2DDevice) { ID3D10Device1 *device = cairo_d2d_device_get_device(mD2DDevice); if (SUCCEEDED(device->GetDeviceRemovedReason())) { return; } mD2DDevice = nullptr; } mozilla::ScopedGfxFeatureReporter reporter("D2D", aAttemptForce); ... (more code) ... Again, since we don't have any "D2D" annotation in the crash report, we must be crashing right there above the ScopedGfxFeatureReporter line. That leaves only the 3 lines of code in the if (mD2DDevice) { ... } block. Bas, any ideas?
Flags: needinfo?(bjacob) → needinfo?(bas)
I'm told that Bas is currently on vacation which is also going to be my case starting tomorrow. We might need a less-than-optimal short-term fix for this. Is this crash entirely specific to one driver version such as 310.70 ? If yes, we can simply blacklist D2D on it.
Flags: needinfo?(bas)
(In reply to Benoit Jacob [:bjacob] from comment #9) > Is this crash entirely specific to one driver version such as 310.70 ? It is only with this version from all we know, see also the correlations listed in this bug.
OK. Is this Windows-7 only or does it also affect other Windows versions?
Hm, reports look windows 7-only, however, Direct2D is specific to Windows Vista/7/8 and 1) there are few Vista users with recent drivers and 2) there are very few Win8 users, so we can't necessarily conclude much (unless you have a really large volume of crash reports, like over 1,000, and none of them is outside of Win7). Filing block request for all Windows versions.
Filed bug 823966 to request a downloaded blocklist entry.
Blocks: 605749
People updating drivers while keeping applications that use them through indirect API's deserve the browser to crash when the device is suddenly lost. There is no crash what so ever once the driver is installed so I'm calling it user error.
For anyone else KB2488113 applies
Just to copy from the related 310.90 thread this issue is occuring for a handful of people who have submitted the bug multiple times, the majority of the list is duplicates from the same users. There is more to this crash then a simple bug in the nvidia drivers, which i cannot reproduce under .70 or .90. Most of the users on that list ARE infact using the version of the Direct3D 10.1 binary from the above KB article. There is no hardware correlation, some are using, one is using a 210, another a 480, another a 650, 670 etc. The last two affected cards would put mine in the affected list. Now everyone there is claiming the crash happens directly at startup. Here i am running multiple instances of Firefox 18 - 20, and all indicate under about:support that Direct2D and Write are enabled and working. It would be negligent to block list the driver at this point. there is not enough individial sample data at this point.
And just to point out, the crash report count does NOT increase as you go further back, it is the same 485 reports (mostly dupes) whether you go 2 or 4 weeks. At this point, I'm more inclined to point the finger at one of hotfix tuesday's 14 patches.......
Had a reply from someone on the evga forums experiencing this, i have asked him to roll back KB2778930 - Vulnerability in Windows kernel-mode driver and provide feedback on if this resolves the issue. If so, Nvidia will have to diagnose the issue or work with Microsoft to improve their broken hotfix.
one of the comments reads "It's been crashing for the passed week. I had one tab open I think it was the steam website. And I clicked restart pc because it had to update but i didn't close the firefox window it just closed it for me when I hit restart. I've reinstalled firefox and restarted my pc. This message still pops up." That makes me wonder about those hotfixes all the more. However, I've got 2 users with fully up to date windows installs, one using 310.70(9600GT), the other 310.90(GTX 480) not having any crashing with Firefox 17 through 21. Theres the chance this is a case of machines suddenly becoming corrupted by the hotfixes, a number of users have already reported being completely unable to boot with the new patches.
(In reply to Danial Horton from comment #16) > Just to copy from the related 310.90 thread > > this issue is occuring for a handful of people who have submitted the bug > multiple times, the majority of the list is duplicates from the same users. At least bug 828911 is happening for many users, otherwise it wouldn't have become the #15 topcrasher. Since bug 828911 alone is enough to warrant the blocklisting, discussion in the present bug is moot.
There are only 11 crashes in 18.0.
(In reply to Scoobidiver from comment #21) > There are only 11 crashes in 18.0. Yes, pretty low volume in both 18 and 19.0b1, now.
We still see these crashes though at very low volume, 2 reports against 46.0.1 currently. This is 100% isolated to NVIDIA 310.70 which was current when this bug was filed but is several versions out of date now. If it'll help we should probably just block these drivers (assuming it's not already). Otherwise I don't see there's much more we can do.
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → INCOMPLETE
Whiteboard: [startupcrash] → [startupcrash][gfx-noted]
You need to log in before you can comment on or make changes to this bug.