Closed Bug 1405110 Opened 7 years ago Closed 7 years ago

Crash in nvumdshim.dll | igd10umd32.dll | nvumdshim.dll | CResource<T>::CLS::FinalConstruct

Categories

(Core :: Audio/Video: Playback, defect, P2)

57 Branch
x86_64
Windows 7
defect

Tracking

()

RESOLVED FIXED
mozilla58
Tracking Status
firefox-esr52 --- unaffected
firefox56 --- fixed
firefox57 --- fixed
firefox58 --- fixed

People

(Reporter: philipp, Assigned: jya, NeedInfo)

References

Details

(Keywords: crash, regression)

Crash Data

Attachments

(2 files)

This bug was filed from the Socorro interface and is 
report bp-300b9f6c-a954-4d5e-b97e-1e6d20171002.
=============================================================
Crashing Thread (105), Name: MediaPDecoder #3
Frame 	Module 	Signature 	Source
Ø 0 	nvumdshim.dll 	nvumdshim.dll@0x28cfd 	
Ø 1 	nvumdshim.dll 	nvumdshim.dll@0x28bea 	
Ø 2 	nvumdshim.dll 	nvumdshim.dll@0x3f3e4 	
Ø 3 	nvumdshim.dll 	nvumdshim.dll@0x2ca5d 	
Ø 4 	igd10umd32.dll 	igd10umd32.dll@0x912fe1 	
Ø 5 	igd10umd32.dll 	igd10umd32.dll@0x4ce4 	
Ø 6 	igd10umd32.dll 	igd10umd32.dll@0x926218 	
Ø 7 	nvumdshim.dll 	nvumdshim.dll@0x42bd0 	
Ø 8 	nvumdshim.dll 	nvumdshim.dll@0x4285e 	
Ø 9 	nvumdshim.dll 	nvumdshim.dll@0x42fbc 	
Ø 10 	nvumdshim.dll 	nvumdshim.dll@0x2e286 	
11 	d3d11.dll 	CResource<ID3D11Texture3D>::CLS::FinalConstruct(CContext*, D3D11DDIARG_CREATERESOURCE const*, SD3D11SharedResourceCreationArgs*, SD3D11CrossLayerData*, D3D10DDI_HRTRESOURCE) 	
12 	d3d11.dll 	TCLSWrappers<CTexture2D>::CLSFinalConstructFn(CTexture2D::CLS*, CContext*, CTexture2D::TConstructorArgs const*) 	
13 	d3d11.dll 	CLayeredObjectWithCLS<CTexture2D>::FinalConstruct(CTexture2D::TConstructorArgs const&, _GUID const&, void**, CLayeredObjectWithCLS<CTexture2D>::SInfo const*) 	
14 	d3d11.dll 	CLayeredObjectWithCLS<CTexture2D>::CreateInstance(CTexture2D::TConstructorArgs&, void*, void*, _GUID const&, void**, CLayeredObjectWithCLS<CTexture2D>::SInfo const*) 	
15 	d3d11.dll 	CDevice::CreateLayeredChild(unsigned int, void const*, unsigned long, ID3D11LayeredUseCounted*, _GUID const&, void**) 	
16 	d3d11.dll 	CBridgeImpl<ID3D11LayeredDevice, ID3D11LayeredDevice, CLayeredObject<CDevice> >::CreateLayeredChild(unsigned int, void const*, unsigned long, ID3D11LayeredUseCounted*, _GUID const&, void**) 	
17 	d3d11.dll 	CD3D11LayeredChild<ID3D11DeviceChild, NDXGI::CDevice, 64>::FinalConstruct(ED3D11DeviceChildType, SLayeredArgs const*, unsigned long, ID3D11LayeredUseCounted*, _GUID const&) 	
18 	d3d11.dll 	NDXGI::CResource::FinalConstruct(NDXGI::CResource::TConstructorArgs const&) 	
19 	d3d11.dll 	NDXGI::CDevice::CreateLayeredChild(unsigned int, void const*, unsigned long, ID3D11LayeredUseCounted*, _GUID const&, void**) 	
20 	d3d11.dll 	CBridgeImpl<ID3D11LayeredDevice, ID3D11LayeredDevice, CLayeredObject<NDXGI::CDevice> >::CreateLayeredChild(unsigned int, void const*, unsigned long, ID3D11LayeredUseCounted*, _GUID const&, void**) 	
21 	d3d11.dll 	NOutermost::CDeviceChild::FinalConstruct(NOutermost::CDeviceChild::TConstructorArgs const&) 	
22 	d3d11.dll 	CUseCountedObject<NOutermost::CDeviceChild>::CUseCountedObject<NOutermost::CDeviceChild>(void*, NOutermost::CDeviceChild::TConstructorArgs const&, NOutermost::CDevice*, _GUID const&, void**) 	
23 	d3d11.dll 	CUseCountedObject<NOutermost::CDeviceChild>::CreateInstance(NOutermost::CDeviceChild::TConstructorArgs const&, NOutermost::CDevice*, void*, void*, _GUID const&, void**) 	
24 	d3d11.dll 	NOutermost::CDevice::CreateLayeredChild(unsigned int, void const*, unsigned long, ID3D11LayeredUseCounted*, _GUID const&, void**) 	
25 	d3d11.dll 	CDevice::CreateAndRecreateLayeredChild<SD3D11LayeredTexture2DCreationArgs>(unsigned int, SD3D11LayeredTexture2DCreationArgs*, ID3D11LayeredUseCounted*, _GUID const&, void**, bool) 	
26 	d3d11.dll 	CDevice::CreateTexture2D_Worker(D3D11_TEXTURE2D_DESC const*, D3D11_SUBRESOURCE_DATA const*, int, ID3D11Texture2D**, SD3D11SharedResourceCreationArgs*, bool) 	
27 	d3d11.dll 	CDevice::CreateTexture2D(D3D11_TEXTURE2D_DESC const*, D3D11_SUBRESOURCE_DATA const*, ID3D11Texture2D**) 	
28 	xul.dll 	mozilla::layers::D3D11YCbCrRecycleAllocator::Allocate(mozilla::gfx::SurfaceFormat, mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits>, mozilla::layers::BackendSelector, mozilla::layers::TextureFlags, mozilla::layers::TextureAllocationFlags) 	gfx/layers/D3D11YCbCrImage.cpp:296
29 	xul.dll 	mozilla::layers::DefaultTextureClientAllocationHelper::Allocate(mozilla::layers::KnowsCompositor*) 	gfx/layers/client/TextureClientRecycleAllocator.cpp:77
30 	xul.dll 	mozilla::layers::TextureClientRecycleAllocator::CreateOrRecycle(mozilla::layers::ITextureClientAllocationHelper&) 	gfx/layers/client/TextureClientRecycleAllocator.cpp:200
31 	xul.dll 	mozilla::layers::TextureClientRecycleAllocator::CreateOrRecycle(mozilla::gfx::SurfaceFormat, mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits>, mozilla::layers::BackendSelector, mozilla::layers::TextureFlags, mozilla::layers::TextureAllocationFlags) 	gfx/layers/client/TextureClientRecycleAllocator.cpp:165
32 	xul.dll 	mozilla::layers::D3D11YCbCrImage::SetData(mozilla::layers::KnowsCompositor*, mozilla::layers::ImageContainer*, mozilla::layers::PlanarYCbCrData const&) 	gfx/layers/D3D11YCbCrImage.cpp:43
33 	xul.dll 	mozilla::VideoData::CreateAndCopyData(mozilla::VideoInfo const&, mozilla::layers::ImageContainer*, __int64, mozilla::media::TimeUnit const&, mozilla::media::TimeUnit const&, mozilla::VideoData::YCbCrBuffer const&, bool, mozilla::media::TimeUnit const&, mozilla::gfx::IntRectTyped<mozilla::gfx::UnknownUnits> const&, mozilla::layers::KnowsCompositor*) 	dom/media/MediaData.cpp:328
34 	xul.dll 	mozilla::FFmpegVideoDecoder<46465650>::DoDecode(mozilla::MediaRawData*, unsigned char*, int, bool*, nsTArray<RefPtr<mozilla::MediaData> >&) 	dom/media/platforms/ffmpeg/FFmpegVideoDecoder.cpp:346
35 	xul.dll 	mozilla::FFmpegVideoDecoder<46465650>::DoDecode(mozilla::MediaRawData*, bool*, nsTArray<RefPtr<mozilla::MediaData> >&) 	dom/media/platforms/ffmpeg/FFmpegVideoDecoder.cpp:214
36 	xul.dll 	mozilla::FFmpegVideoDecoder<46465650>::ProcessDecode(mozilla::MediaRawData*) 	dom/media/platforms/ffmpeg/FFmpegVideoDecoder.cpp:180
37 	xul.dll 	mozilla::detail::ProxyRunnable<mozilla::MozPromise<nsTArray<RefPtr<mozilla::MediaData> >, mozilla::MediaResult, 1>, RefPtr<mozilla::MozPromise<nsTArray<RefPtr<mozilla::MediaData> >, mozilla::MediaResult, 1> > ( mozilla::VorbisDataDecoder::*)(mozilla::MediaRawData*), mozilla::VorbisDataDecoder, mozilla::MediaRawData*>::Run() 	obj-firefox/dist/include/mozilla/MozPromise.h:1394
38 	xul.dll 	mozilla::TaskQueue::Runner::Run() 	xpcom/threads/TaskQueue.cpp:246
39 	xul.dll 	nsThreadPool::Run() 	xpcom/threads/nsThreadPool.cpp:225
40 	xul.dll 	nsThread::ProcessNextEvent(bool, bool*) 	xpcom/threads/nsThread.cpp:1446
41 	xul.dll 	NS_ProcessNextEvent(nsIThread*, bool) 	xpcom/threads/nsThreadUtils.cpp:480
42 	xul.dll 	mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp:339
43 	xul.dll 	MessageLoop::RunHandler() 	ipc/chromium/src/base/message_loop.cc:319
44 	xul.dll 	MessageLoop::Run() 	ipc/chromium/src/base/message_loop.cc:299
45 	xul.dll 	nsThread::ThreadFunc(void*) 	xpcom/threads/nsThread.cpp:506
46 	nss3.dll 	_PR_NativeRunThread 	nsprpub/pr/src/threads/combined/pruthr.c:397
47 	nss3.dll 	pr_root 	nsprpub/pr/src/md/windows/w95thred.c:95
48 	ucrtbase.dll 	_o__CIpow 	
49 	kernel32.dll 	BaseThreadInitThunk 	
50 	mozglue.dll 	patched_BaseThreadInitThunk 	mozglue/build/WindowsDllBlocklist.cpp:824
51 	ntdll.dll 	__RtlUserThreadStart 	
52 	ntdll.dll 	_RtlUserThreadStart

crashes with this signature are spiking up in volume in 56.0. user comments often refer to playing some video on youtube for a while.

correlations of affected systems look very similar to those in bug 1404456 (windows 7, intel sandybridge gpu in use) but this time in a dual gpu-setup with nvidia...

Adapter device id facet
1 	0x0116 	915 	43.28 %
2 	0x0126 	604 	28.57 %
3 	0x0166 	576 	27.25 %
4 	0x0106 	19 	0.90 %

Adapter driver version facet
1 	9.17.10.2843 	509 	24.08 %
2 	8.15.10.2361 	491 	23.23 %
3 	9.17.10.4101 	277 	13.10 %
4 	9.17.10.3517 	158 	7.47 %
5 	8.15.10.2725 	133 	6.29 %
6 	8.15.10.2696 	120 	5.68 %
7 	9.17.10.3347 	88 	4.16 %
8 	8.15.10.2778 	71 	3.36 %
9 	9.17.10.3372 	49 	2.32 %
10 	9.17.10.3040 	48 	2.27 %

the stack goes through a codepath that changed in bug 1223270 - if that were the regressor, it would fit perfectly with the regression range dug up for bug 1404456...
Flags: needinfo?(jyavenard)
nvumdshim.dll is a nvidia driver stuff.

so how could it be the same as intel sandybridge gpu in use?
Flags: needinfo?(jyavenard)
Could one of you please ping your Intel contacts and ask them to look at this?
Flags: needinfo?(milan)
Flags: needinfo?(jyavenard)
[@ nvumdshimx.dll | igd10umd64.dll | nvumdshimx.dll | igd10umd64.dll | RtlAllocateHeap | nvumdshimx.dll | stdext::_Hash<T>::insert ] looks like the 64bit variant of this crash.
Crash Signature: [@ nvumdshim.dll | igd10umd32.dll | nvumdshim.dll | CResource<T>::CLS::FinalConstruct] → [@ nvumdshim.dll | igd10umd32.dll | nvumdshim.dll | CResource<T>::CLS::FinalConstruct] [@ nvumdshimx.dll | igd10umd64.dll | nvumdshimx.dll | igd10umd64.dll | RtlAllocateHeap | nvumdshimx.dll | stdext::_Hash<T>::insert ]
Hardware: x86 → All
Optimus mixes the drivers, so I'd expect to see Intel driver in the Nvidia GPU path.  I guess the other way is possible as well, as it seems to be what we're seeing here.
Flags: needinfo?(milan)
Crash Signature: [@ nvumdshim.dll | igd10umd32.dll | nvumdshim.dll | CResource<T>::CLS::FinalConstruct] [@ nvumdshimx.dll | igd10umd64.dll | nvumdshimx.dll | igd10umd64.dll | RtlAllocateHeap | nvumdshimx.dll | stdext::_Hash<T>::insert ] → [@ nvumdshim.dll | igd10umd32.dll | nvumdshim.dll | CResource<T>::CLS::FinalConstruct] [@ nvumdshim.dll | igd10iumd32.dll | nvumdshim.dll | CResource<T>::CLS::FinalConstruct] [@ nvumdshimx.dll | igd10umd64.dll | nvumdshimx.dll | igd10umd64.dll | RtlAllo…
is it possible that bug 1223270 caused/surfaced this crash spike?
Flags: needinfo?(nical.bugzilla)
Flags: needinfo?(matt.woodrow)
I would be easy to test if that's the cause, by disabling direct upload...

However, if that does fix the crashes, this only indicates that we have a bug somewhere in our surface allocation/recycling.

It would be better to fix *that* problem.

Joe, is there any knowledge of crashes with similar stack trace?
We also have bug 1405838, bug 1404456 (and previous bug 1207993) all showing igd10umd32.dll crashing
Flags: needinfo?(jyavenard) → needinfo?(joseph.k.olivas)
See Also: → 1405838, 1207993
(In reply to [:philipp] from comment #6)
> is it possible that bug 1223270 caused/surfaced this crash spike?

That does seem plausible, but as jya said, we should try fix the underlying issue.

This looks like a crash in the parent process, so is this non-e10s?

It might be worth disabling direct upload in the parent process, and possibly even only allow it within the GPU process so that crashes as a result of it don't bring the whole browser down.
Flags: needinfo?(matt.woodrow)
Flags: needinfo?(nical.bugzilla)
(In reply to Matt Woodrow (:mattwoodrow) from comment #8)
> This looks like a crash in the parent process, so is this non-e10s?
> 
> It might be worth disabling direct upload in the parent process, and
> possibly even only allow it within the GPU process so that crashes as a
> result of it don't bring the whole browser down.

on 56.0 it's happening in the content process 60% of the time vs 40% in the parent process. so it's not a non-e10s crash only...
Comment on attachment 8916555 [details]
Bug 1405110 - P2. Never attempts to upload to D3D11 surface in parent process.

https://reviewboard.mozilla.org/r/187700/#review192920

::: commit-message-068d6:3
(Diff revision 1)
> +Bug 1405110 - P2. Never attempts to upload to D3D11 surface in parent process. r?mattwoodrow
> +
> +Accessing the graphic device driver from the parent process is more likely to cause a driver crash.

It's not really more likely to crash (as far as we know), just that a crash in the parent process is a much bigger problem (whole browser dies) than a crash in a tab process.
Attachment #8916555 - Flags: review?(matt.woodrow) → review+
Comment on attachment 8916554 [details]
Bug 1405110 - P1. Lock device when allocating D3D11 surface.

https://reviewboard.mozilla.org/r/187698/#review192922
Attachment #8916554 - Flags: review?(matt.woodrow) → review+
Pushed by jyavenard@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/3a3981033259
P1. Lock device when allocating D3D11 surface. r=mattwoodrow
https://hg.mozilla.org/integration/autoland/rev/97b7a8c359db
P2. Never attempts to upload to D3D11 surface in parent process. r=mattwoodrow
Assignee: nobody → jyavenard
Crash Signature: [@ nvumdshim.dll | igd10umd32.dll | nvumdshim.dll | CResource<T>::CLS::FinalConstruct] [@ nvumdshim.dll | igd10iumd32.dll | nvumdshim.dll | CResource<T>::CLS::FinalConstruct] [@ nvumdshimx.dll | igd10umd64.dll | nvumdshimx.dll | igd10umd64.dll | RtlAllo… → [@ igd10umd64.dll | AlignedHeapFree16] [@ nvumdshim.dll | igd10umd32.dll | nvumdshim.dll | CResource<T>::CLS::FinalConstruct] [@ nvumdshim.dll | igd10iumd32.dll | nvumdshim.dll | CResource<T>::CLS::FinalConstruct] [@ nvumdshimx.dll | igd10umd64.dll | n…
Hardware: All → x86_64
would this fix be upliftable to 57?
Flags: needinfo?(jyavenard)
(In reply to [:philipp] from comment #17)
> would this fix be upliftable to 57?

yes easily, it should apply as-is.

however, has it shown to have any effects whatsoever on those crashes?
Flags: needinfo?(jyavenard)
the nvidia/intel-dual gpu signatures aren't happening at all on nightly recently (no users with this hardware?), so it won't be possible to say unfortunately.
Well, the change can't hurt anyway..
Comment on attachment 8916554 [details]
Bug 1405110 - P1. Lock device when allocating D3D11 surface.

Approval Request Comment
[Feature/Bug causing the regression]: unknown, likely always there. just more exposed with bug 1223270
[User impact if declined]: maybe crashes
[Is this code covered by automated tests?]: no
[Has the fix been verified in Nightly?]: no
[Needs manual test from QE? If yes, steps to reproduce]:  will verify with crash reports
[List of other uplifts needed for the feature/fix]: both patches to uplift
[Is the change risky?]: low risk. 
[Why is the change risky/not risky?]: we lock the device when attempting to create a D3D11 surface as is done elsewhere in the code (but not everywhere). It is still a shot in the dark however.
[String changes made/needed]: none
Attachment #8916554 - Flags: approval-mozilla-beta?
Comment on attachment 8916554 [details]
Bug 1405110 - P1. Lock device when allocating D3D11 surface.

Crash fix that seems to have worked in Nightly, Beta57+
Attachment #8916554 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
None of the signature shows up but https://crash-stats.mozilla.com/signature/?signature=igd10umd64.dll%20%7C%20AlignedHeapFree16

this one is still present :(

Maybe it's too early for the others...
of the remaining signature:
https://crash-stats.mozilla.com/report/index/d0e746a8-113c-4dea-93ff-d566e0171016#allthreads

That's a crash in D3D1 in gfxFont.. Why are we using that?
above I meant D2D1
Blocks: 1409141
jya, I'm putting together a dot release for 56 and think this would be good to include. It seems to have fixed a lot of the crashes in beta. Can you request uplift to m-r if you think that's a good idea? Thanks!
Flags: needinfo?(jyavenard)
For win7, I think bug 1409141 is all that was needed (it makes this fix redundant as it's just never used).
However, it could likely help with other platforms, however, there's been no crash on anything but win7

If ESR52 is affected it has nothing to do with the various fixes pushed. ESR52 doesn't have the regressing code.
Flags: needinfo?(jyavenard)
But it looks like we uplifted code from both those bugs to 57. You're sure we only need the fix in 1409141?
Flags: needinfo?(jyavenard)
yes.

We did uplift both as we were hoping to not have to disable things 100% for everyone.

But we did and that's what 1409141 is about.

we don't need this bug if 1409141 is in
Flags: needinfo?(jyavenard)
Calling this 56:fixed by bug 1409141.
You need to log in before you can comment on or make changes to this bug.