1074378 - crash in D3D10::CResourceCopy::ResourceCopyRegionGPU(D3D10::CResource*, unsigned long, unsigned long, unsigned long, unsigned long, D3D10::CResource*, unsigned long, D3D10_DDI_BOX const*)

Status: RESOLVED FIXED

(Core :: Graphics: Layers, defect)

Description

[Tracy Walker [:tracy]]

This bug was filed from the Socorro interface and is 
report bp-8e695700-a909-4a48-be92-d04702140928.
=============================================================


Frame 	Module 	Signature 	Source
0 	igd10umd32.dll 	D3D10::CResourceCopy::ResourceCopyRegionGPU(D3D10::CResource*, unsigned long, unsigned long, unsigned long, unsigned long, D3D10::CResource*, unsigned long, D3D10_DDI_BOX const*) 	
1 	igd10umd32.dll 	D3D10::CDevice::ResourceUpdateSubResourceUP(D3D10::CResource*, unsigned int, D3D10_DDI_BOX const*, void const*, unsigned int, unsigned int, int) 	
2 	igd10umd32.dll 	D3D10API::ResourceUpdateSubResourceUP10(D3D10DDI_HDEVICE, D3D10DDI_HRESOURCE, unsigned int, D3D10_DDI_BOX const*, void const*, unsigned int, unsigned int) 	
3 	d3d11.dll 	CContext::ID3D11DeviceContext1_UpdateSubresource_<0>(ID3D11DeviceContext1*, ID3D11Resource*, unsigned int, D3D11_BOX const*, void const*, unsigned int, unsigned int) 	
4 	xul.dll 	mozilla::layers::DataTextureSourceD3D11::Update(mozilla::gfx::DataSourceSurface*, nsIntRegion*, mozilla::gfx::IntPointTyped<mozilla::gfx::UnknownUnits>*) 	gfx/layers/d3d11/TextureD3D11.cpp
5 	xul.dll 	mozilla::layers::BufferTextureHost::Upload(nsIntRegion*) 	gfx/layers/composite/TextureHost.cpp
6 	xul.dll 	mozilla::layers::BufferTextureHost::Lock() 	gfx/layers/composite/TextureHost.cpp
7 	xul.dll 	mozilla::layers::ContentHostTexture::Lock() 	gfx/layers/composite/ContentHost.h
8 	xul.dll 	mozilla::layers::ContentHostBase::Composite(mozilla::layers::EffectChain&, float, mozilla::gfx::Matrix4x4 const&, mozilla::gfx::Filter const&, mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits> const&, nsIntRegion const*) 	gfx/layers/composite/ContentHost.cpp
9 	xul.dll 	mozilla::layers::ThebesLayerComposite::RenderLayer(nsIntRect const&) 	gfx/layers/composite/ThebesLayerComposite.cpp
10 	xul.dll 	mozilla::layers::RenderLayers<mozilla::layers::ContainerLayerComposite>(mozilla::layers::ContainerLayerComposite*, mozilla::layers::LayerManagerComposite*, mozilla::gfx::IntRectTyped<mozilla::RenderTargetPixel> const&) 	gfx/layers/composite/ContainerLayerComposite.cpp
11 	xul.dll 	mozilla::layers::ContainerRender<mozilla::layers::ContainerLayerComposite>(mozilla::layers::ContainerLayerComposite*, mozilla::layers::LayerManagerComposite*, nsIntRect const&) 	gfx/layers/composite/ContainerLayerComposite.cpp
12 	xul.dll 	mozilla::layers::ContainerLayerComposite::RenderLayer(nsIntRect const&) 	gfx/layers/composite/ContainerLayerComposite.cpp
13 	xul.dll 	mozilla::layers::LayerManagerComposite::Render() 	gfx/layers/composite/LayerManagerComposite.cpp
14 	xul.dll 	mozilla::layers::LayerManagerComposite::EndTransaction(void (*)(mozilla::layers::ThebesLayer*, gfxContext*, nsIntRegion const&, mozilla::layers::DrawRegionClip, nsIntRegion const&, void*), void*, mozilla::layers::LayerManager::EndTransactionFlags) 	gfx/layers/composite/LayerManagerComposite.cpp
15 	xul.dll 	mozilla::layers::LayerManagerComposite::EndEmptyTransaction(mozilla::layers::LayerManager::EndTransactionFlags) 	gfx/layers/composite/LayerManagerComposite.cpp
16 	xul.dll 	mozilla::layers::CompositorParent::CompositeToTarget(mozilla::gfx::DrawTarget*, nsIntRect const*) 	gfx/layers/ipc/CompositorParent.cpp
17 	xul.dll 	mozilla::layers::CompositorParent::CompositeCallback() 	gfx/layers/ipc/CompositorParent.cpp
18 	xul.dll 	MessageLoop::RunTask(Task*) 	ipc/chromium/src/base/message_loop.cc
19 	xul.dll 	MessageLoop::DeferOrRunPendingTask(MessageLoop::PendingTask const&) 	ipc/chromium/src/base/message_loop.cc
20 	xul.dll 	MessageLoop::DoWork() 	ipc/chromium/src/base/message_loop.cc
21 	xul.dll 	base::MessagePumpForUI::DoRunLoop() 	ipc/chromium/src/base/message_pump_win.cc
22 	xul.dll 	base::MessagePumpWin::RunWithDispatcher(base::MessagePump::Delegate*, base::MessagePumpWin::Dispatcher*) 	ipc/chromium/src/base/message_pump_win.cc
23 	xul.dll 	base::MessagePumpWin::Run(base::MessagePump::Delegate*) 	ipc/chromium/src/base/message_pump_win.h
24 	xul.dll 	MessageLoop::RunHandler() 	ipc/chromium/src/base/message_loop.cc
25 	xul.dll 	MessageLoop::Run() 	ipc/chromium/src/base/message_loop.cc
26 	xul.dll 	base::Thread::ThreadMain() 	ipc/chromium/src/base/thread.cc
27 	xul.dll 	`anonymous namespace'::ThreadFunc(void*) 	tools/profiler/platform-win32.cc
28 	kernel32.dll 	BaseThreadInitThunk 	
29 	ntdll.dll 	__RtlUserThreadStart 	
30 	ntdll.dll 	_RtlUserThreadStart

Comment 1

[Benjamin Smedberg]

Is this specific to D3D11?

Crash is in the D3D APIs under mozilla::layers::DataTextureSourceD3D11::Update. The crash is near-null at 0x30. dmajor can you look and see if you can decipher what the locals we pass at http://hg.mozilla.org/releases/mozilla-aurora/annotate/4e0bde619641/gfx/layers/d3d11/TextureD3D11.cpp#l497 are? In particular if mTexture is null, or the size arguments are really wacky.

Comment 2

[(Away)]

mCompositor->GetDC()->UpdateSubresource(mTexture, 0, &box, data, map.mStride, map.mStride * mSize.height); 

0bd9f4d0  58404515 Return address: DataTextureSourceD3D11::Update+0x2af
0bd9f4d4  04582824 DC this ptr
0bd9f4d8  290593d0 mTexture
0bd9f4dc  00000000 0
0bd9f4e0  0bd9f55c &box
0bd9f4e4  2dc0ab0c data
0bd9f4e8  00001558 map.mStride
0bd9f4ec  003d5d00 map.mStride * mSize.height

Comment 3

[(Away)]

> Is this specific to D3D11?

All of the 105 aurora crashes last month are D3D11.
If you include beta (which are 10x as many) it's about 94%.

Comment 4

[Benjamin Smedberg]

Hrm, that data doesn't look obviously bad.

All of these are with intel graphics. And *all* of them have adapter driver version == 8.15.10.1749

https://crash-stats.mozilla.com/search/?signature=%3DD3D10%3A%3ACResourceCopy%3A%3AResourceCopyRegionGPU%28D3D10%3A%3ACResource*%2C+unsigned+long%2C+unsigned+long%2C+unsigned+long%2C+unsigned+long%2C+D3D10%3A%3ACResource*%2C+unsigned+long%2C+D3D10_DDI_BOX+const*%29&_facets=adapter_vendor_id&_facets=adapter_device_id&_facets=adapter_driver_version&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform

nical, do you know whether this is a known-old driver version that we could be blocklisting, or how I'd figure that out?

Comment 5

[Nicolas Silva [:nical]]

I think that the intel driver 8.15.10.1749 was released around mid 2009 so it sounds old enough that we should blacklist it if it's consistently causing issues.

Comment 6

[Nicolas Silva [:nical]]

Attachment: Block driver version 8,15,10,1749

Comment 7

[Bas Schouten (:bas.schouten)]

Comment on attachment 8498306 [details] [diff] [review]
Block driver version 8,15,10,1749

Review of attachment 8498306 [details] [diff] [review]:
-----------------------------------------------------------------

::: widget/windows/GfxInfo.cpp
@@ +963,5 @@
>      IMPLEMENT_INTEL_DRIVER_BLOCKLIST(DRIVER_OS_WINDOWS_7, IntelGMA950,   V(8,15,10,1930));
>      IMPLEMENT_INTEL_DRIVER_BLOCKLIST(DRIVER_OS_WINDOWS_7, IntelGMA3150,  V(8,14,10,1972));
>      IMPLEMENT_INTEL_DRIVER_BLOCKLIST(DRIVER_OS_WINDOWS_7, IntelGMAX3000, V(7,15,10,1666));
>      IMPLEMENT_INTEL_DRIVER_BLOCKLIST(DRIVER_OS_WINDOWS_7, IntelGMAX4500HD, V(7,15,10,1666));
> +    IMPLEMENT_INTEL_DRIVER_BLOCKLIST(DRIVER_OS_WINDOWS_7, IntelGMAX4500HD, V(8,15,10,1749)); // Bug 1074378

This new blocklist entry makes the one above here pointless. I'm also unsure whether we want to blacklist D3D9 on these devices. I suspect we only want to blacklist D3D11 layers. (in addition to D2D which is already blacklisted there)

Comment 8

[Nicolas Silva [:nical]]

Attachment: v2 (only blocks D3D11 layers for this driver version specifically)

Comment 9

[Bas Schouten (:bas.schouten)]

Comment on attachment 8498774 [details] [diff] [review]
v2 (only blocks D3D11 layers for this driver version specifically)

Review of attachment 8498774 [details] [diff] [review]:
-----------------------------------------------------------------

::: widget/windows/GfxInfo.cpp
@@ +979,5 @@
> +    APPEND_TO_DRIVER_BLOCKLIST(DRIVER_OS_ALL,
> +      (nsAString&) GfxDriverInfo::GetDeviceVendor(VendorIntel),
> +      (GfxDeviceFamily*) GfxDriverInfo::GetDeviceFamily(IntelGMAX4500HD),
> +      nsIGfxInfo::FEATURE_DIRECT3D_11_LAYERS, nsIGfxInfo::FEATURE_BLOCKED_DRIVER_VERSION,
> +      DRIVER_EQUAL, V(8,15,10,1749), "8.15.10.1749");

Should we make this LESS_THAN_OR_EQUAL? Well, for now let's do EQUAL.

Comment 10

[Nicolas Silva [:nical]]

(In reply to Bas Schouten (:bas.schouten) from comment #9)
> Should we make this LESS_THAN_OR_EQUAL? Well, for now let's do EQUAL.

As Benjamin said in comment 4, all of the crashes have the driver version 8.15.10.1749, so I think EQUAL is what we want.

Comment 11

[Nicolas Silva [:nical]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/13ba124946bd

Comment 12

[Nicolas Silva [:nical]]

Comment on attachment 8498774 [details] [diff] [review]
v2 (only blocks D3D11 layers for this driver version specifically)

Approval Request Comment
[Feature/regressing bug #]:
[User impact if declined]: intel users with a specific (and oldish) driver experience crashes.
[Describe test coverage new/current, TBPL]:
[Risks and why]: low, just blocklists a nasty driver version
[String/UUID change made/needed]:

Comment 13

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/13ba124946bd

Comment 14

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/6c46c21a04f9
https://hg.mozilla.org/releases/mozilla-beta/rev/bbc35ec2c90e

Comment 15

[Ryan VanderMeulen [:RyanVM]]

Backed out from 33 at Sylvestre's request.
https://hg.mozilla.org/releases/mozilla-release/rev/7683a98b0400

Comment 16

[Sylvestre Ledru [:Sylvestre]]

For some context, Kairo noticed that the number of crashes increased with RC. Since we are going to build a new 33.0, we backout the three patches (bug 1074378, bug 1076825 and bug 1044975) to go back to the beta 9 levels (we were happy with them). We are unsure which one causes the increase.
https://crash-analysis.mozilla.com/rkaiser/2014-10-09/2014-10-09.firefox.33.explosiveness.html

Leaving the status flag as affected, we could take it back for 33.1 once we have more information (with 34 in beta)

Comment 17

[Robert Kaiser]

See bug 1081171 on new the crashes we saw in RC.

Comment 18

[Robert Kaiser]

As bug 1081171 comment #15 and 16 explain, we should try backing this out on 34 Beta as well to see if it's the actual source of those crashes. If it is, we at least have that data point once the gfx team can work on beta issues.

Comment 19

[Lawrence Mandel [:lmandel] (use needinfo)]

Ryan - Can you please assist with a backout on beta?
Kairo/Bas - Do you want to pull this patch off all all branches?

Note that beta3 has already gtb so this backout will apply to beta4.

Comment 20

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/3e2c92836231

Comment 21

[Robert Kaiser]

(In reply to Lawrence Mandel [:lmandel] from comment #19)
> Ryan - Can you please assist with a backout on beta?
> Kairo/Bas - Do you want to pull this patch off all all branches?

We don't see too much volume of bug 1081171 on Nightly or Aurora, but there's some. If we are sure this is the cause of them (which we try to find out with the backout), then I think backing out everywhere is reasonable and then getting a proper fix for this bug here in once we have that.

> Note that beta3 has already gtb so this backout will apply to beta4.

It's too bad that we need to wait almost a week for results on this (b4 only goes out during the day on Tuesday, so the results from Wednesday which we get to see on Thursday are the first to show any data on this), but it's better than to wait even longer until a proper fix can be figured out.

Comment 22

[Lawrence Mandel [:lmandel] (use needinfo)]

(In reply to Robert Kaiser (:[e-mail]) from comment #21)
> It's too bad that we need to wait almost a week for results on this (b4 only
> goes out during the day on Tuesday, so the results from Wednesday which we
> get to see on Thursday are the first to show any data on this), but it's
> better than to wait even longer until a proper fix can be figured out.

Agreed but we will still have time to respond if this is a high enough priority for the gfx team during the 34 beta cycle.

Comment 23

[Liz Henry (:lizzard) (relman/hg->git project)]

Thanks. Here's hoping it brings down the crash rate later next week!

Comment 24

[Robert Kaiser]

I think we should back this out everywhere as we are seeing the bug 1081171 crashes on 35 and 36 still. And then we should see if we can get an actually proper fix in place.

Comment 25

[Nicolas Silva [:nical]]

(In reply to Robert Kaiser (:[e-mail]) from comment #24)
> I think we should back this out everywhere as we are seeing the bug 1081171
> crashes on 35 and 36 still. And then we should see if we can get an actually
> proper fix in place.

Yes, this should get backed out.

Comment 26

[Robert Kaiser]

Ryan, can you back this out on Aurora and Nightly as well please?

Once that's done, we can mark bug 1081171 fixed or otherwise resolved and hopefully work on a better fix for the original issue here.

Comment 27

[Ryan VanderMeulen [:RyanVM]]

Going out on a limb that 33/34 are wontfix at this point.

https://hg.mozilla.org/integration/mozilla-inbound/rev/015dfc3cfab2
https://hg.mozilla.org/releases/mozilla-aurora/rev/8d985f1fa188

Comment 28

[Ryan VanderMeulen [:RyanVM]]

Merge of backout:
https://hg.mozilla.org/mozilla-central/rev/015dfc3cfab2

Comment 29

[Nicolas Silva [:nical]]

Attachment: Block D3D11 for intel driver version 8.15.10.1749 on Windows 7

This is essentially the same patch except that it only blocks windows 7 since all reports seem to be there.
What caused this to be backed out in the first place was that we used to fallback from D3D11 to D3D9, which is not the case anymore (now we should have software compositing instead) So this ought to be safe.

Comment 30

[Nicolas Silva [:nical]]

(In reply to Nicolas Silva [:nical] from comment #29)
> What caused this to be backed out in the first place was that we used to
> fallback from D3D11 to D3D9, which is not the case anymore (now we should
> have software compositing instead) So this ought to be safe.

Nevermind this is not the case yet, so I'll have to update the patch.

Comment 31

[Nicolas Silva [:nical]]

Attachment: Blocklist all features for intel driver version 8.15.10.1749 on Windows 7

updated patch

Comment 32

[Bas Schouten (:bas.schouten)]

Comment on attachment 8524670 [details] [diff] [review]
Blocklist all features for intel driver version 8.15.10.1749 on Windows 7

Review of attachment 8524670 [details] [diff] [review]:
-----------------------------------------------------------------

::: widget/windows/GfxInfo.cpp
@@ +970,5 @@
> +    APPEND_TO_DRIVER_BLOCKLIST(DRIVER_OS_WINDOWS_7,
> +      (nsAString&) GfxDriverInfo::GetDeviceVendor(VendorIntel),
> +      (GfxDeviceFamily*) GfxDriverInfo::GetDeviceFamily(IntelGMAX4500HD),
> +      GfxDriverInfo::allFeatures, nsIGfxInfo::FEATURE_BLOCKED_DRIVER_VERSION,
> +      DRIVER_EQUAL, V(8,15,10,1749), "8.15.10.2342");

nit: Correct the string as well please.

Comment 33

[Nicolas Silva [:nical]]

(In reply to Bas Schouten (:bas.schouten) from comment #32)
> Comment on attachment 8524670 [details] [diff] [review]
> Blocklist all features for intel driver version 8.15.10.1749 on Windows 7
> 
> Review of attachment 8524670 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: widget/windows/GfxInfo.cpp
> @@ +970,5 @@
> > +    APPEND_TO_DRIVER_BLOCKLIST(DRIVER_OS_WINDOWS_7,
> > +      (nsAString&) GfxDriverInfo::GetDeviceVendor(VendorIntel),
> > +      (GfxDeviceFamily*) GfxDriverInfo::GetDeviceFamily(IntelGMAX4500HD),
> > +      GfxDriverInfo::allFeatures, nsIGfxInfo::FEATURE_BLOCKED_DRIVER_VERSION,
> > +      DRIVER_EQUAL, V(8,15,10,1749), "8.15.10.2342");
> 
> nit: Correct the string as well please.

This string is for the recommended driver version. So I put the same version as on of the other intel blacklist entries. I don't know if specifying a recommended version makes much sense since ideally users would use the latest drivers, so this string would get quickly out of date. Not sure which version makes more sense in the string here.

Comment 34

[Nicolas Silva [:nical]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/40e56f99632b

Comment 35

[Nicolas Silva [:nical]]

Comment on attachment 8524670 [details] [diff] [review]
Blocklist all features for intel driver version 8.15.10.1749 on Windows 7

Approval Request Comment
[Feature/regressing bug #]:
[User impact if declined]: Crashes on Windows when using this intel driver version.
[Describe test coverage new/current, TBPL]: None
[Risks and why]: Adds to the amount of people that don't enjoy hardware acceleration and WebGL, since this blacklists those features.
[String/UUID change made/needed]: None

Comment 36

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/40e56f99632b

Comment 37

[Liz Henry (:lizzard) (relman/hg->git project)]

There were only a few crashes for early 34 betas and then the crash rate went up for beta 7, 8, and 9. Beta 10 looks a little better, so far.   

34.0b10: 69
34.0b9: 545
34.0b8: 445
34.0b7: 70
34.0b6: 18
34.0b5: 24

Comment 38

[Liz Henry (:lizzard) (relman/hg->git project)]

My point was, even though the driver block fix in Comment 35 didn't land for 34.0b10, the crash rate for beta 10 for this signature looks substantially lower than it was for 34.0b8 and 0b9.   I'm not sure how to judge the risk of taking the patch for 34.

Comment 39

[Liz Henry (:lizzard) (relman/hg->git project)]

Sorry to keep commenting.  I hope this is useful for making a decision. I can't tell if some other fix that landed in beta 10 has changed the situation or fixed the crash.  If so, then maybe we don't need to blocklist this driver for Win7.

Comment 40

[Lawrence Mandel [:lmandel] (use needinfo)]

Comment on attachment 8524670 [details] [diff] [review]
Blocklist all features for intel driver version 8.15.10.1749 on Windows 7

Beta+
Aurora+

Comment 41

[Benoit Girard (:BenWa)]

https://hg.mozilla.org/releases/mozilla-beta/rev/6991921d6f34

Comment 42

[Benoit Girard (:BenWa)]

https://hg.mozilla.org/releases/mozilla-aurora/rev/61c227747aba

Comment 43

[Robert Kaiser]

See bug 1097321 comment #26, I think that unblocking patch caused this  to reappear with now taking 25% of all 35.0b4 crashes, (and bug 1098597 is another 50%).