909607 - crash in _ftol2_sse with OMTC=1

Status: RESOLVED FIXED

(Core :: Graphics: Layers, defect)

Description

[Makoto Kato [:m_kato]]

This bug was filed from the Socorro interface and is 
report bp-eabfee01-da7c-4319-afa8-4c43c2130827 .
 ============================================================= 
0 	dxgi.dll 	_ftol2_sse 	
1 	xul.dll 	mozilla::layers::CompositorD3D11::VerifyBufferSize() 	gfx/layers/d3d11/CompositorD3D11.cpp
2 	xul.dll 	mozilla::layers::CompositorD3D11::UpdateRenderTarget() 	gfx/layers/d3d11/CompositorD3D11.cpp
3 	xul.dll 	mozilla::layers::CompositorD3D11::BeginFrame(mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits> const *,gfxMatrix const &,mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits> const &,mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits> *,mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits> *) 	gfx/layers/d3d11/CompositorD3D11.cpp
4 	xul.dll 	mozilla::layers::LayerManagerComposite::Render() 	gfx/layers/composite/LayerManagerComposite.cpp
5 	xul.dll 	mozilla::layers::LayerManagerComposite::EndTransaction(void (*)(mozilla::layers::ThebesLayer *,gfxContext *,nsIntRegion const &,nsIntRegion const &,void *),void *,mozilla::layers::LayerManager::EndTransactionFlags) 	gfx/layers/composite/LayerManagerComposite.cpp
6 	xul.dll 	mozilla::layers::CompositorParent::Composite() 	gfx/layers/ipc/CompositorParent.cpp
7 	xul.dll 	MessageLoop::RunTask(Task *) 	ipc/chromium/src/base/message_loop.cc
8 	xul.dll 	MessageLoop::DoWork() 	ipc/chromium/src/base/message_loop.cc

Comment 1

[Makoto Kato [:m_kato]]

- Step
(This is copy from Bug 720676 comment #5 by Brian Carpenter)
1. Set layers.offmainthreadcomposition.enabled to True
2. Restart Browser
3. Minimize Browser
4. Maximize Browser


Correct stack on debugger is the following.

(1e34.e64): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=00000000 ebx=0626a4e0 ecx=06250818 edx=07c7f99c esi=0626a4e8 edi=00000001
eip=5549e61f esp=07c7f970 ebp=07c7f9dc iopl=0         nv up ei pl nz na po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010202
dxgi!memcpy+0xa9fc:
5549e61f 8b08            mov     ecx,dword ptr [eax]  ds:002b:00000000=????????
0:014> k
ChildEBP RetAddr
07c7f9dc 602894d7 dxgi!memcpy+0xa9fc
07c7fa40 604beba4 xul!mozilla::layers::CompositorD3D11::VerifyBufferSize+0x7f
07c7fa5c 604bec4a xul!mozilla::layers::CompositorD3D11::UpdateRenderTarget+0x16
07c7fabc 6067993f xul!mozilla::layers::CompositorD3D11::BeginFrame+0x2a
07c7fb34 6067bffc xul!mozilla::layers::LayerManagerComposite::Render+0x153
07c7fb94 602e11bf xul!mozilla::layers::LayerManagerComposite::EndTransaction+0x5
f
07c7fba4 60779f09 xul!mozilla::layers::LayerManagerComposite::EndEmptyTransactio
n+0x19
07c7fbd8 5ff79973 xul!mozilla::layers::CompositorParent::Composite+0xc7
07c7fbe8 5ff7a0d4 xul!MessageLoop::RunTask+0x15
07c7fbf4 5ff7a601 xul!MessageLoop::DeferOrRunPendingTask+0x30
07c7fc30 5ff7a839 xul!MessageLoop::DoWork+0x7d
07c7fc54 5ff79c94 xul!base::MessagePumpDefault::Run+0xae
07c7fc8c 5ff79d48 xul!MessageLoop::RunHandler+0x51
07c7fcac 5ff7d025 xul!MessageLoop::Run+0x19
07c7fd90 5fe85a56 xul!base::Thread::ThreadMain+0x95
07c7fd98 76ba850d xul!ThreadEntry+0xb
07c7fda4 7718bf39 KERNEL32!BaseThreadInitThunk+0xe
07c7fde8 7718bf0c ntdll!__RtlUserThreadStart+0x72
07c7fe00 00000000 ntdll!_RtlUserThreadStart+0x1b

Comment 2

[Nick Cameron [:nrc]]

(In reply to Makoto Kato (:m_kato) from comment #1)
> - Step
> (This is copy from Bug 720676 comment #5 by Brian Carpenter)
> 1. Set layers.offmainthreadcomposition.enabled to True
> 2. Restart Browser
> 3. Minimize Browser
> 4. Maximize Browser

Can repro. You don't even need to maximise, just restore the browser to whatever size it was before.

Comment 3

[Nick Cameron [:nrc]]

Attachment: Check if we are minimised before compositing

Comment 4

[Nick Cameron [:nrc]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/e8be00af7e2c

Comment 5

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/e8be00af7e2c

Comment 6

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

I just hit this using non-OMTC, D3D10 in-process compositor:

https://crash-stats.mozilla.com/report/index/f6efb4bf-e067-406d-8813-8c9762140506

Window was not minimized, I was resizing a window that had a pdf.js doc open.