Closed Bug 1268905 Opened 4 years ago Closed 4 years ago

crash in tosqep.dll/tosqep64.dll (toshiba av library)

Categories

(Core :: Audio/Video: Playback, defect, P1, critical)

47 Branch
x86
Windows NT
defect

Tracking

()

RESOLVED FIXED
mozilla49
Tracking Status
firefox46 --- unaffected
firefox47 + fixed
firefox48 --- fixed
firefox49 --- fixed

People

(Reporter: philipp, Assigned: gerald)

References

Details

(Keywords: crash, regression)

Crash Data

Attachments

(1 file)

[Tracking Requested - why for this release]:

This bug was filed from the Socorro interface and is 
report bp-9069b2da-dfd0-4195-9ac3-b3bdf2160429.
=============================================================
Crashing Thread (61)
Frame 	Module 	Signature 	Source
Ø 0 	tosqep.dll 	tosqep.dll@0x7189 	
Ø 1 	tosqep.dll 	tosqep.dll@0x7066 	
Ø 2 	tosqep.dll 	tosqep.dll@0x63a9 	
Ø 3 	tosqep.dll 	tosqep.dll@0x5249 	
Ø 4 	igd10umd32.dll 	igd10umd32.dll@0x14b17c 	
Ø 5 	igd10umd32.dll 	igd10umd32.dll@0x14b41c 	
Ø 6 	igd10umd32.dll 	igd10umd32.dll@0x14fede 	
Ø 7 	igd10umd32.dll 	igd10umd32.dll@0x8785f 	
Ø 8 	igd10umd32.dll 	igd10umd32.dll@0x6e34d 	
Ø 9 	igd10umd32.dll 	igd10umd32.dll@0x78c3d 	
10 	d3d11.dll 	CVideoProcessor::CLS::FinalConstruct(CContext*, CVideoProcessor::TConstructorArgs const*) 	
11 	d3d11.dll 	TCLSWrappers<CVideoProcessor>::CLSFinalConstructFn(CVideoProcessor::CLS*, CContext*, CVideoProcessor::TConstructorArgs const*) 	
12 	d3d11.dll 	CLayeredObjectWithCLS<CVideoProcessor>::FinalConstruct(CVideoProcessor::TConstructorArgs const&, _GUID const&, void**, CLayeredObjectWithCLS<CVideoProcessor>::SInfo const*) 	
13 	d3d11.dll 	CLayeredObjectWithCLS<CVideoProcessor>::CreateInstance(CVideoProcessor::TConstructorArgs&, void*, void*, _GUID const&, void**, CLayeredObjectWithCLS<CVideoProcessor>::SInfo const*) 	
14 	d3d11.dll 	CDevice::CreateLayeredChild(unsigned int, void const*, unsigned long, ID3D11LayeredUseCounted*, _GUID const&, void**) 	
15 	d3d11.dll 	NDXGI::CDevice::CreateLayeredChild(unsigned int, void const*, unsigned long, ID3D11LayeredUseCounted*, _GUID const&, void**) 	
16 	d3d11.dll 	NOutermost::CDevice::CreateLayeredChild(unsigned int, void const*, unsigned long, ID3D11LayeredUseCounted*, _GUID const&, void**) 	
17 	d3d11.dll 	CDevice::CreateVideoProcessor(ID3D11VideoProcessorEnumerator*, unsigned int, ID3D11VideoProcessor**) 	
18 	msvproc.dll 	CxCodeVideoProcMFTDataHandler::CreateD3D11VideoProcessor() 	
19 	msvproc.dll 	CxCodeVideoProcMFTDataHandler::OnBeginStreaming2(int) 	
20 	msvproc.dll 	CxCodeVideoProcMFTDataHandler::OnBeginStreaming() 	
21 	msvproc.dll 	CMFTSimpleDataHandler::ProcessMessage(_MFT_MESSAGE_TYPE, unsigned long) 	
22 	msvproc.dll 	CxCodeVideoProcMFT::ProcessMessage(_MFT_MESSAGE_TYPE, unsigned long) 	
23 	xul.dll 	mozilla::MFTDecoder::SendMFTMessage(_MFT_MESSAGE_TYPE, unsigned long) 	dom/media/platforms/wmf/MFTDecoder.cpp
24 	xul.dll 	mozilla::MFTDecoder::SetMediaTypes(IMFMediaType*, IMFMediaType*, long (*)(IMFMediaType*, void*), void*) 	dom/media/platforms/wmf/MFTDecoder.cpp
25 	xul.dll 	mozilla::D3D11DXVA2Manager::ConfigureForSize(unsigned int, unsigned int) 	dom/media/platforms/wmf/DXVA2Manager.cpp
26 	xul.dll 	mozilla::WMFVideoMFTManager::ConfigureVideoFrameGeometry() 	dom/media/platforms/wmf/WMFVideoMFTManager.cpp
27 	xul.dll 	mozilla::WMFVideoMFTManager::Output(__int64, RefPtr<mozilla::MediaData>&) 	dom/media/platforms/wmf/WMFVideoMFTManager.cpp
28 	xul.dll 	mozilla::WMFVideoMFTManager::Output(__int64, RefPtr<mozilla::MediaData>&) 	dom/media/platforms/wmf/WMFVideoMFTManager.cpp
29 	xul.dll 	mozilla::WMFMediaDataDecoder::ProcessOutput() 	dom/media/platforms/wmf/WMFMediaDataDecoder.cpp
30 	nss3.dll 	nss3.dll@0x2196f 	
31 	xul.dll 	mozilla::WMFMediaDataDecoder::ProcessDecode(mozilla::MediaRawData*) 	dom/media/platforms/wmf/WMFMediaDataDecoder.cpp
32 	xul.dll 	nsRunnableMethodImpl<void ( mozilla::DataStorage::*)(char const*), 1, char const*>::Run() 	xpcom/glue/nsThreadUtils.h
33 	xul.dll 	mozilla::TaskQueue::Runner::Run() 	xpcom/threads/TaskQueue.cpp
34 	xul.dll 	nsThreadPool::Run() 	xpcom/threads/nsThreadPool.cpp
35 	xul.dll 	nsThread::ProcessNextEvent(bool, bool*) 	xpcom/threads/nsThread.cpp
36 	xul.dll 	NS_ProcessNextEvent(nsIThread*, bool) 	xpcom/glue/nsThreadUtils.cpp
37 	xul.dll 	mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp
38 	xul.dll 	MessageLoop::RunHandler() 	ipc/chromium/src/base/message_loop.cc
39 	xul.dll 	MessageLoop::Run() 	ipc/chromium/src/base/message_loop.cc
40 	xul.dll 	nsThread::ThreadFunc(void*) 	xpcom/threads/nsThread.cpp
41 	nss3.dll 	_PR_NativeRunThread 	nsprpub/pr/src/threads/combined/pruthr.c
42 	nss3.dll 	pr_root 	nsprpub/pr/src/md/windows/w95thred.c
43 	msvcr120.dll 	_callthreadstartex 	f:\dd\vctools\crt\crtw32\startup\threadex.c:376
44 	msvcr120.dll 	msvcr120.dll@0x2c000 	
45 	kernel32.dll 	BaseThreadInitThunk 	
46 	ntdll.dll 	__RtlUserThreadStart 	
47 	ntdll.dll 	_RtlUserThreadStart

this crash with involvement of a toshiba module seems to be on the rise since 47 devedition/beta builds and i'm tentatively filing the crash in the A/V component.
it is apparently only happening with a number of intel gpus and mainly on windows 10:

Rank 	Adapter device id 	Count 	%
1 	0x0116 	93 	32.98 %
2 	0x0106 	83 	29.43 %
3 	0x0166 	57 	20.21 %
4 	0x0126 	25 	8.87 %
5 	0x0416 	22 	7.80 %
Deep inside WMFVideoMFTManager::ConfigureVideoFrameGeometry()/mozilla::D3D11DXVA2Manager::ConfigureForSize()
Component: Audio/Video → Audio/Video: Playback
Anthony -- Can you find someone to work on this?  (It's showing up as a new regression starting in Fx47.)
Flags: needinfo?(ajones)
Matt - can we blacklist D3D11 DXVA when this Toshiba module is involved.
Flags: needinfo?(ajones) → needinfo?(matt.woodrow)
We don't currently have module detection built into our blacklisting.

Jeff, do you remember how we blacklisted previously for bad dlls?
Flags: needinfo?(matt.woodrow) → needinfo?(jmuizelaar)
Do you want to avoid having the dll loaded or do something different if it is loaded?
Flags: needinfo?(jmuizelaar) → needinfo?(matt.woodrow)
I was thinking we want to blacklist DXVA if it is loaded.
Flags: needinfo?(matt.woodrow) → needinfo?(jmuizelaar)
There's the code in DoesD3D11DeviceWork(). Do we have any idea how many people have this dll?
Flags: needinfo?(jmuizelaar)
Anthony, can you please help us find out how common tosqep.dll is?
Flags: needinfo?(anthony.s.hughes)
(In reply to Matt Woodrow (:mattwoodrow) from comment #8)
> Anthony, can you please help us find out how common tosqep.dll is?

The short answer is that I don't think it's very common but I cannot know for sure.

Looking just at these crashes, on average we're seeing 33 unique installs reporting this crash per day. We can assume that represents about 330 users since we process 10% of crashes on Release per day. That works out to 0.0008% of users experiencing this crash.

Looking more broadly, TOSHIBA accounts for 1.21% of *all* crashes reported. I have no way of knowing what percentage of TOSHIBA users also have this DLL but I'd have to assume they all do since the DLL comes from TOSHIBA software. However, restricting the search to those crashes with this DLL only on TOSHIBA it accounts for 1.16% of TOSHIBA users.

So, to answer your question, how common is this DLL? My best guess is that it's not very common. Even if it was common, this crash is affecting a minority of users with this DLL. We should probably reach out to Toshiba to get a better answer.
Flags: needinfo?(anthony.s.hughes)
Tracked for Fx47 even though it's not high volume but every beta build there are ~30 or so crashes according to crash-stats. If there is a dll blocklist that we want to plan for before this hits release channel, that's a good idea.
Flags: needinfo?(ajones)
Gerald: Can you please blacklist these Toshiba driver versions that are crashing for DXVA ASAP. i.e. we can use software H.264 instead for these crashing driver versions?
Flags: needinfo?(ajones) → needinfo?(gsquelart)
Priority: -- → P1
Gerald: You can probably just call your IsH264DecoderBlacklisted() code from WMFDecoderModule.cpp into WMFVideoMFTManager::CanUseDXVA(), and add the affected versions to your blacklist.
I'll look into it very soon.
Assignee: nobody → gsquelart
Flags: needinfo?(gsquelart)
Attachment #8753168 - Flags: review?(cpearce) → review+
Comment on attachment 8753168 [details]
MozReview Request: Bug 1268905 - Disable D3D11 with some Toshiba DLLs - r?cpearce

https://reviewboard.mozilla.org/r/53048/#review49872
Blocks: 1269204
With https://hg.mozilla.org/integration/mozilla-inbound/rev/db5cf62c53dd, we're blocking specific versions of those libraries (and I assume we're going to do the same in bug 1269204 and bug 1273406), but we don't know if other versions are affected and we might know only when D3D11 DXVA hits release.

What's the plan if other libraries are affected in release? Can we disable DXVA with a pref?
Flags: needinfo?(anthony.s.hughes)
Crash Signature: [@ tosqep.dll@0x7189] [@ tosqep.dll@0x80c8] [@ tosqep64.dll@0x830e0] [@ tosqep.dll@0x8028] [@ tosqep.dll@0x29928] [@ tosqep64.dll@0xa3ec] [@ tosqep64.dll@0x8f8c] → [@ tosqep.dll@0x7189] [@ tosqep.dll@0x80c8] [@ tosqep64.dll@0x830e0] [@ tosqep.dll@0x8028] [@ tosqep.dll@0x29928] [@ tosqep64.dll@0xa3ec] [@ tosqep64.dll@0x8f8c] [@ tosqep.dll@0x6ff8] [@ tosqep.dll@0x716a] [@ tosqep.dll@0x71d9] [@ tosqep.dll@0x7…
Crash Signature: tosqep.dll@0x70bd] → tosqep.dll@0x70bd] [@ tosqep.dll@0x9bda]
See Also: → 1269204
(In reply to Marco Castelluccio [:marco] from comment #17)
> What's the plan if other libraries are affected in release? Can we disable DXVA with a pref?

I'll defer this to one of the devs on the bug as I don't know the answer to this question.
Flags: needinfo?(anthony.s.hughes)
https://hg.mozilla.org/mozilla-central/rev/db5cf62c53dd
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla49
The fix here (and descendants) depend on code from bug 1257028, which has not landed on beta-47. Without it, our fix would disable all D3D versions, instead of falling back from 11 to 9.
I'm working on a partial uplift:
  https://treeherder.mozilla.org/#/jobs?repo=try&revision=b85428541e2cf149afbf288645c516700d774ac5
Depends on: 1257028
Blocks: 1273691
Blocks: 1274115
Signature [@ tosqep.dll@0x71d9] moved to bug 1274115 (as it is not covered by the fix in this bug here).
Crash Signature: [@ tosqep.dll@0x7189] [@ tosqep.dll@0x80c8] [@ tosqep64.dll@0x830e0] [@ tosqep.dll@0x8028] [@ tosqep.dll@0x29928] [@ tosqep64.dll@0xa3ec] [@ tosqep64.dll@0x8f8c] [@ tosqep.dll@0x6ff8] [@ tosqep.dll@0x716a] [@ tosqep.dll@0x71d9] [@ tosqep.dll@0x7… → [@ tosqep.dll@0x7189] [@ tosqep.dll@0x80c8] [@ tosqep64.dll@0x830e0] [@ tosqep.dll@0x8028] [@ tosqep.dll@0x29928] [@ tosqep64.dll@0xa3ec] [@ tosqep64.dll@0x8f8c] [@ tosqep.dll@0x6ff8] [@ tosqep.dll@0x716a] [@ tosqep.dll@0x70bd] [@ tosqep.dll@0x9…
This fix will also catch [@ tosqep64.dll@0x8eb4 ].
Crash Signature: tosqep.dll@0x9bda] → tosqep.dll@0x9bda] [@ tosqep64.dll@0x8eb4]
Fixed in 48 by landing bug 1273691 there.
Blocks: 1346765
You need to log in before you can comment on or make changes to this bug.