DisplayLink (dlumd10.dll, dlumd11.dll) Startup crash in @0x0 | CContext::UMQueryHS_ConstBuf_(D3D10DDI_HRTCORELAYER, unsigned int, unsigned int)

RESOLVED FIXED in Firefox 40

Status

()

defect
--
critical
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: milan, Assigned: jrmuizel)

Tracking

(Blocks 1 bug, {topcrash-win})

32 Branch
mozilla43
x86
Windows NT
Points:
---
Dependency tree / graph
Bug Flags:
in-testsuite ?

Firefox Tracking Flags

(firefox40+ fixed, firefox41+ fixed, firefox42+ fixed, firefox43+ fixed)

Details

(crash signature)

Attachments

(7 attachments, 1 obsolete attachment)

3.35 KB, patch
jrmuizel
: review+
milan
: checkin+
Details | Diff | Splinter Review
2.96 KB, patch
jrmuizel
: review+
milan
: checkin+
Details | Diff | Splinter Review
3.28 KB, patch
jrmuizel
: review+
milan
: checkin+
Details | Diff | Splinter Review
1.31 KB, patch
jrmuizel
: review+
milan
: checkin+
Details | Diff | Splinter Review
3.42 KB, patch
jrmuizel
: review+
bas.schouten
: review+
Details | Diff | Splinter Review
2.96 KB, patch
bas.schouten
: review+
Details | Diff | Splinter Review
3.28 KB, patch
bas.schouten
: review+
Details | Diff | Splinter Review
+++ This bug was initially created as a clone of Bug #1021265 +++

This bug was filed from the Socorro interface and is 
report bp-7553c1b0-b28e-45a2-837d-494972140530.
=============================================================

See https://bugzilla.mozilla.org/show_bug.cgi?id=1021265#c109 in particular.  We want to also look for "dlumd10.dll" and "dlumd11.dll", not just "dlumd32.dll".
Assignee: nobody → jmuizelaar

Updated

4 years ago
Summary: DisplayLink (dlumd32.dll) Startup crash in @0x0 | CContext::UMQueryHS_ConstBuf_(D3D10DDI_HRTCORELAYER, unsigned int, unsigned int) → DisplayLink (dlumd10.dll, dlumd11.dll) Startup crash in @0x0 | CContext::UMQueryHS_ConstBuf_(D3D10DDI_HRTCORELAYER, unsigned int, unsigned int)

Comment 1

4 years ago
this startup crash is spiking after the firefox 40 update

Comment 2

4 years ago
[Tracking Requested - why for this release]:
also setting the tracking flag to get this startup crash on the radar of relman, since this signature is #8 on windows in early crash data for firefox 40
Startup crash... tracking.
Milan, Jeff, is it something actionable? thanks
Flags: needinfo?(milan)
Flags: needinfo?(jmuizelaar)
We may need a different bug. The proposal in comment 0 would not fix this issue. The reports on v40 are not about dlumd10 or dlumd11.

The reports we're seeing have dlumd32.dll, and igd10umd32.dll, and their DisplayLink driver version is <= V(8,6,1,36484). The block in DoesD3D11DeviceWork ought to have kicked in on these systems.

The only thing I can imagine is that these machines satisfied this check:

  if (gfxPrefs::Direct2DForceEnabled() ||
      gfxPrefs::LayersAccelerationForceEnabled())

Jeff/Milan does my reasoning make sense? What can we do, if anything?
The block in DoesD3D11DeviceWork did kick in, at least on a few crashes I looked at, because we got the "[GFX1-]:DisplayLink: too old version" message which signifies this.  Which probably means it isn't the force/force situation above.  I think we have a bug in the initialization, will take a look.
Ah, good call to check the error annotations!

displaylink    85 	100.00 %
old            85 	100.00 %
too            85 	100.00 %
Bug 1107299 removed some of the code paths from bug 1021265; given the spike in 40, this isn't a direct regression from that bug, but need to trace code a bit more.
Flags: needinfo?(bas)
I think I can reproduce this locally.
I haven't searched for when, but we have regressed bug 1021265 completely.  When I simulate "bad DisplayLink driver" locally, I still end up in accelerated everything.  I'll get a trunk patch shortly.
Comment on attachment 8647045 [details] [diff] [review]
Actually do something if DisplayLink driver is of a bad version. r=jrmuizel

Bad version of DisplayLink driver leads to a startup crash.  This got fixed in bug 1021265, and then got unfixed somewhere along the way.
Attachment #8647045 - Flags: approval-mozilla-release?
Attachment #8647045 - Flags: approval-mozilla-beta?
Attachment #8647045 - Flags: approval-mozilla-aurora?
Comment on attachment 8647045 [details] [diff] [review]
Actually do something if DisplayLink driver is of a bad version. r=jrmuizel

Preapproving it. Please land that as soon as Jeff approved the patch (hopefully, he will)
Attachment #8647045 - Flags: approval-mozilla-release?
Attachment #8647045 - Flags: approval-mozilla-release+
Attachment #8647045 - Flags: approval-mozilla-beta?
Attachment #8647045 - Flags: approval-mozilla-beta+
Attachment #8647045 - Flags: approval-mozilla-aurora?
Attachment #8647045 - Flags: approval-mozilla-aurora+
Flags: needinfo?(milan)
Attachment #8647070 - Flags: review?(jmuizelaar)
Assignee

Updated

4 years ago
Attachment #8647045 - Flags: review?(jmuizelaar) → review+
The "beta" and "aurora" patches are based on what's currently mozilla-beta and mozilla-aurora.  Right around the merge, I don't know what that actually is.
Assignee

Updated

4 years ago
Attachment #8647070 - Flags: review?(jmuizelaar) → review+
Assignee

Updated

4 years ago
Attachment #8647078 - Flags: review?(jmuizelaar) → review+
None of these patches appear to apply to mozilla-release.
Flags: needinfo?(milan)
Assignee

Comment 19

4 years ago
Posted patch release-patchSplinter Review
Flags: needinfo?(jmuizelaar)
Attachment #8647222 - Flags: review+
Is there any way to manually verify this fix?
Milan, can you verify that?
When testing, I was simulating the problem and the fix in a local build by forcing the "you have a bad version of  a DisplayLink driver" even though I don't even have that driver.

To test on the installed version, one would need to find and install an old DisplayLink driver, we don't have a spoofing setup for these.  "Old version" is defined as older or same as version 8.6.1.36484
Flags: needinfo?(milan)
Flags: needinfo?(bas)
Oh, and the test is - without this fix, Firefox would crash, and if it doesn't, the about:support would be showing D3D11 as the compositor of choice.  With the fix, no crash, compositor of choice Basic OMTC.
https://hg.mozilla.org/mozilla-central/rev/c67e052b3db8
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla43
Milan and I talked about this in our 1:1. We were wondering if this could get some unit test coverage.
Flags: in-testsuite?
(In reply to philipp from comment #27)
> sorry to bug again, but this is still showing up in 40.0.2 data, for example:
> https://crash-stats.mozilla.com/report/index/64931a12-486a-491d-9eca-
> 62c6a2150817
> https://support.mozilla.org/en-US/questions/1078464

dlumd11.dll 	8.6.1.35912 	ACA742FA10574AD0859528BEB55A278D1 	dlumd32.pdb

Now we are back to the issue for which this bug was actually opened:

(In reply to Milan Sreckovic [:milan] from comment #0)
> We want to also look for "dlumd10.dll" and "dlumd11.dll", not just
> "dlumd32.dll".
Flags: needinfo?(milan)
Good point - I opened another bug 1195844, this one got overloaded with the regression fix.
Flags: needinfo?(milan)
See https://bugzilla.mozilla.org/show_bug.cgi?id=1195844#c27
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
The patch above doesn't quite do it.

We need to find out if we have a bad version of DisplayLink driver DLLs before we make a call to create the D3D11 device.  The way the code worked before this patch was to create the D3D11 device, then check the DisplayLink versions - by then it was too late, as we were crashing in the D3D11 device creation.  The patch changed it to check those versions before making the call toe D3D11 device creation.  However, the versions are checked on loaded DisplayLink DLLs, and it is the D3D11 device creation that currently loads those DLLs, so at the time that we want to check, those DLLs are not loaded.
Looking at the crashes, they were not in the creation, but later, so this change should solve the "asking for DLL versions too early".
Attachment #8657152 - Flags: review?(jmuizelaar)
Attachment #8657152 - Flags: review?(bas)
Assignee

Comment 33

4 years ago
Comment on attachment 8657152 [details] [diff] [review]
Part 2. Make sure DLLs are loaded before we check the version.

Review of attachment 8657152 [details] [diff] [review]:
-----------------------------------------------------------------

Let's do this more by reverting the original and fixing things differently.
Assignee

Updated

4 years ago
Attachment #8657152 - Flags: review?(jmuizelaar) → review-
Comment on attachment 8657152 [details] [diff] [review]
Part 2. Make sure DLLs are loaded before we check the version.

Talked with Jeff, I'll get a patch that backs out the original change and just does minimal diff (from what the code looked like before this bug) instead.
Attachment #8657152 - Flags: review?(bas)
Attachment #8657164 - Flags: review?(jmuizelaar)
Attachment #8657164 - Flags: review?(bas)
Attachment #8657164 - Flags: review?(bas) → review+
Assignee

Updated

4 years ago
Attachment #8657164 - Flags: review?(jmuizelaar) → review+
OK, let's checkin Part 2 on nightly, and see over the weekend if crashes go away.
Keywords: checkin-needed
(In reply to Milan Sreckovic [:milan] from comment #36)
> OK, let's checkin Part 2 on nightly, and see over the weekend if crashes go away.

This crash is pretty much never seen on nightly, and very rarely on aurora. We'll likely need a beta build.
I'll prepare a beta patch and if this stays on nightly I'll ask for the uplift.
Attachment #8657262 - Flags: review?(bas) → review+
Attachment #8657264 - Flags: review?(bas) → review+
hey Milan, is there a try run for this ?
Flags: needinfo?(milan)
Flags: needinfo?(milan)
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/60cc73ec4053
Status: REOPENED → RESOLVED
Last Resolved: 4 years ago4 years ago
Resolution: --- → FIXED
Wes did the patches for the other branches also land (but just not show up here?) They are marked as fixed in the tracking flags but that may be leftover from the earlier landing.
Flags: needinfo?(wkocher)
Milan does this still need uplift? Thanks!
Flags: needinfo?(milan)
I haven't uplifted anything from this bug today.
Flags: needinfo?(wkocher)
If this needs to be uplifted to Beta41 and Aurora42, we need to reset status-ff41 and status-ff42 to affected. Otherwise it will not show up in RelEng's queries.
Comment on attachment 8657262 [details] [diff] [review]
Part 2. The beta patch equivalent.

Approval Request Comment
What the Part 1 was aiming to fix but didn't quite manage.
Flags: needinfo?(milan)
Attachment #8657262 - Flags: approval-mozilla-aurora?
Comment on attachment 8657264 [details] [diff] [review]
Part 2. The aurora patch equivalent.

Approval Request Comment
What the Part 1 was aiming to fix but didn't quite manage.
Attachment #8657264 - Flags: approval-mozilla-beta?
(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #46)
> Milan does this still need uplift? Thanks!

Yes, we want this uplifted.
Comment on attachment 8657262 [details] [diff] [review]
Part 2. The beta patch equivalent.

Let's try with this patch.
Attachment #8657262 - Flags: approval-mozilla-aurora? → approval-mozilla-aurora+
Comment on attachment 8657264 [details] [diff] [review]
Part 2. The aurora patch equivalent.

looks like a simple crash fix, let's uplift to Beta41.
Attachment #8657264 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Let's keep checking if beta 9 has any of these.

Comment 57

4 years ago
it's looking very promising so far, no more reports on 41.0b9 - thank you!
You need to log in before you can comment on or make changes to this bug.