1479795 - Crash in igd10iumd64.dll with Intel driver 23.20.16.4973

Status: ASSIGNED

(Core :: Graphics, defect, P3)

Description

[[:philipp]]

This bug was filed from the Socorro interface and is
report bp-f11ffa7b-64e5-4fb2-bda9-ecf350180720.
=============================================================

Top 10 frames of crashing thread:

0 igd10iumd64.dll igd10iumd64.dll@0x903f6f 
1 d3d11.dll CContext::UMQueryOM_DepthState_ 
2 igd10iumd64.dll igd10iumd64.dll@0xbf705 
3 igd10iumd64.dll igd10iumd64.dll@0x114de 
4 d3d11.dll d3d11.dll@0x17af3f 
5 igd10iumd64.dll igd10iumd64.dll@0xbf5e0 
6 igd10iumd64.dll igd10iumd64.dll@0x8ebad3 
7 d3d11.dll CDevice::CondObjectLock::CondObjectLock 
8 d3d11.dll CContext::TID3D11DeviceContext_Draw_<2> 
9 d2d1.dll CD3DDeviceLevel1::DrawDynamicVB 

=============================================================

these content crashes are on the increase for users on windows 10 with an intel gpu. the graphics driver version heavily skews towards the recently released v23.20.16.4973, so perhaps we can attempt some workaround through blocklisting it or else reach out to intel about this...

Adapter device id facet
1 	0x5912 	2848 	52.30 %
2 	0x5916 	1397 	25.66 %
3 	0x5917 	455 	8.36 %
4 	0x591b 	416 	7.64 %
5 	0x5902 	190 	3.49 %

Comment 1

[Andrew Osmond [:aosmond] (he/him)]

User reported this for Intel HD Graphics 610:

https://downloadcenter.intel.com/product/98910/Intel-HD-Graphics-610

Latest version is 24.20.100.6194. There doesn't seem to be any issues with that driver and Firefox 60+ (although there are some crashes with 52 ESR / 32-bit). Probably safe just to block it and let them upgrade.

Comment 2

[Andrew Osmond [:aosmond] (he/him)]

Update crash signatures to take the report sets which are dominated (90+%) by users with adapter version 23.20.16.4973.

Comment 5

[Andrew Osmond [:aosmond] (he/him)]

Adapter device ID graphics from crash reports suggest at least 5 different adapter device IDs are seeing significant crash rates, so it is likely the driver itself than some specific combination of driver and device.

Comment 6

[Andrew Osmond [:aosmond] (he/him)]

Attachment: 0001-Bug-1479795-Block-Intel-graphics-driver-23.20.16.497.patch, v1

try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=3566e7470df99dd1b27779e450d3c14c34eb0484

Comment 7

[Jeff Muizelaar [:jrmuizel]]

How common is this driver?

Comment 8

[Andrew Osmond [:aosmond] (he/him)]

(In reply to Andrew Osmond [:aosmond] from comment #6)
> Created attachment 8997036 [details] [diff] [review]
> 0001-Bug-1479795-Block-Intel-graphics-driver-23.20.16.497.patch, v1
> 
> try:
> https://treeherder.mozilla.org/#/
> jobs?repo=try&revision=3566e7470df99dd1b27779e450d3c14c34eb0484

1.93% from gfx telemetry

Comment 9

[Jeff Muizelaar [:jrmuizel]]

Of all users? Are all the crashes Direct2D related?

Comment 10

[Andrew Osmond [:aosmond] (he/him)]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #9)
> Of all users? Are all the crashes Direct2D related?

That's probably what the telemetry search result is yielding:

https://firefoxgraphics.github.io/telemetry/#view=hwsearch&vendor=&drivers=23.20.16.4973

It would be good to know how many of them are Windows 10, since there doesn't seem to be non-Windows 10 crashes. Maybe it is a smaller subset of the population (1.92% seems high to block, even for 200 crashes/day...).

As for examples from the crash reports, here are a few samples:

D2D:

https://crash-stats.mozilla.com/report/index/2aeec56f-fa99-4037-bdf7-2d4eb0180802
https://crash-stats.mozilla.com/report/index/ac3711e8-e3ec-4341-b239-2be1a0180802
https://crash-stats.mozilla.com/report/index/ab9fdc86-a12b-4d2d-9a67-5574f0180802

Video related (I think):

https://crash-stats.mozilla.com/report/index/0d3117fe-13db-4a61-bd35-c496b0180802

Even WebRender:

https://crash-stats.mozilla.com/report/index/a500870e-6f58-40dc-a820-7da8a0180731

Comment 11

[Gabriele Svelto [:gsvelto] (PTO)]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #7)
> How common is this driver?

This is a WHQL driver shipped via Windows Update so it's going to be pretty common.

Comment 12

[Jeff Muizelaar [:jrmuizel]]

Some additional information:
- 23.20.16.4973 is the most common driver on Gen9 hardware with 14.07% of Gen9 users.
- We don't have great evidence to suggest that 23.20.16.4973 is the only driver affected.
- It's possible to run this driver on the quantum reference hardware by manually installing it.
- It's not clear that black listing this driver will give an overall improvement in user experience.

Comment 13

[GMA]

https://downloadcenter.intel.com/download/27894/Intel-Graphics-Driver-for-Windows-10

Maybe you can suggest to Intel Gen9 users(Skylake) & Gen9.5(Kaby Lake/Coffee Lake) to update to the latest Intel 24.20.100.6229 driver. Intel has been regularly updating and fixing their driver for Gen9/9.5 GPUs.

Comment 14

[Jeff Muizelaar [:jrmuizel]]

Unfortunately, it's not practical to have users update their drivers manually.

Comment 15

[Darkspirit]

The crash in comment 0 is from Win 10. Microsoft should distribute a stable driver through Windows Update.

Comment 16

[[:philipp]]

the volume of this crash is steadily creeping up. user comments often refer to video playback (commonly on facebook, but also twitter, imgur and other common sites) and messages from windows like "Application firefox.exe has been blocked from accessing graphics hardware" at the time of the crash.
all the reports are happening when d2d1.1 is active and most of them also contain a gfx critical error section.

Comment 18

[Gabriele Svelto [:gsvelto] (PTO)]

I found another instance of this, most of the comments mention watching videos on YouTube, Facebook or just scrolling on Facebook (which must have triggered an auto-play video). Is there a way to at least prevent Firefox from crashing? While it's not our fault many of the users reporting crashes appear to be very annoyed.

Comment 19

[Gabriele Svelto [:gsvelto] (PTO)]

And this one too, there must be even more.

Comment 20

[Jeff Muizelaar [:jrmuizel]]

I've reached out to Microsoft to see if there's anything they can do to help with the situation here.

Comment 21

[[:philipp]]

did we receive a reply yet?

Comment 22

[Jeff Muizelaar [:jrmuizel]]

No.

Comment 23

[Gabriele Svelto [:gsvelto] (PTO)]

Just found another fairly high volume signature with similar stack. Not all the crashes are for the driver version specified in the title but over 90% of them are.

Comment 24

[Asif Youssuff]

Report from a user here: https://www.reddit.com/r/firefox/comments/bvy041/help_youtube_videos_choppy_some_pages_randomly/

with crash id: https://crash-stats.mozilla.org/report/index/76f6ffdc-8f23-47b1-96d3-de3ae0190604

about:support - https://pastebin.com/6Uyi1Vd3

Comment 26

[Jeff Muizelaar [:jrmuizel]]

For the record this driver is the fifth most common Intel driver and seems to be used by roughly 2.2% of our users

Comment 27

[Jessie [:jbonisteel] pls NI]

We received a response from MSFT, here is what they have said:

Intel has already published much newer drivers (26.20.100.7262/26.20.100.7263) as critical on WU for this hardware (it went to retail 100% in December). Remaining users on these older drivers are likely due to their OEMs targeting these specific older drivers, and for old hardware such as this, we generally don't see OEMs push newer drivers. This is currently all by design from a business standpoint.

I wonder if we can try to encourage people to update their drivers? This might not be straightforward, depending on their hardware.

Comment 28

[Wayne Mery (:wsmwk)]

Is bug 1678367 comment 3 an example of this? The user crashes many times per day, apparently related to use of High Contrast with Intel driver version 23.20.16.4973

Comment 29

[BugBot [:suhaib / :marco/ :calixte]]

Since the crash volume is low (less than 5 per week), the severity is downgraded to S3. Feel free to change it back if you think the bug is still critical.

For more information, please visit auto_nag documentation.