Closed Bug 623317 Opened 9 years ago Closed 2 years ago

Find out the percentage of users who have up-to-date graphics drivers

Categories

(Socorro :: Data request, task)

x86
macOS
task
Not set

Tracking

(Not tracked)

RESOLVED WORKSFORME

People

(Reporter: joe, Unassigned)

References

Details

Attachments

(9 files)

We need to find out the percentage of our users who have up-to-date graphics drivers. The best way to do this is probably to collect the top N unique vendor/device pairs out of our crash statistics, then find the breakdown of driver version on each of those pairs.
the first pass of this is simple and pretty quick since that data is in the .csv files

the list of top 13 vendor/device pairs with more 2000 reports for jan 1-4 looks like this:

count  vendor, device

8532 AdapterVendorID: 8086, AdapterDeviceID: 29c2 
7164 AdapterVendorID: 8086, AdapterDeviceID: 2772 
5243 AdapterVendorID: 8086, AdapterDeviceID: 2a42 
4916 AdapterVendorID: 0000, AdapterDeviceID: 0000, , \Registry\Machine\System\CurrentControlSet\Control\Video\{23A77BF7-ED96-40EC-AF06-9B1F4867732A}\0000 
3588 AdapterVendorID: 8086, AdapterDeviceID: 2a02 
2819 AdapterVendorID: 8086, AdapterDeviceID: 2572 
2519 AdapterVendorID: 1106, AdapterDeviceID: 3344 
2463 AdapterVendorID: 10de, AdapterDeviceID: 0322 
2421 AdapterVendorID: 8086, AdapterDeviceID: 2562 
2322 AdapterVendorID: 8086, AdapterDeviceID: 29c2
2151 AdapterVendorID: 8086, AdapterDeviceID: 2a42
2117 AdapterVendorID: 8086, AdapterDeviceID: 0046 
2027 AdapterVendorID: 1039, AdapterDeviceID: 6330 

( I can attach the full list later )

If I understand the second part of this, we want to take a combo pair like the first one

   8532 AdapterVendorID: 8086, AdapterDeviceID: 29c2 

and make a second pass over those 8500 reports with this combo to look for driver version info out of the modules list.   In the most generic search that might entail  checking the module list for any .dll that looks something like this list.

  #grep -i "Module|"nvwgf2um.dll 
  #grep -i "Module|"igdumd32.dll 
  #grep -i "Module|"igdumdx32.dll 
  #grep -i "Module|"ig4icd32.dll 
  #grep -i "Module|"ig4dev32.dll 
  #grep -i "Module|"d3d9.dll 
  #grep -i "Module|"d3d10_1core.dll  
  #grep -i "Module|"d3d8thk.dll  
  #grep -i "Module|"d3d10_1core.dll 
  #grep -i "Module|"d2d1.dll 
  #grep -i "Module|"nvwgf2um.dll 
  #grep -i "Module|"aticfx32.dll 

does that sound about right?
adaptor vendor and device id pairs for all crash reports between jan 1-4 that were processed and had this info.
here is a chart to help figure out what the long tail looks like and how deep we need to go for the value of "N" to get reasonable coverage.

there are about 1600 pair combo's if we try and look at the top quarter of those that might get us around the median of all reported pair combo's
looks like we might get a relatively small amount of driver version info for the reports where we get vendor/device id info.

in the first 100 reports I looked at for  VendorID: 8086, AdapterDeviceID: 29c2
there are the only 7 reports where we see driver version info.

http://crash-stats.mozilla.com/report/index/f0a6246f-6d57-4907-b250-496432110104
d3d9.dll 6.1.7600.16385	
d3d10_1core.dll 6.1.7600.16385	
d3d8thk.dll 6.1.7600.16385	

http://crash-stats.mozilla.com/report/index/738e5bc4-5367-46cc-b339-ec8912110104
d3d9.dll 6.1.7600.16385	
d3d8thk.dll 6.1.7600.16385	

http://crash-stats.mozilla.com/report/index/b3724519-571d-4792-8dcc-f618d2110104
d3d9.dll 6.1.7600.16385	
d3d10_1core.dll 6.1.7600.20781	
d3d8thk.dll 6.1.7600.16385	

http://crash-stats.mozilla.com/report/index/83d04194-478e-4989-afdd-30bcd2110104
d3d9.dll 6.1.7600.16385	
d3d10_1core.dll 6.1.7600.16385	
d3d8thk.dll 6.1.7600.16385	

http://crash-stats.mozilla.com/report/index/91c349c0-f81c-458d-9242-5938c2110104
d3d9.dll 6.1.7600.16385	
d3d10_1core.dll 6.1.7600.16385	
d3d8thk.dll 6.1.7600.16385	

http://crash-stats.mozilla.com/report/index/d2edcbc8-69cd-4dc3-8182-9b2d52110104
d3d9.dll 5.3.2600.2180	
d3d8thk.dll 5.3.2600.2180	

http://crash-stats.mozilla.com/report/index/00b6e60f-e412-4747-adfa-cf5dd2110104
d3d9.dll 5.3.2600.2180	
d3d8thk.dll 5.3.2600.2180

http://crash-stats.mozilla.com/report/index/e64b3f0f-94f6-4f7f-ab42-91dc82110104
d3d9.dll 6.1.7600.16385	
d3d10_1core.dll 6.1.7600.20781	
d3d8thk.dll 6.1.7600.16385	

the rest of the reports look like this sample where I didn't get any hits scrapping for driver .dll's.  

http://crash-stats.mozilla.com/report/index/1ce68e32-5f0a-4a6b-b399-c0d962110104
http://crash-stats.mozilla.com/report/index/47d43642-1099-47ef-aa67-99ef62110104
http://crash-stats.mozilla.com/report/index/5039e59a-f2ab-46cf-94ba-7bba02110104
http://crash-stats.mozilla.com/report/index/d7f3c990-e33a-4775-a111-c03f52110104
http://crash-stats.mozilla.com/report/index/1ce68e32-5f0a-4a6b-b399-c0d962110104

could be my search isn't broad enough, or maybe drivers aren't loaded at the time of the crash about 93% of the time.  does that sound right?
I don't know that we're actually annotating the crash report with the info about whether we enabled hardware acceleration or not. If we didn't enable hwaccel, I guess we might not have the driver DLLs loaded in our process.

(Also, the first crash in your list at the bottom is a plugin process crash, so you might want to filter those out as they're not likely to have useful info.)
ok, I can filter out plugin process reports.

I let a job run last night for a few hours to grab about 10% of the
AdapterVendorID: 8086, AdapterDeviceID: 29c2  crash reports

here's the driver info counts those reports contained.

  89 d3d8thk.dll 6.1.7600.16385
  72 d3d10_1core.dll 6.1.7600.16385
  27 d3d8thk.dll 5.3.2600.2180
  15 d3d10_1core.dll 6.1.7600.20781
  14 d3d8thk.dll 5.3.2600.5512
   9 igdumdx32.dll 8.14.10.1930
   9 igdumd32.dll 8.14.10.1930
   7 d3d8thk.dll 6.0.6000.16386
   4 igdumd32.dll 7.14.10.1461
   4 d3d8thk.dll 5.3.0.900
   2 igdumd32.dll 8.15.10.1749
   1 ig4icd32.dll 8.14.10.1930
   1 ig4dev32.dll 8.14.10.1930
   1 d3d8thk.dll 6.1.7100.0
   1 d3d8thk.dll 5.2.3790.0
   1 d3d10_1core.dll 7.0.6002.18107

Is this basically the kind of list we are looking for for each of the vendor device pair combos?
Yeah. The versions of ig*.dll are likely to be the important ones. I don't remember precisely which DLL matters, though.
To have the driver version breakdown, the driver version should be added to App Notes in crash reports.

Here are the reasons:
* For Intel: Assuming that ig*.dll are only loaded if D2D/D3D features are used, you will see only recent Intel drivers because of current blocklisting for D2D/D3D features.
* For NVIDIA: each nvw*.dll has its own version that may be different from the driver version.
* For ATI: the same for ati*.dll.
* For S3 graphics: the same for S3*.dll.
(In reply to comment #8)
> To have the driver version breakdown, the driver version should be added to
> App Notes in crash reports.
> 

so we should get a bug on file for that and see if tedm or someone can do something quickly.

if it can't be done in the next few weeks, then is the data we have like in comment 6 worthwhile pulling and help us to build the blacklist needed to ship firefox 4 like talked about in Bug 623338 ?
Blocks: 623338
Depends on: 624703
Some additional information that may be useful here is in 
https://bugzilla.mozilla.org/show_bug.cgi?id=624508#c6
> Some additional information that may be useful here is in ...
Can we have a copy of the comment in this bug? (bug 624508 has restricted access)
Aha, sorry. 
"All crashes 1/1/2011 to 1/10/2011 for Windows NT and version
started with 6.1.7600:

4.0b8
d2d1.dll    53515
d2d132.dll    1
no_d2d_module    58270

4.0b9pre
d2d1.dll    7799
no_d2d_module    4172"
ok, re-ran the script to gather another sample using all the AdapterVendorID: 8086, AdapterDeviceID: 29c2  crash reports for jan 11.

Also incorporated changes to filter out any plugin process crashes/hangs per comment 6 and open up the .dll search per comment 8 to look for

* Module|nvw* 
* Module|ati*
* Module|S3*

2102 crash reports found 1217 matching module instances, and 60 unique driver versions.

Its not clear to me if anyone is finding the data that I'm able to pull out useful towards the goal of constructing the block list.  

Are there any more tweeks we need to make to refine the lists to make them more useful?

Should I continue down the list of VendorID/DeviceID pairs, trying to build a larger list of module versions?
we can also reorder the list by to sort by 

 version_number, driver_name, count_found

5.2.3790.0 d3d8thk.dll 8
5.3.2600.2180 d3d8thk.dll 4
5.3.2600.2180 d3d9.dll 4
5.3.2600.5512 d3d8thk.dll 10
5.3.2600.5512 d3d9.dll 10
5.3.2600.5512 d3dim700.dll 2
5.3.3790.1830 d3d9.dll 4
5.3.3790.3959 d3d9.dll 4
6.0.6000.16386 d3d8thk.dll 11
6.0.6000.16386 d3d9.dll 4
6.0.6001.18000 d3d10_1.dll 3
6.0.6001.18000 d3d10_1core.dll 3
6.0.6002.18005 d3d9.dll 7
6.1.7068.0 d3d10_1.dll 1
6.1.7068.0 d3d10_1core.dll 1
6.1.7068.0 d3d8thk.dll 1
6.1.7068.0 d3d9.dll 1
6.1.7100.0 d3d10_1.dll 1
6.1.7100.0 d3d10_1core.dll 1
6.1.7100.0 d3d8thk.dll 1
6.1.7100.0 d3d9.dll 1
6.1.7600.16384 d3d10_1.dll 2
6.1.7600.16384 d3d10_1core.dll 2
6.1.7600.16384 d3d8thk.dll 2
6.1.7600.16384 d3d9.dll 2
6.1.7600.16385 d3d10_1.dll 269
6.1.7600.16385 d3d10_1core.dll 224
6.1.7600.16385 d3d8thk.dll 239
6.1.7600.16385 d3d9.dll 239
6.1.7600.20781 d3d10_1core.dll 45
6.1.7600.20830 d3d10_1.dll 6
6.1.7600.20830 d3d10_1core.dll 6
6.14.10.4906 igfxpph.dll 1
6.14.10.4926 igfxpph.dll 2
6.14.10.4926 igfxres.dll 2
6.14.10.4926 igfxress.dll 2
6.14.10.4926 igfxsrvc.dll 2
6.14.10.4990 igfxpph.dll 1
6.14.10.5029 igfxpph.dll 1
6.14.10.5029 igfxrenu.lrc 1
6.14.10.5029 igfxsrvc.dll 1
7.0.6002.18107 d3d10_1.dll 8
7.0.6002.18107 d3d10_1core.dll 8
7.14.10.1322 igdumd32.dll 1
7.14.10.1409 igdumd32.dll 1
7.14.10.1666 igfxpph.dll 1
7.14.10.1666 igfxrenu.lrc 1
7.14.10.1666 igfxsrvc.dll 1
7.15.10.1554 igdumd32.dll 2
7.15.10.1554 igdumdx32.dll 2
8.14.10.1930 ig4dev32.dll 2
8.14.10.1930 ig4icd32.dll 2
8.14.10.1930 igdumd32.dll 7
8.14.10.1930 igdumdx32.dll 8
8.14.10.1930 igfxpph.dll 1
8.15.10.1749 igdumd32.dll 15
8.15.10.1912 igdumd32.dll 3
8.15.10.1912 igdumdx32.dll 3
9.27.952.3001 D3DX9_42.dll 2
The fixing of bug 624703 is required to have reliable crash stats in Beta 10.

However, if we want to have crash trends before, first isolate crashes by AdapterVendor and AdapterID pairs, then for each pair (a few hundreds), sort out by OS version and:
* for AdapterVendor 8086: igdumd32.dll or igdumd64.dll versions
* for AdapterVendor 1002: aticfx32.dll or aticfx64.dll versions
* for AdapterVendor 10de: nvwgf2um.dll versions
* for AdapterVendor 5333: S3DDX10_32.dll or S3DDX10_64.dll versions
* for other AdapterVendor: nothing
> The fixing of bug 624703 is required to have reliable crash stats in Beta 10.
Now we have crash stats in 4.0b10, we can have the driver version breakdown for each vendorID/deviceID pair.
Is it valuable to have the list also contain the signature?

count signature  Device/AdapterVersion

206 \N AdapterVendorID: 0000, AdapterDeviceID: 0000, AdapterDriverVersion: , , \Registry\Machine\System\CurrentControlSet\Control\Video\{23A77BF7-ED96-40EC-AF06-9B1F4867732A}\0000 
 156 mozcrt19.dll@0x1327f AdapterVendorID: 8086, AdapterDeviceID: 2a42, AdapterDriverVersion: 8.15.10.2202 
 109 mozcrt19.dll@0x1327f AdapterVendorID: 8086, AdapterDeviceID: 0046, AdapterDriverVersion: 8.15.10.2202 
  69 F957328252________________________ AdapterVendorID: 0000, AdapterDeviceID: 0000, AdapterDriverVersion: , , \Registry\Machine\System\CurrentControlSet\Control\Video\{23A77BF7-ED96-40EC-AF06-9B1F4867732A}\0000 
  68 mozcrt19.dll@0x1327f AdapterVendorID: 8086, AdapterDeviceID: 27ae, AdapterDriverVersion: 8.15.10.1930
  65 \N AdapterVendorID: 8086, AdapterDeviceID: 29c2, AdapterDriverVersion: 8.15.10.1930 
  64 \N AdapterVendorID: 8086, AdapterDeviceID: 2a42, AdapterDriverVersion: 8.15.10.2202 
  60 js::mjit::EnterMethodJIT(JSContext*, JSStackFrame*, void*, js::Value*) AdapterVendorID: 0000, AdapterDeviceID: 0000, AdapterDriverVersion: , , \Registry\Machine\System\CurrentControlSet\Control\Video\{23A77BF7-ED96-40EC-AF06-9B1F4867732A}\0000 
  59 mozcrt19.dll@0x1327f AdapterVendorID: 1002, AdapterDeviceID: 68b8, AdapterDriverVersion: 8.801.0.0 
  58 \N AdapterVendorID: 8086, AdapterDeviceID: 2a42, AdapterDriverVersion: 8.15.10.1749 
  57 \N AdapterVendorID: 8086, AdapterDeviceID: 2a02, AdapterDriverVersion: 8.15.10.1930 
  54 mozcrt19.dll@0x1327f AdapterVendorID: 10de, AdapterDeviceID: 0640, AdapterDriverVersion: 8.17.12.5896 
  51 mozcrt19.dll@0x1327f AdapterVendorID: 10de, AdapterDeviceID: 0640, AdapterDriverVersion: 8.15.11.8593 
  50 \N AdapterVendorID: 8086, AdapterDeviceID: 2572, AdapterDriverVersion: 6.14.10.4396 
  49 mozcrt19.dll@0x1327f AdapterVendorID: 10de, AdapterDeviceID: 0e22, AdapterDriverVersion: 8.17.12.6658 


or just a listing with counts of all the  Vendor/Devices Pair info like:

count  Vendor  Adapter

1860 AdapterVendorID: 0000, AdapterDeviceID: 0000, AdapterDriverVersion: , , \Registry\Machine\System\CurrentControlSet\Control\Video\{23A77BF7-ED96-40EC-AF06-9B1F4867732A}\0000
 505 AdapterVendorID: 8086, AdapterDeviceID: 2a42, AdapterDriverVersion: 8.15.10.2202
 414 AdapterVendorID: 8086, AdapterDeviceID: 2572, AdapterDriverVersion: 6.14.10.4396
 398 AdapterVendorID: 8086, AdapterDeviceID: 29c2, AdapterDriverVersion: 8.15.10.1930
 385 AdapterVendorID: 8086, AdapterDeviceID: 2a02, AdapterDriverVersion: 8.15.10.1930
 361 AdapterVendorID: 8086, AdapterDeviceID: 2772, AdapterDriverVersion: 6.14.10.4926
 312 AdapterVendorID: 10de, AdapterDeviceID: 0322, AdapterDriverVersion: 5.6.7.3
 294 AdapterVendorID: 8086, AdapterDeviceID: 0046, AdapterDriverVersion: 8.15.10.2202
 293 AdapterVendorID: 8086, AdapterDeviceID: 2a42, AdapterDriverVersion: 8.15.10.1749
 274 AdapterVendorID: 8086, AdapterDeviceID: 27ae, AdapterDriverVersion: 6.14.10.4926
 261 AdapterVendorID: 8086, AdapterDeviceID: 2772, AdapterDriverVersion: 6.14.10.4436
 250 AdapterVendorID: 8086, AdapterDeviceID: 29c2, AdapterDriverVersion: 6.14.10.4906
 232 AdapterVendorID: 10de, AdapterDeviceID: 0110, AdapterDriverVersion: 5.6.7.3
 218 AdapterVendorID: 8086, AdapterDeviceID: 2562, AdapterDriverVersion: 6.14.10.4342
 217 AdapterVendorID: 5333, AdapterDeviceID: 8d04, AdapterDriverVersion: 6.14.10.12
 205 AdapterVendorID: 8086, AdapterDeviceID: 29c2, AdapterDriverVersion: 6.14.10.5009
ah, I should have read comment 15 before posting my last.  I can see what's needed.  might take me a little while to put it together since I'm traveling and at conferences.
2011 02 07  crash counts for:

os_ver AdapterVendorID: 8086, 
   with AdapterDeviceID: X AdapterDriverVersion: Y

let me know if this is what we are looking for.
Attached file vendor 1002 report
not sure if this is what we want for the vendorId="nothing" report, but it appears we have many reports that contain:

AdapterVendorID: 0000, AdapterDeviceID: 0000, AdapterDriverVersion: \Registry..key
Component: Socorro → General
Product: Webtools → Socorro
Is this data request still relevant, i.e. does it still need to be done? If not, can you close the bug? Thanks.
Component: General → Data request
QA Contact: socorro → data-request
I'd say it's still very much useful, but there is now a much simpler way to implement it.

Instead of looking at driver versions, look at the feature reports in AppNotes. For example, for WebGL,

   WebGL?  means we tried to initialize a WebGL context
   WebGL+  means we sucessfully initialized it. With default prefs, that entails current drivers
   WebGL-  means we failed to initialize it. In most cases, with default prefs, that is because of a blacklisted driver.

It would be very useful to automatically extract stats from crash reports, telling us what % of users get those features (also 'D3D10 Layers', 'D2D', etc).

I tried to do it at some point,
http://people.mozilla.org/~bjacob/gfx_features_stats/

but I suck at servers so I didn't get to the point where I'd set a cron job to update this page, etc.

Also see bug 640286 which has scripts to gather this sort of data.
If "Adapter Vendor ID" and "Adapter Device ID" fields were filled in in crash reports, it would be also easier to breakdown.
this is a better question for data platform / telemetry these days
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.