1164826 - Firefox aborts in shutdown with ActivClient (acpkcs211.dll)

Status: RESOLVED INCOMPLETE

(Core :: Security: PSM, defect, P3)

Description

[[e-mail]]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0
Build ID: 20150505103531

Steps to reproduce:

Click red x


Actual results:

Firefox aborted


Expected results:

Normal closure

Comment 1

[[e-mail]]

crash report 25879e8b-4157-420a-8f98-04f652150514

Comment 2

[[e-mail]]

Also crash reorts bp-25adc049-683f-45cb-b2b8-cc4882150526, bp-646de3e3-9285-422b-97b5-683a82150526, bp-731cefb7-8aa8-46de-927d-023bf2150526 ,bp-110ea156-c3e3-4027-864d-9b7832150526, bp-23241523-e2eb-47ea-a7ff-fe1712150526, bp-e35a3f37-5baa-46de-beb1-2b4712150526, bp-53372c61-e079-446e-8f2f-9d4d02150526

Comment 3

[[e-mail]]

1. After starting in safe mode, firefox does not error.

Comment 4

[Wayne Mery (:wsmwk)]

(In reply to stuart.j.newman from comment #3)
> 1. After starting in safe mode, firefox does not error.

Which addons do you have installed?


(In reply to stuart.j.newman from comment #2)
> Also crash reorts bp-25adc049-683f-45cb-b2b8-cc4882150526,

 __RtlUserThreadStart | _RtlUserThreadStart
 0 		@0x52e333ae
1 	ntdll.dll	__RtlUserThreadStart
2 	ntdll.dll	_RtlUserThreadStart

Comment 5

[[e-mail]]

What is invalid?

Comment 6

[Wayne Mery (:wsmwk)]

Because, most likely, an addon caused the hang.
Hence, my question Which addons do you have installed?

Comment 7

[[e-mail]]

Attachment: list of add-ons

Comment 8

[Wayne Mery (:wsmwk)]

If I would start by disabling NFCE extension. Does the hang happen with it disabled?
Also, which antivirus program(s) are installed on your computer?

Comment 9

[[e-mail]]

Firefox fails with the NFCE extension disabled.  The antivirus is Symantec Endpoint Protection 12.1.5.

The OS is Windows 7 Enterprise SP1.

Comment 10

[Wayne Mery (:wsmwk)]

Does it reliably, i.e. multiple times, not abort on shutdown if you have started Firefox in safe mode?
And most of your crashes have a website URL - what happens in a normally started Firefox if you do not access any websites?

Comment 11

[[e-mail]]

It now consistently aborts when I start firefox in safe mode.  It also aborts if I terminate without going to a web site.

Comment 12

[Wayne Mery (:wsmwk)]

1. please fine your crash IDs and post them here.  see https://support.mozilla.org/en-US/kb/firefox-crashes-troubleshoot-prevent-and-get-help#w_get-help-fixing-this-crash

2. does crashing stop if you start MS Windows in safe mode?

Comment 13

[[e-mail]]

Attachment: Doc1.docx

View of NFCE functions

Comment 14

[[e-mail]]

Here is the list of crash reports:


Submitted Crash Reports
Report ID 	Date Submitted
bp-6987b175-9a12-4d94-bb21-3bf802150529
	2015-05-29	09:09
bp-886c643c-7da2-4dae-8090-11e212150529
	2015-05-29	09:09
bp-a7f8ee49-e2b7-41d8-9bfa-0336c2150529
	2015-05-29	09:09
bp-b96e1ca2-a47e-40e7-bc74-307852150529
	2015-05-29	09:08
bp-929296ec-22b6-4c86-942b-9879d2150529
	2015-05-29	09:08
bp-bbc99829-a370-4460-9fb1-d5a862150529
	2015-05-29	09:07
bp-ff1c448c-9571-4533-8345-3f04f2150529
	2015-05-29	09:07
bp-fcc2fbbc-c602-429f-8941-ce9de2150529
	2015-05-29	08:56
bp-bdab19e5-484f-48e1-8e2f-352622150529
	2015-05-29	08:54
bp-2e5e790b-24a0-45f5-9f4d-8f7152150529
	2015-05-29	08:40
bp-f8ab58a2-a1aa-4ea5-94f2-997432150529
	2015-05-29	08:27
bp-52ff3879-fbb0-40bf-8885-3f1032150529
	2015-05-29	08:27
bp-074a4a3a-d6fd-4748-bce5-358f52150529
	2015-05-29	08:25
bp-c7b5e4b1-1fc2-43d8-bd0e-715562150529
	2015-05-29	08:24
bp-8a5e8878-45a9-4f33-8faa-c52922150529
	2015-05-29	08:22
bp-737b98a5-388d-47ad-ad60-acdcb2150529
	2015-05-29	08:22
bp-cdd162ae-0617-4489-8d28-6e1992150529
	2015-05-29	08:22
bp-e6d2749b-18c9-48a3-8888-c9b3d2150529
	2015-05-29	08:20
bp-e6090b4d-9017-4c56-8ebb-e4ac42150529
	2015-05-29	08:20
bp-87d4a249-0bf4-4941-8860-9770d2150529
	2015-05-29	08:09
bp-9184f155-8a99-4ba4-b5f0-35c2f2150529
	2015-05-29	08:06
bp-2523ad34-03b9-4c90-9868-442f12150529
	2015-05-29	08:05
bp-fa434066-b741-48d3-91f1-c09622150529
	2015-05-29	08:04
bp-905624df-5eed-4727-95bc-39c572150529
	2015-05-29	07:52
bp-f61903d3-223e-44aa-840d-248032150529
	2015-05-29	07:49
bp-8a4cbfcc-016d-4736-a565-6106b2150529
	2015-05-29	07:33
bp-77b785a3-a3f2-4116-b893-1777d2150529
	2015-05-29	07:32
bp-8e0c8887-b42a-443e-96d3-2ca622150529
	2015-05-29	06:18
bp-963099a2-7b76-42ac-b2d8-258fc2150528
	2015-05-28	15:14
bp-f3c92d09-5c63-4759-8019-98e7d2150528
	2015-05-28	15:14
bp-cd0ce8ab-95dd-4026-be4b-442e32150528
	2015-05-28	09:44
bp-0d0c8573-db16-4b90-b922-0339b2150528
	2015-05-28	07:43
bp-ddeabeb4-e055-47d8-b043-115ed2150528
	2015-05-28	07:43
bp-88827503-69ba-4a7e-bfce-8cd132150528
	2015-05-28	06:50
bp-b17450a4-a6d8-42e6-9d30-4e98f2150526
	2015-05-26	15:23
bp-fd47fc7e-e6a0-45ac-913e-b17c52150526
	2015-05-26	13:48
bp-7257b67c-4ea9-43bd-96f1-b02582150526
	2015-05-26	13:39
bp-30b0c5e3-02ab-4ff9-83f9-f0a382150526
	2015-05-26	13:24
bp-8460b408-64db-4e9b-8219-05d0c2150526
	2015-05-26	12:44
bp-c49aae84-5da2-4504-8e35-d45172150526
	2015-05-26	12:44
bp-1c0a697c-2ad9-452a-a7e2-e3c902150526
	2015-05-26	12:44
462f86c2-2cf7-4248-92c3-499006e97d7f
	2015-05-26	10:11
bp-3925632e-fd04-4d41-8888-9e1282150526
	2015-05-26	09:35
bp-25adc049-683f-45cb-b2b8-cc4882150526
	2015-05-26	09:15
bp-646de3e3-9285-422b-97b5-683a82150526
	2015-05-26	08:31
bp-731cefb7-8aa8-46de-927d-023bf2150526
	2015-05-26	08:25
bp-110ea156-c3e3-4027-864d-9b7832150526
	2015-05-26	07:02
bp-23241523-e2eb-47ea-a7ff-fe1712150526
	2015-05-26	06:55
bp-e35a3f37-5baa-46de-beb1-2b4712150526
	2015-05-26	06:55
bp-53372c61-e079-446e-8f2f-9d4d02150526
	2015-05-26	06:24
bp-2b2d3a58-852c-4b9f-9356-5e4432150519
	2015-05-19	10:00
bp-1b687edc-497c-4cb5-a9ac-c0de12150519
	2015-05-19	09:57
bp-7d42745a-a710-4cc8-b160-8a6062150519
	2015-05-19	09:46
bp-e779a048-3a8a-4e09-9018-025022150519
	2015-05-19	09:45
bp-606951f5-c4a7-4b6f-a4c3-ab1c72150519
	2015-05-19	09:44
bp-91ebc244-f71e-42d4-9cf8-2d4742150519
	2015-05-19	09:43
bp-a1ce1099-5279-49ec-9b10-c45812150519
	2015-05-19	09:21
bp-f5121542-99dc-47ca-9c1e-6fed52150519
	2015-05-19	06:58
bp-1846394b-ddd3-4499-8561-326362150514
	2015-05-14	07:47
bp-cb6f586f-5c13-45cb-8ba3-aeb882150514
	2015-05-14	07:46
bp-0082adc3-3207-48c8-8eea-511592150514
	2015-05-14	07:40
bp-81328e35-d740-44f6-9d73-f5ff42150514
	2015-05-14	07:38
bp-25879e8b-4157-420a-8f98-04f652150514
	2015-05-14	06:59
bp-becbf399-d30b-429e-b4d9-fc3282150514
	2015-05-14	06:59
bp-26a63f67-ca62-4f9e-a0e9-eb1c92150513
	2015-05-13	15:27
bp-b1b500c9-f4a7-4eb7-ab52-d44282150513
	2015-05-13	15:25
bp-1be33e86-670d-4d6d-b404-0abd52150513
	2015-05-13	15:19
bp-0ac29a5e-e83f-4cc1-a976-251eb2150513
	2015-05-13	15:05
bp-46dd4c11-4a13-4dbc-b2e8-741f72150513
	2015-05-13	15:02
7c7290e9-490b-4b66-8f9f-418fcf72322a
	2015-05-13	07:24
bp-3aaf7540-f3c8-445f-a124-460da2150512
	2015-05-12	09:43
bp-70b7f4bd-f847-412a-949d-b32492150512
	2015-05-12	09:38
bp-1ed59ee6-42f3-4a79-adaa-961c62150512
	2015-05-12	09:38
bp-a6ad98c1-b349-49fe-a15c-550ef2150512
	2015-05-12	09:29
bp-44df28d2-8d57-42b4-97c1-8a69d2150512
	2015-05-12	09:28
bp-9bf356cf-4079-42c4-9764-6dd3a2150512
	2015-05-12	09:21
bp-98ca32ae-8f9c-42cc-a163-f4e5b2150512
	2015-05-12	09:17
bp-806c93be-1a76-4007-9a4e-3e3c22150512
	2015-05-12	09:13
bp-3f424c85-186f-43b9-b35f-927752150512
	2015-05-12	09:12
bp-a66e50ba-bbd6-4fac-b1c1-c84752150512
	2015-05-12	09:10
bp-0fdc6461-7eb3-4bc1-bd15-85d552150512
	2015-05-12	08:53
bp-d0e116cf-b249-414b-8c55-f07af2150512
	2015-05-12	08:52
bp-1c12a2c6-af65-4045-8bbb-ec6502150512
	2015-05-12	08:42
bp-04b71269-4e48-407e-9d68-4aca72150512
	2015-05-12	08:24
bp-e1458fb3-3f40-46bb-be72-4ffdf2150512
	2015-05-12	08:22
bp-9f9ad8a5-a778-4809-aa9b-e5c312150512
	2015-05-12	07:56
bp-57b46c49-47f5-4d5a-b80b-f9cf72150512
	2015-05-12	07:35
bp-998f7de6-a1e5-4511-9ec7-7eead2150512
	2015-05-12	07:34
bp-e7f2cf18-2155-46b3-821e-3e2552150512
	2015-05-12	07:22
bp-fb099371-5ece-4fe5-b922-586f82150512
	2015-05-12	07:20
bp-405c96d4-2131-47eb-b40b-173c12150512
	2015-05-12	07:19
bp-3149915b-b497-44ca-ba09-30bb52150512
	2015-05-12	07:11
bp-8828a6ea-16fe-4bbc-b2f0-d020e2150512
	2015-05-12	07:10
bp-8ff2de1f-0744-42d1-bb89-f2f992150512
	2015-05-12	07:10
bp-7eba67cc-0fdf-4853-9304-138e32150512
	2015-05-12	07:09
bp-b197b816-59e1-4209-b12b-01b832150512
	2015-05-12	07:07
bp-d1f7df2b-1a75-493f-85be-630352150512
	2015-05-12	07:06
bp-e0d4a8e7-1f16-481c-8838-702132150512
	2015-05-12	07:01
bp-0baba06e-80ec-4aef-bd73-f32c42150512
	2015-05-12	06:47
bp-24444168-eec7-4f4a-a1ad-abe092150512
	2015-05-12	06:34
bp-eb61ed2b-f839-4ed4-be0f-97eb82150512
	2015-05-12	06:32
bp-9f4b314b-12bc-475c-a886-fe2862150511
	2015-05-11	15:48
bp-1a85569b-c458-4814-a003-374532150511
	2015-05-11	15:44
bp-7a2f5978-cc38-4860-ac5e-b77042150511
	2015-05-11	07:16
bp-2583cd43-ac24-40e9-a087-2b1752150511
	2015-05-11	07:15
bp-78e753dc-159d-4758-8493-665342150511
	2015-05-11	07:15
bp-b3c50789-b704-4674-acd6-f47fc2150508
	2015-05-08	07:23
bp-bb6aff77-0db2-462a-b464-c8d2c2150508
	2015-05-08	07:03
bp-50ee495e-04c9-4354-9b40-1a9592150507
	2015-05-07	15:15
bp-aff7c78c-6f31-4a34-8519-a1c962150507
	2015-05-07	13:59
bp-a8f877c8-fb8b-475c-ad8f-02e572150507
	2015-05-07	13:50
bp-682228fc-7b4b-4f8e-a7d1-a7e892150507
	2015-05-07	13:50
bp-aa826d46-4a5c-4c2a-b4ca-6479c2150507
	2015-05-07	10:13
bp-2287b650-4424-4b2e-a8a1-fa9ed2150507
	2015-05-07	07:44
bp-97cb0c1e-efb9-41a6-8616-20dad2150507
	2015-05-07	07:26
bp-6870a836-3996-4fee-9920-af0142150507
	2015-05-07	06:40
bp-ae9b66a8-7d21-4c5a-8f64-1967b2150505
	2015-05-05	15:05
bp-7066ef92-e09b-4765-8072-363ce2150505
	2015-05-05	14:59
bp-c6e3f41c-e198-4a7a-a5b4-45ed62150505
	2015-05-05	13:17
bp-87c68eee-5b91-45f6-8a4a-35c5c2150505
	2015-05-05	09:34
bp-f2a4faa2-b3b4-4ff6-93d9-ea7552150505
	2015-05-05	09:34
bp-21fd0de7-efb9-4ed3-bfc8-33fa12150501
	2015-05-01	14:27
bp-e27a2e5a-619a-481c-8f91-7ea622150501
	2015-05-01	10:16
bp-2f0fe613-a581-473d-9bc3-b768a2150501
	2015-05-01	06:48
bp-835e6b9a-1057-42f4-804e-12b2a2150501
	2015-05-01	06:42
bp-0076fecb-b54c-4963-8758-faea12150430
	2015-04-30	09:06
bp-fb42849c-59c9-45b0-8461-ec2622150430
	2015-04-30	08:36
bp-0cf60f71-61ac-4f97-b7f1-cc2642150430
	2015-04-30	08:34
bp-1c860349-16f7-4bf3-8d91-e13b42150430
	2015-04-30	08:24
bp-47c6dcc0-e35b-474d-89b3-1f0962150430
	2015-04-30	08:02
bp-8213d9b8-1ab4-40be-ba87-dde392150430
	2015-04-30	07:15
bp-390a6e50-c69f-4e79-8c11-7313d2150428
	2015-04-28	09:19
bp-16047e11-3c7b-4870-8040-0ea6e2150326
	2015-03-26	11:16
bp-84e47066-4c43-4b5f-ac94-5a50e2150312
	2015-03-12	14:15
3e5647c8-98de-4daa-84a1-47b34a270374
	2015-03-02	08:40
bp-6fcc6461-b9a1-437e-b1c0-45a9c2150212
	2015-02-12	08:21
bp-7b91204d-da53-4aa8-b845-972bc2150126
	2015-01-26	06:50
bp-86aa5a15-184f-4b13-a22e-f26bf2150126
	2015-01-26	06:49
94d452ea-6204-4fd4-962b-f48d64381f57
	2014-12-31	17:59
69902193-c540-4c52-8715-9a1aa0444a12
	2014-12-04	08:19
d847f80d-eb44-4e48-a328-5c289ed66b7a
	2014-11-17	13:52
bp-18ea035f-2065-4089-9b7d-a061e2141024
	2014-10-24	14:41
bp-e7699dbf-c8db-4ce0-bf99-291e42140825
	2014-08-25	09:39
bp-7ab8eaf9-1d82-43d2-a9ad-f0e1e2140813
	2014-08-13	08:50
bp-dd29b83f-af7c-476d-9fa3-bfdd22140813
	2014-08-13	08:44
78bcb235-4dbc-484c-823f-b4b96797a364
	2014-08-11	07:46
aa53b059-0f33-4b1f-a6df-d43ff6939d55
	2014-07-21	06:29
c54ba63b-f4a1-40fc-b44e-0f0f0e636205
	2014-06-17	06:24
d29ceeb7-cbe0-44f8-a18c-1358c1f90d5a
	2014-06-15	17:47
248b5d76-bdba-4906-869d-cc9af61b6d14
	2014-06-15	17:46
165d36fe-b03c-4fd3-8e82-e13185e11aad
	2014-06-15	17:39
392c6e8b-1d9c-41a0-b5de-17f933184fea
	2014-06-15	17:33
c0486fbe-f44e-4cb2-8aa2-87d90418ebe4
	2014-06-15	17:31
f323cb61-38cb-48f7-bacd-991d6725ecf7
	2014-06-15	17:30
60f54c16-6e6d-49ff-a6da-a34ce1bc37f8
	2014-06-15	17:30
7a0c4ab2-d340-4935-b78a-51d9c766ea49
	2014-06-15	17:30
7cc0de6f-c9c2-448a-afc6-f7e639da02f1
	2014-05-12	11:26
91e2e319-3022-4769-8903-d3e579f134d4
	2014-04-21	11:23
81cd9399-0236-44ed-b123-62b4043bc14e
	2014-04-09	05:01
ad57e4cb-8760-4567-80ee-8f66c8a519ad
	2014-01-09	12:02
a6b967c1-78fd-4d8a-9b3d-51127a358c6d
	2014-01-09	12:00
176bf3be-9868-447b-93a7-7e2e7d284d0e
	2013-11-25	11:52
fb78ef2c-07ba-46b3-a035-1c60a12c0f43
	2013-11-25	11:52
641c9696-5aac-4b81-904c-2afb7329efca
	2013-11-25	11:49
47ce3594-c2ae-44ec-932c-dba74b21009a
	2013-11-25	11:49
4588f59b-e87e-412e-b1ae-6e8d7abdc597
	2013-11-25	11:46
27fedde7-f19f-47fa-8046-62f03d615ef7
	2013-11-25	11:46
c15260e1-c0f6-4162-941b-c9a00f6a8bed
	2013-11-15	11:24
b52d7592-35d1-4652-aed8-d281fbbdf3ab
	2013-11-12	11:31
67ffc86c-9874-4d0f-a5c3-7e29145dd6df
	2013-09-24	11:56
bp-f5ff6875-237f-4b29-a5cc-5b18d2121127
	2012-11-27	09:15
bp-67f84f44-2a21-48e9-b42b-260b52121127
	2012-11-27	09:15
bp-8b613599-dc62-41d5-a979-82d822121127
	2012-11-27	09:15
bp-425a3aaf-1620-44ec-8d56-c53e72121018
	2012-10-18	15:10
bp-946ca930-4d02-4937-a93f-0b6452121018
	2012-10-18	15:10
bp-3cf84f8b-be86-4895-b3d0-43fc82110802
	2011-08-02	10:13
bp-e0e4fb16-0d44-4f70-be18-1d6ea2110802
	2011-08-02	10:13
bp-6158278a-f321-4f22-a206-e28542110711
	2011-07-11	15:17
It appers that the NFCE function to use PKI is the cause of the crashes.  It connects Firefox to ActivClient (acpkcs211.ldll) to access the smartcard.

Comment 15

[Wayne Mery (:wsmwk)]

> It appers that the NFCE function to use PKI is the cause of the crashes.  It connects Firefox to 
> ActivClient (acpkcs211.dll) to access the smartcard.

Comment 16

[[e-mail]]

I am able to confirm that the problem occurs as a result of Firefox connecting to acpkcs211.dll from ActivClient 7.0.2 64 bit

Comment 17

[Milan Sreckovic [:milan] (needinfo for best results)]

A note - some of these look like crashes *in* safe mode, on Windows 8.1 (e.g., https://crash-stats.mozilla.com/report/index/edb1741c-1d6a-47d8-aa51-67e3f2151005).

Comment 18

[Ryan VanderMeulen [:RyanVM]]

Aaron, any idea what might be going on here?

Comment 19

[Benjamin Smedberg]

I don't know how one disables PKCS modules in general, but to check whether we are in safe mode use nsIXULRuntime.inSafeMode.

Also we can add the known-bad versions module to the DLL blocklist at http://mxr.mozilla.org/mozilla-central/source/mozglue/build/WindowsDllBlocklist.cpp#63

Comment 20

[[e-mail]]

To disable a PKCS module, use options -> advanced -> Certificates -> Security Devices to unload the module

Comment 21

[(No longer employed by Mozilla) Aaron Klotz]

The dumps are showing a background thread crashing on a bad address. Unfortunately the return address is also null so I don't know where this came from. I can add the blocklist entry.

Comment 22

[(No longer employed by Mozilla) Aaron Klotz]

I see acpkcs211.dll and its version number in the older crashes. I don't see acpkcs211.dll in the module list on some of the newer crashes. Can you provide us with any more info here?

Comment 23

[(No longer employed by Mozilla) Aaron Klotz]

Looks like ActivClient changed its DLL naming scheme at some point in the upgrade. I'll continue investigating, but I'd like to figure out which of those DLLs is equivalent to acpkcs211.dll as the provider of an entry point to Firefox.

Comment 24

[Robert Kaiser]

As I expected, the bug here and the startup crashes in 42 seem to not really be related. The ActivClient issue was a shutdown issue, but right now, this signature, which is mostly startup, has correlations with other stuff:

  __RtlUserThreadStart | _RtlUserThreadStart|EXCEPTION_ACCESS_VIOLATION_EXEC (109 crashes)
     52% (57/109) vs.   2% (687/31457) LavasoftTcpService.dll
     70% (76/109) vs.  25% (7912/31457) kernel.appcore.dll
     70% (76/109) vs.  27% (8447/31457) bcryptPrimitives.dll
     70% (76/109) vs.  27% (8457/31457) SHCore.dll
     70% (76/109) vs.  27% (8458/31457) combase.dll
     69% (75/109) vs.  27% (8344/31457) WINMMBASE.dll
     50% (54/109) vs.  10% (3102/31457) dbgcore.dll
     50% (54/109) vs.  14% (4452/31457) windows.storage.dll
     40% (44/109) vs.  10% (3011/31457) dataexchange.dll
     40% (44/109) vs.  10% (3011/31457) msvcp_win.dll
     40% (44/109) vs.  10% (3018/31457) dcomp.dll
     40% (44/109) vs.  10% (3039/31457) ucrtbase.dll
     40% (44/109) vs.  11% (3362/31457) twinapi.appcore.dll

I'm pretty sure that ActivClient is not LavasoftTcpService, or is it?

Comment 25

[(No longer employed by Mozilla) Aaron Klotz]

(In reply to Robert Kaiser (:[e-mail]) from comment #24)
> I'm pretty sure that ActivClient is not LavasoftTcpService, or is it?

Definitely distinct.

Comment 26

[Liz Henry (:lizzard) (relman/hg->git project)]

This is still showing up as one of the main startup crashes on 43. Should we be using a new bug to track this issue?

Comment 27

[(No longer employed by Mozilla) Aaron Klotz]

(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #26)
> This is still showing up as one of the main startup crashes on 43. Should we
> be using a new bug to track this issue?

Separate bug for LavasoftTcpService.dll (I think that's what you're referring to).

Comment 28

[[e-mail]]

Here are the crash report numbers for the last 10 crashes:
bp-0cf092cd-7a59-4bf8-a82f-cb5d02151221
	2015-12-21	06:56
bp-6061e23e-2f54-421d-aed5-44adf2151218
	2015-12-18	07:54
bp-e269a5ca-7d2e-46a0-8ddf-ee85f2151218
	2015-12-18	07:32
bp-b354bbef-6b1a-4ea6-9ca0-e3c7e2151218
	2015-12-18	07:31
bp-cc7180f0-2608-4853-8b1a-b7ab92151217
	2015-12-17	15:28
bp-f7b440f8-dd72-4481-b148-8cd522151217
	2015-12-17	15:28
bp-002cde2a-c65c-4318-b0e2-6b1082151217
	2015-12-17	15:28
bp-64869d91-4906-431a-a297-67e152151217
	2015-12-17	15:23
bp-6ade88d5-f54a-49f9-b154-a511a2151217
	2015-12-17	15:08
bp-fede72bc-108a-4f00-bcbd-40e532151217

Comment 29

[Dana Keeler (she/her) (use needinfo) [:keeler] (on leave)]

Aaron, any update on determining what the new dll is called?

Comment 30

[(No longer employed by Mozilla) Aaron Klotz]

The crash reports in comment 28 list a whole bunch of DLLs that presumably comprise the ActiveClient software, but it's not obvious from that list which one is the main PKCS module that we're loading.