Closed Bug 1649937 Opened 4 years ago Closed 2 years ago

GlobalSign: Incorrect OCSP Delegated Responder Certificate

Categories

(CA Program :: CA Certificate Compliance, task)

Product:
CA Program
Component:
CA Certificate Compliance
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: ryan.sleevi, Assigned: douglas.beattie)

Details

(Whiteboard: [ca-compliance] [ocsp-failure])

Attachments

(3 files, 1 obsolete file)

GlobalSign-OneCRL-Request-20200731.xlsx
12.85 KB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
Details
Batch 1 - 20 - GlobalSign - Key Destruction - Report of Independent CPA Management Assertions_FINAL signed.pdf
392.14 KB, application/pdf
Details
2021 - GlobalSign - ISAE3000 Type II - OCSP EKU - Public - vFINAL signed.pdf
776.39 KB, application/pdf
Details

The following was originally reported to m.d.s.p. at https://www.mail-archive.com/dev-security-policy@lists.mozilla.org/msg13493.html

GlobalSign has issued one or more OCSP Delegated Responders, as defined within RFC 6960, Section 2.6 and Section 4.2.2.2, without including the id-pkix-ocsp-nocheck response, as required by the Baseline Requirements, Version 1, Section 13.2.5 through Version 1.7.0, Section 4.9.9

Example certificate: https://crt.sh/?id=2329203344

GlobalSign confirms receipt of this report and is investigating the issue.

Thanks for confirming. I noticed you went and CC'd yourself on all the other bugs related to this, and I'm encouraged to see that degree of proactiveness in learning from other CAs, although I do hope we'll see meaningfully distinct independent responses :)

Note that the recommended way for CAs to stay aware of incidents and incident responses is by subscribing to the "CA Certificates Component". If you click your profile (in the top right of Bugzilla), under "Preferences" there is a tab for "Component Watching". From there, you can monitor the Product of "NSS" with the Component of "CA Certificate Compliance", which will ensure you are notified on all compliance issues, subject to the "Email Preferences" tab. I would strongly encourage a CA to make sure they're subscribed to all comments, at a minimum.

Thanks Ryan, we are already subscribed to all bugs / comments through our SOC from where we triage it to different stakeholders / subject matter experts, but for this one - given criticality - I would like to get them independently in my inbox immediately.

GlobalSign posted the following on the relevant MDSP thread (https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/EzjIkNGfVEE):

GlobalSign recognizes the reported security issue and associated risk, and is working on a plan to remediate the impacted CA hierarchies with first priority on terminating those branches that include issuing CA with private keys outside of GlobalSign's realm. We will soon share an initial plan on our Bugzilla ticket https://bugzilla.mozilla.org/show_bug.cgi?id=1649937.

One question we have for the root store operators specifically is what type of assurance they are looking for on the key destruction activities. In the past we've both done key destruction ceremonies without and with (e.g. in the case of addressing a compliance issue like
https://bugzilla.mozilla.org/show_bug.cgi?id=1591005) an external auditor witnessing the destruction and issuing an independent ISAE3000 witnessing report.

We are still collecting all the details to provide a comprehensive overview, report and full plan of action but for now I can confirm that we will effectively revoke the following CA on July 8 2020. These are the branches in our WebPKI trusted hierarchies that have issuing CA with private keys outside of GlobalSign's environment and where some of those issuing CA contain the OCSP Signing EKU:

We are currently working on doing the necessary reissuance and replacement activities on the approximately 1 million alive leaf certificates (mostly S/MIME, client authentication) in the above hierarchy.

I'll share another batch of CA we will revoke during the July 8 revocation ceremony over the weekend or latest by next Tuesday depending on how analysis and replacement activities progress.

(In reply to Arvid Vermote from comment #5)

We are still collecting all the details to provide a comprehensive overview, report and full plan of action but for now I can confirm that we will effectively revoke the following CA on July 8 2020. These are the branches in our WebPKI trusted hierarchies that have issuing CA with private keys outside of GlobalSign's environment and where some of those issuing CA contain the OCSP Signing EKU:

Thanks. Because it took me a second to parse what was going on: These are intermediates that themselves have issued sub-CAs with the OCSPSigning EKU (e.g. https://crt.sh/?caid=1367 , which was the one originally mentioned in Bug 991209)

Following affected CA chained to WebPKI roots have been revoked July 8 16:00-17:00 UTC:

CN SHA256 crt.sh
GlobalSign Qualified Timestamping ECC CA 2020 C2FEACD674878C7B0C2325A2ECED0A333DB7780A86DFEC3758100EFC0101C665 https://crt.sh/?id=2839140405
GlobalSign HV ECC DV SSL CA 2018 4B0D1392D39157353207A64CCB14683DDE9D2CED1FB58B16E038BE5707C27813 https://crt.sh/?id=970083107
GlobalSign R6 RSA EV SSL CA 2019 57264B82A864DBA1C11EF3F80ABB94CAC3660662B0C22F571FF993B3FBCF76FB https://crt.sh/?id=1476654013
GlobalSign R6 Admin CA - SHA256 - G3 C5B679106958152F83FB5886DDC41F0785193EF67C6975BE3E509F17F29B7A86 https://crt.sh/?id=164243753
GlobalSign ECC EV SSL CA 2019 0D3176C58F321AA34C57C8DF7C17D1F4E76C797EC116C9F1D697748ED1FCE7D9 https://crt.sh/?id=2329203344
GlobalSign RSA EV SSL CA 2019 0D6E46784F3B694E9C7506786417BC6F87F9D2F73D19B5E8081612B21137B766 https://crt.sh/?id=2220986544

As per Comment #5 we have been working on replacing all affected certificates under the trusted root hierarchies in order to terminate the trusted root chains. At the time of writing we managed to reissue or migrate roughly 800K certificates to other, unaffected hierarchies.

One of the major affected parties that did not yet succeed migrating is "Kyushu Electric Power Co., Inc", which has leafs under the "JCAN Public CA1 - G4" (https://crt.sh/?caid=51044), chained to the "Trusted Root CA SHA256 G2" (https://crt.sh/?caid=1423). These certificates are critical to the operation of the electric grid provided by that party, and they indicated they need 72 additional hours to replace certificates and avoid disruption to electrical grid operations.

Based on this we plan to revoke following CA on July 11 16:00 UTC. Pending further internal progress we might add additional CAs to this revocation batch, we will confirm the CA revoked after the execution of the key ceremony.

CN SHA256 crt.sh
Trusted Root CA G2 6E32A35B599E9087BB1AB35CE73022EC2E26AF34BE388919419C95700CD8E7FB https://crt.sh/?id=1862521
Trusted Root CA SHA256 G2 01FD73EF5E70F526FC9C11F65FE2EE6F7125B3693949227FFD8E459E583C458A https://crt.sh/?id=3179271
GlobalSign PersonalSign 2 CA - SHA256 - G4 27D6FDAF80297846DFEFF82E7F58B9A48AC9E3EE93A112B1BBE243EE1A97447C https://crt.sh/?id=2839140428
Trafigura PTE Ltd S/MIME ICA 2020 5E7FCB9C97BDA56993B1658D120232761D665A3644534300FA6A5BEC5E0D5795 https://crt.sh/?id=2369948428
GlobalSign Qualified CA 2 FD3A0F3DD4480092B6D450473DEB9201A0B308A8807833A3C738F8A07EB81ED3 https://crt.sh/?id=509714291
GlobalSign Qualified CA 3 0AA9F2E7D95C718B7D1EB7CCDBD0164E86057AE9D66922BC60F9903F94A0F0EF https://crt.sh/?id=509714292
GlobalSign CA 4 for AATL 9DDC2E0D55B461E0C73228282DF56B2BEF224CA2385681D17B6E8C077852573C https://crt.sh/?id=405831326

We are further working on a full remediation plan for a total of over 60 impacted CA. We currently expect this plan and the incident report to be finalized by July 17. GlobalSign has created a separate Bugzilla ticket for not revoking all affected ICA within 7 days as stipulated in section #4.9.1.2 of the SSL Baseline Requirements: https://bugzilla.mozilla.org/show_bug.cgi?id=1651447

GlobalSign understands revocation is not sufficient to remediate the security risk associated with the current issue and is currently working with a qualified WebTrust auditor to plan for a first witnessed key destruction ceremony and delivering an ISAE3000 report on the execution of the destruction activities, We are also actively engaging with auditors to explore available options to report upon the fact of non-performance of OCSP signing by affected CA keys.

As per the above, the following affected CA have now been revoked too.

CN sha256 crt.sh
Trusted Root CA G2 6E32A35B599E9087BB1AB35CE73022EC2E26AF34BE388919419C95700CD8E7FB https://crt.sh/?id=1862521
Trusted Root CA SHA256 G2 01FD73EF5E70F526FC9C11F65FE2EE6F7125B3693949227FFD8E459E583C458A https://crt.sh/?id=3179271
GlobalSign PersonalSign 2 CA - SHA256 - G4 27D6FDAF80297846DFEFF82E7F58B9A48AC9E3EE93A112B1BBE243EE1A97447C https://crt.sh/?id=2839140428
Trafigura PTE Ltd S/MIME ICA 2020 5E7FCB9C97BDA56993B1658D120232761D665A3644534300FA6A5BEC5E0D5795 https://crt.sh/?id=2369948428
GlobalSign Qualified CA 2 FD3A0F3DD4480092B6D450473DEB9201A0B308A8807833A3C738F8A07EB81ED3 https://crt.sh/?id=509714291
GlobalSign Qualified CA 3 0AA9F2E7D95C718B7D1EB7CCDBD0164E86057AE9D66922BC60F9903F94A0F0EF https://crt.sh/?id=509714292
GlobalSign CA 4 for AATL 9DDC2E0D55B461E0C73228282DF56B2BEF224CA2385681D17B6E8C077852573C https://crt.sh/?id=405831326

How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.

On July 1st 2020 21:06 UTC a security issue was uncovered on the mozilla.dev.security.policy discussion group related to the way some CA, including GlobalSign, included the OCSP Signing EKU in certain issuing CA. Including the EKU effectively allows the issuing CA to also act in the role of a delegated OCSP responder for the parent CA, which can be abused to manipulate the validity status of the issuing CA itself and other issuing CA and certificates that share the same parent.

A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.

Time (UTC) Activity
July 1 2020 21:06 Security issue disclosed on mozilla.dev.security.policy
July 1 2020 21:42 CISO is notified of the post and the fact GlobalSign is affected
July 1 2020 22:22 CISO notifies the leadership and compliance team and mobilizes an investigation team
July 2 2020 19:31 Investigation and impact analysis completed, initial remediation plan finalized: focus on remediation of hierarchies containing third-party operated issuing CA keys first and remediate the affected issuing CA / keys under GlobalSign control as a second priority
July 3 2020 01:00 Key manager and compliance team start preparing for initial re-issuance of issuing CA within hierarchies that contain third-party operated issuing CA keys outside of GlobalSign's controls (Trusted Root)
July 3 2020 01:00 Incident team starts preparation activities to work with customers affected by the initial revocation activities
July 3 2020 13:00 GlobalSign board & leadership are debriefed by CISO and approve the plan of action: revoke the affected Trusted Root hierarchies (the GlobalSign hierarchies that contain third-party operated issuing CA keys outside of GlobalSign's controls) on July 8 16:00 UTC
July 3 2020 14:00 Incident team starts to work with customers affected by the initial revocatioApril 21 2021n activities to re-issue of certificates under alternate hierarchies or prepare for swapping their issuing CA with a new one to be generated on July 5 2020
July 5 2020 09:00 Key ceremony to generate new issuing CA for affected customers under Trusted Root hierarchies
July 5 2020 11:00 Key ceremony concluded, start setting up new issuing CA and re-issuance activities for customers
July 7 2020 12:00 Second board & leadership debriefing and discussion on remediation activities for internal CA
July 7 2020 13:00 GlobalSign receives official letter from a regional infrastructure provider in Japan, which has leafs under the "JCAN Public CA1 - G4" (https://crt.sh/?caid=51044), chained to the "Trusted Root CA SHA256 G2" (https://crt.sh/?caid=1423). The letter detailed that a revocation of the hierarchy on July 8 2020 UTC would have significant impact on the companies' ongoing relief and recovery efforts related to restoring their services in the context of the ongoing flooding disaster on southern part of Japan.
July 8 2020 01:00 GlobalSign board meets and discusses the letter received from a regional infrastructure provider in Japan and decides to postpone the revocation of Trusted Root hierarchies by 72 hours.
July 8 2020 16:00 First batch of revocations of affected issuing CA as detailed in comment #7
July 11 2020 16:00 Second batch of revocations including the Trusted Root hierarchies, as detailed in comment #8
July 15 2020 12:00 Presentation to board & leadership team and approval of full and final remediation plan

Whether your CA has stopped, or has not yet stopped, certificate issuance or the process giving rise to the problem or incident. A statement that you have stopped will be considered a pledge to the community; a statement that you have not stopped requires an explanation.

GlobalSign has ceased including the OCSP signing EKU in any newly generated issuing CA.

In a case involving certificates, a summary of the problematic certificates. For each problem: the number of certificates, and the date the first and last certificates with that problem were issued. In other incidents that do not involve enumerating the affected certificates (e.g. OCSP failures, audit findings, delayed responses, etc.), please provide other similar statistics, aggregates, and a summary for each type of problem identified. This will help us measure the severity of each problem.

The below table lists all affected active and revoked, but unexpired, issuing CA chained to WebPKI roots. To illustrate the full impact of the issue and contextualize remediation activities the overview also includes any "sibling" CA that share the same keypair as one affected by the OCSP EKU, and issuing CA which have an affected parent CA.

ID CN SHA256 Reason for being affected crt.sh
1 AbbVie AATL ICA 2020 154E4834B28D4FB1F90FEE935D0DDE46C45A177FC1425A028C685C32855A85AD Contains OCSP EKU https://crt.sh/?id=2369948023
2 CRB Group SMIME CA 2019 6A5F4C1678CA65E59F060D57CDFF665065314861D53A8E7D1450CA92D96CA102 Contains OCSP EKU https://crt.sh/?id=2029982659
3 DexKo Global SMIME CA 2019 ABC86706C98D6BF67372F908EC01ADF631B191D733AE89F8343EB047B108144B Contains OCSP EKU https://crt.sh/?id=2029984306
4 GlobalSign AATL Partners CA 2019 83FC891B350D9E0D7EBE6DD2A6BFE3D0B0F4653FCA048615A5DEEBBC039A3F66 Contains OCSP EKU https://crt.sh/?id=1436918881
5 GlobalSign Issuing CA for AATL Partners 2019 67C46DC17762667844F1596089375FF45E05C2B316C89499F6E7FAB78C8F0379 Contains OCSP EKU https://crt.sh/?id=1703475173
6 GlobalSign Qualified Time Stamping CA 2019 74ABE5E5CCEB75491FF72C4CF325405D8ADBFE390E189CF430BA60E62798878E Contains OCSP EKU https://crt.sh/?id=1490728721
7 Qu\C3\A1litas Compa\C3\B1\C3\ADa de Seguros S.A. de C.V. B716B089FE4E53D1A2EF7BA57AC85E68EC722CF61052C25A59626AD3B15C5F40 Contains OCSP EKU https://crt.sh/?id=1814826066
8 Ford Motor Company - Enterprise Issuing CA01 4C241CFE3D3FFB60CA88D6B06A552AB1CF0EF7D8D2E08DA15282B55192EBBD29 Contains OCSP EKU https://crt.sh/?id=392882654
9 Ford Motor Company - Enterprise Issuing CA01 3802E424516F78EEAC329AAE9B1F60A412DBE1D5B095D7AC9DC0DCDDE3C1F5FB Contains OCSP EKU https://crt.sh/?id=306624237
10 GlobalSign CodeSigning CA - G3 4047C9D69260C07213BCB8608A7EC5E2838A56B79F67847812EAC0778D0D27F1 Contains OCSP EKU https://crt.sh/?id=157564305
11 GlobalSign PersonalSign 1 CA - G3 F068DEAA18CC02D5A8BE35CB8338327910291F6E62E7216A934764A1ABA4A800 Contains OCSP EKU https://crt.sh/?id=2369948051
12 GlobalSign PersonalSign 2 CA - G3 925EE7D5A22AD7FBE9BAB54D7C8D0B9A74F7E35A8AF6AF645E2E8C3519A7092F Contains OCSP EKU https://crt.sh/?id=2369947954
13 GlobalSign PersonalSign 2 CA - SHA256 - G3 B778748A792B8F91F04B01BAFC31A31ED7EF6A712AFF80B6610D9AADEE207ADF Contains OCSP EKU https://crt.sh/?id=24592899
14 GlobalSign PersonalSign 2 CA - SHA256 - G4 27D6FDAF80297846DFEFF82E7F58B9A48AC9E3EE93A112B1BBE243EE1A97447C Contains OCSP EKU https://crt.sh/?id=2839140428
15 GlobalSign PersonalSign 2 ECC CA SHA 384 - G4 46038F6326228CDB56619C52266613DA04C8CA499E0D03B0EDCFFC110D5CFC70 Contains OCSP EKU https://crt.sh/?id=405618313
16 GlobalSign PersonalSign 2 RSA CA SHA 384 - G4 5CDD809CF44F5F8665EAC15055504C5B06B787AC18294505BDBAB4A77E50D776 Contains OCSP EKU https://crt.sh/?id=405618295
17 GlobalSign PersonalSign 3 CA - G3 B1FE3AEBF963A7880E74B0B0556681EA8B1CCCE3E69A7D3B10A68ACBE86E48A1 Contains OCSP EKU https://crt.sh/?id=2369948436
18 GlobalSign SMIME CA 2018 C8192C32F7B49C7F32A1CA001595A7F9E36C9E72058D6EAA1BAB7752A8C16718 Contains OCSP EKU https://crt.sh/?id=549505576
19 JCAN Public CA1 - G4 7B464DC384FDB1A525C2CC279ED0C7CFAD24BECF72C46A7D7093D157C217607E Contains OCSP EKU https://crt.sh/?id=163676419
20 NAESB Issuing CA - SHA384 - G3 128DED1A8AD60C24B4254E31DB94FC4392BF93ED5434472AA43A0B9856106068 Contains OCSP EKU https://crt.sh/?id=2369948019
21 NAESB Issuing CA - SHA384 - G3 0986B5A1C7314EFB04FB648B9E2B57CF4842FD1D4345D28E52094C90A9FECBFE Shares private key with OCSP EKU containing issuing CA #20 https://crt.sh/?id=18068129
22 SHECA DV Secure Server CA 393B8B15CABC3886FB2E416495D63C8BADD8DCAF87552076C8A0A9637C24DE47 Contains OCSP EKU https://crt.sh/?id=1225556701
23 SHECA EV Secure Server CA 147C447FEEB86202B503314FCAF0036BEAAEF437C39B56B358EC446A9D20387F Contains OCSP EKU https://crt.sh/?id=1229139434
24 SHECA OV Secure Server CA 77EAC476453CB732257FF166A5EBD1656CB1F673B68E28DF41774133979FA2A4 Contains OCSP EKU https://crt.sh/?id=1225556702
25 ATT Organization Validated CA 2019 7AA45D6F5B14DAB1C6844C19C2804E14B5811E6EDE1F02B0AEF065A7B359C68F Contains OCSP EKU https://crt.sh/?id=1490728430
26 CrowdStrike OV SSL Issuing CA 2020 AE03B9AD17106A28785830B1DCD636797C4C64D81CB8D161595DBAF83433E64C Contains OCSP EKU https://crt.sh/?id=2839140453
27 DPDHL Global TLS CA - I4 94C663E9EA5C27EE4F64127F9B425863E991A9E156C07DF1A00803AE31764162 Contains OCSP EKU https://crt.sh/?id=1814823951
28 DPDHL User CA I3 AF1898D7F0638751C075D0142D4E2A0EA731FC622324F153FE1BF3B6AFD9AF13 Contains OCSP EKU https://crt.sh/?id=1596016275
29 DPDHL User CA I3 2E0191751CA0CBA81C3A6338DEE1A02B8D6BCC4F1F8261B809BCCE7ABAF1A43D Contains OCSP EKU https://crt.sh/?id=12729527
30 DPDHL User CA I3 BCE3A5BD8D9082636C5BFE3E0B71ACEE551E24E3BD035887D2661ADA65AFF484 Contains OCSP EKU Not in crt.sh
31 DPDHL User CA I3 F037621405E0F356507E239FADD647842D3B50857C3CFF840859174F72F6FD18 Contains OCSP EKU https://crt.sh/?id=329514052
32 DPDHL User CA I4 C25C4EDBC36E3FB7C3D937BEE9F2D29E36AFB07CFA3188262E0D5FDC919E0D77 Contains OCSP EKU https://crt.sh/?id=2369948075
33 Giesecke and Devrient CA 632FD697BACAF1ED232517EC9B7622B7C25E1448B0CC626B33286719E351CE8A Contains OCSP EKU https://crt.sh/?id=196919504
34 GlobalSign CA 4 for AATL EBA34C7B109671614C367E1DE075124C3954CE19F85FACF61090EC319F7F1A7F Contains OCSP EKU https://crt.sh/?id=1229139435
35 GlobalSign CA 4 for AATL 9DDC2E0D55B461E0C73228282DF56B2BEF224CA2385681D17B6E8C077852573C Contains OCSP EKU https://crt.sh/?id=405831326
36 GlobalSign CA 5 for AATL 306E9739E3458FF4546877B704B2E3905E58B235D64E32F4F026AC91B7295D15 Contains OCSP EKU https://crt.sh/?id=408789250
37 GlobalSign CA 6 for AATL BE1FFC0E1FF6088104F43E327E7C7DC72A9CA7B0DF05793123ABE32DEACEE76F Contains OCSP EKU https://crt.sh/?id=2369988390
38 GlobalSign CA for AATL - SHA256 - G3 2E8820DC0EAFAE3D6D285C057ECE14470B377438B002CEDD4C72B4F343A54F43 Contains OCSP EKU https://crt.sh/?id=2369947889
39 GlobalSign CodeSigning CA - SHA256 - G2 BE40813869AB27A071D12AD6A8830583EBC3B618E3F2346359F4B11A1C9434EE Contains OCSP EKU https://crt.sh/?id=1703475054
40 GlobalSign CodeSigning CA - SHA256 - G3 FB54EEA9BCE8E9EA9782154F3D414277FB709F49B947D73978AC278546C2CE03 Contains OCSP EKU https://crt.sh/?id=26749929
41 GlobalSign ECC EV SSL CA 2019 0D3176C58F321AA34C57C8DF7C17D1F4E76C797EC116C9F1D697748ED1FCE7D9 Contains OCSP EKU https://crt.sh/?id=2329203344
42 GlobalSign Extended Validation CodeSigning CA - SHA256 - G2 1E864278C20881B671C0C6D2E14B61150AD1F13CF92C6EC14B550DCBC47E1541 Contains OCSP EKU https://crt.sh/?id=1703475088
43 GlobalSign Extended Validation CodeSigning CA - SHA256 - G3 DD038E87E0B4D2C369680D3DE78638AB39FC1D7E50632996921101768DB8D4D8 Contains OCSP EKU https://crt.sh/?id=41285443
44 GlobalSign HV ECC DV SSL CA 2018 4B0D1392D39157353207A64CCB14683DDE9D2CED1FB58B16E038BE5707C27813 Contains OCSP EKU https://crt.sh/?id=970083107
45 GlobalSign HV RSA DV SSL CA 2018 54C37A8E853FD1D6378D378B939307EC321A31CC1A5A89E7180633BC13F18762 Contains OCSP EKU https://crt.sh/?id=970082980
46 GlobalSign PersonalSign 1 CA - SHA256 - G3 F5D2D2BA6817A7A9AA0E21354BBF0E6F95C5E287EE88CF2F279F0FFEC4EDAC15 Contains OCSP EKU https://crt.sh/?id=147619379
47 GlobalSign PersonalSign 3 CA - SHA256 - G3 701B432AC0CDD4D9CF95B4B884C32BF5CCA90D44E0161ABD13B934D68E380472 Contains OCSP EKU https://crt.sh/?id=163079175
48 GlobalSign PersonalSign Partners CA - SHA256 - G2 4E707867946AC05343C6BA8FF121EA66A758037913257A8EE4974350D39A1034 Contains OCSP EKU https://crt.sh/?id=2369948041
49 GlobalSign PersonalSign Partners CA - SHA256 - G2 C8F1D691B4152C26033C977FE77978D9C82143D46B243B9C9BA7228E000E15BB Shares private key with OCSP EKU containing issuing CA #48 https://crt.sh/?id=12721528
50 GlobalSign PersonalSign Partners CA - SHA256 - G2 118262C2088EE1528E20D836D2070854707C0D8F8E80FBE396F9ECD4B9141B5B Shares private key with OCSP EKU containing issuing CA #48 https://crt.sh/?id=12715740
51 GlobalSign Qualified CA 1 F5709A2D2F68B53BF6F645BB178ADF95346F89FDA5C63BFDE08042A26492AAB2 Contains OCSP EKU https://crt.sh/?id=509714293
52 GlobalSign Qualified CA 2 FD3A0F3DD4480092B6D450473DEB9201A0B308A8807833A3C738F8A07EB81ED3 Contains OCSP EKU https://crt.sh/?id=509714291
53 GlobalSign Qualified CA 3 0AA9F2E7D95C718B7D1EB7CCDBD0164E86057AE9D66922BC60F9903F94A0F0EF Contains OCSP EKU https://crt.sh/?id=509714292
54 GlobalSign Qualified Timestamping ECC CA 2020 C2FEACD674878C7B0C2325A2ECED0A333DB7780A86DFEC3758100EFC0101C665 Contains OCSP EKU https://crt.sh/?id=2839140405
55 GlobalSign R6 Admin CA - SHA256 - G3 C5B679106958152F83FB5886DDC41F0785193EF67C6975BE3E509F17F29B7A86 Contains OCSP EKU https://crt.sh/?id=164243753
56 GlobalSign R6 RSA EV SSL CA 2019 57264B82A864DBA1C11EF3F80ABB94CAC3660662B0C22F571FF993B3FBCF76FB Contains OCSP EKU https://crt.sh/?id=1476654013
57 GlobalSign RSA DV SSL CA 2018 9E898ED03FA46969690DAD73C7296675045FF9B5A0100A399BEB8435A98F5185 Contains OCSP EKU https://crt.sh/?id=970083106
58 GlobalSign RSA EV QWAC CA 2019 EDC734C501501DC7A27448FA02C74931F8578BF297B173F34B841E82C6691926 Contains OCSP EKU https://crt.sh/?id=1490728500
59 GlobalSign RSA EV SSL CA 2019 0D6E46784F3B694E9C7506786417BC6F87F9D2F73D19B5E8081612B21137B766 Contains OCSP EKU https://crt.sh/?id=2220986544
60 GlobalSign Timestamping CA - G3 95C6A747DD0BC755A1941827E894B8083592241B792541E2EB1B30FB9B13F57F Contains OCSP EKU https://crt.sh/?id=2392141070
61 GlobalSign Timestamping CA - SHA256 - G3 BE33D1C57EBDDD927B57BDB604BE457B552FE568E7F3DCBA093C39ED1C30A239 Contains OCSP EKU https://crt.sh/?id=2369948437
62 LinQuest SMIME CA 2020 113138DD7B216725840238E2D7EEECB3738DB139064B24CB853FC270A49E6057 Contains OCSP EKU https://crt.sh/?id=2369948433
63 NAESB Issuing CA - SHA384 - G4 C4C7C436BD88E8E68DB00297DF83ACC819E198639BA00522C8E3245876898523 Contains OCSP EKU https://crt.sh/?id=2369948432
64 RNP ICPEdu OV SSL CA 2019 42CFDDA6F660B8E5B4C1C411965A4519312559E3262F8DB69D2DAE17B26B3BA3 Contains OCSP EKU https://crt.sh/?id=1476651440
65 Trafigura PTE Ltd S/MIME ICA 2020 5E7FCB9C97BDA56993B1658D120232761D665A3644534300FA6A5BEC5E0D5795 Contains OCSP EKU https://crt.sh/?id=2369948428
66 GlobalSign CodeSigning CA - G2 FFFE077503FD72F0E5338B0A7B4E218E7D1FF82E493E7E852AE51AA1C7585D17 Contains OCSP EKU https://crt.sh/?id=1476651569
67 DPDHL User CA I3 EBE87BB4188502709F444055259ABB22BC51B88C908419A13559DFC8EF6630D1 Contains OCSP EKU https://crt.sh/?id=12729526
68 Ford Motor Company - Enterprise Issuing CA01 CF73B52D041B7309B439D16247414B90C9D26E44E38748A36500D5829B5187F9 Contains OCSP EKU https://crt.sh/?id=215376217
69 Ford Motor Company - Enterprise Issuing CA01 3B9668F59F55FA3838FC2A3B80B7F9B5B13D1A46F1EAA6E0BCFF04C54198056C Contains OCSP EKU https://crt.sh/?id=215376215
70 GlobalSign CA for AATL on HV 1C9266902A31C3941B506D44D0D4D06EC9DB7655E65F9557659FAB768B290B1B Contains OCSP EKU https://crt.sh/?id=1119260014
71 GlobalSign CA for AATL on HV BCBD04D4AED962C9D25AFE0CFAF8638CE1431652988EC5217329E7559AC3C671 Contains OCSP EKU https://crt.sh/?id=163322577
72 GlobalSign PersonalSign 1 CA - G3 254BE91C1ABCB28DB5E4D675A29A1E788460B06591F1BA8497CBD17837E27ABE Shares private key with OCSP EKU containing issuing CA #11 https://crt.sh/?id=18068232
73 GlobalSign PersonalSign 2 CA - G3 64E71601F7050921DEE039C03493615E488F12FC3FCECBADF438AA467EE1D41A Shares private key with OCSP EKU containing issuing CA #12 https://crt.sh/?id=17569373
74 GlobalSign PersonalSign 3 CA - G3 C228D93DBE5536A120AC24ED934467BAD7292F8B7EB202634B17070A89C5FE9B Shares private key with OCSP EKU containing issuing CA #17 https://crt.sh/?id=18068118
75 Liberty University External Issuing CA 01 1F91212C6BFC333C6EB52A685525E1E5B9E3AC1EF7A5A86649F5F95C721D8898 Contains OCSP EKU Not in crt.sh
76 Liberty University External Issuing CA 01 CA005AA75E33594BD1DEDC584E1E74E5198EBB1DE88929ED4F3E2E9FFCE3873B Contains OCSP EKU https://crt.sh/?id=36391364
77 MSC Trustgate.com RSA AATL CA 2019 3A882530C03EA615E5EF4DADBD7C8660912FA93FAF5088716FB46A8E1FFA9218 Contains OCSP EKU https://crt.sh/?id=1119259389
78 VWFS CA for AATL CF89A41DFEE5F71740DEF602735DDBF1DEBE0CB816D73980D9A583C5881CE778 Contains OCSP EKU https://crt.sh/?id=163688490
79 Crown Prince Court CA BF5EDFBEEB85999C5169CBF3F4DB63B679AD2E1E2272FC3795F9F9921E6D0487 Contains OCSP EKU https://crt.sh/?id=7890405
80 Crown Prince Court CA A0133BE5B14E02310A2D4BEAB601094F1194EE8BD6FD29DDFE7B9347467C2EEC Contains OCSP EKU https://crt.sh/?id=10105729
81 Crown Prince Court CA F164AD5E4CE9EFC0A144CA902EA2ED46C464D2D508CA919A23095CDF30D4DC68 Contains OCSP EKU Not in crt.sh
82 Crown Prince Court CA for AATL DF45EEAED905C58D730EC5497B59B3AB4CCE7C6459953DF9CA5C1F031AC06DD8 Contains OCSP EKU Not in crt.sh
83 Crown Prince Court CA for AATL D21076207F79A9B04137D40A4FFE6DD08921CCF49E6EB60277FF4593E076D538 Contains OCSP EKU Not in crt.sh
84 Crown Prince Court CA for AATL 70BDB19C31F5EF105B29376E35EA5ED8EEBE13CB5C0758C32DFC4C5F7230A173 Shares private key with OCSP EKU containing issuing CA #86 Not in crt.sh
85 DPDHL TLS CT CA I3 9153E4420DDC7EB4E6E864AA0377DADF4082ECD35052113638E05D3C296BC006 Contains OCSP EKU https://crt.sh/?id=786990298
86 DPDHL TLS SHA 2 CA I3 25BACC40A5392B82AADEA04903905A467121F28220E6F2F7E0FE982AAFC14FA6 Contains OCSP EKU https://crt.sh/?id=8527797
87 DPDHL TLS SHA 2 CA I3 5A405535C112A0A81AF0D2ACCA3C3F9BC1A677586CDBC633CB4F5F778E1A3550 Contains OCSP EKU https://crt.sh/?id=329514048
88 DPDHL TLS SHA 2 CA I3 23A74704D77A03CFD3FF19E62C500848214E6C60FD2AAEF7DCE7A8F9EE9F9232 Contains OCSP EKU https://crt.sh/?id=27823827
89 DPDHL TLS SHA 2 CA I3 1C942A22A016A1E5559DAE77EC5CE8671F98AE0BA4AC2DC259418E8E1E9F94AD Contains OCSP EKU https://crt.sh/?id=9881176
90 Southern Company External Issuing CA 1 FB953C4FC0045846D02491C8ECCF387BA34347C17ABB0EA6D59F6DE4D2F1EA04 Contains OCSP EKU https://crt.sh/?id=11501550
91 Trusted Root CA G2 6E32A35B599E9087BB1AB35CE73022EC2E26AF34BE388919419C95700CD8E7FB Has child CAs operated by third parties containing OCSP EKU https://crt.sh/?id=1862521
92 Trusted Root CA SHA256 G2 01FD73EF5E70F526FC9C11F65FE2EE6F7125B3693949227FFD8E459E583C458A Has child CAs operated by third parties containing OCSP EKU https://crt.sh/?id=3179271
93 CBMM SMIME CA 2019 C346D9137E05254C6EEAC99AC2F6748A0C5D3AFC6B7B9B1E00C40ADF4D85655D Parent CA contains OCSP EKU https://crt.sh/?id=1596016282
94 Accenture Federal Services External CA A9C8E971259A2ED6E65F721E07AA967C72C7CDB47C7BE1288D87BF08D2F3580D Parent CA contains OCSP EKU https://crt.sh/?id=215376216
95 EY LLP SHA256 CA - G2 1557F65BA61C958B74EFA4A582BBAEBDD62A6D9B65FE95A80D5ED518F46ED87F Parent CA contains OCSP EKU https://crt.sh/?id=970084237
96 GROB-WERKE GmbH und Co. KG SMIME CA 1608BF87414CDCFAB4279102A19702D9D5996A91329E3DF2F80495473AAD86C6 Parent CA contains OCSP EKU https://crt.sh/?id=872293772
97 Hyperion SMIME CA 2018 DCD77E34B46D530AEF645A513389CD4FFC0F7D196A115B8F62A5FD0D557D46C6 Parent CA contains OCSP EKU https://crt.sh/?id=721305503
98 Spirit AeroSystems SHA256 CA - G2 8609BDCEF95E4A4D426497B5CD8ED4B001C953A5C14471CAAF58FB650DF8ABF0 Parent CA contains OCSP EKU https://crt.sh/?id=215376218
99 HERTZ SHA256 CA - G2 64FD7F66B805EF0FAE09DAC06EAD1A9AB5C28E6F24AD0759996D349987FEE7E2 Parent CA contains OCSP EKU https://crt.sh/?id=408789253

In a case involving certificates, the complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem. In other cases not involving a review of affected certificates, please provide other similar, relevant specifics, if any.

See above.

Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

In the past, in the context of customers that operated their own CA within the GlobalSign hierarchy (branded Trusted Root), GlobalSign has been confronted with multiple software packages that refused to properly operate if the OCSP-Signing EKU was not being present in the full chain, including Microsoft Active Directory Certificate Services (ref. https://bugzilla.mozilla.org/show_bug.cgi?id=991209#c3) and Ascertia's ADSS. After discussion it was decided to include this EKU in some issuing CAs where required, so long as this was never to be combined with digitalSignature KU. At the time of discussion and risk assessment, it was concluded that OCSP responses would not validate if the digitalSignature KU was absent.

In December 2018, as a result of internal questions related to browser requirements for new issuing CAs for 2019 and beyond, the decision was made to add digitalsignature KU in all ICA with keys under GlobalSign's control in order that they could potentially be used as direct OCSP signers (with response pre-generation) at some time in the future. It was perceived that the id-pkix-ocsp-nocheck extension was not required due to fact these were never considered to be or ever set up as delegated responders.

From January 2019 the requirement to include at least one EKU in all issuing CAs came into force. The combination of requiring at least one EKU and the concepts of nested EKU and EKU constraining lead to GlobalSign misinterpreting that OCSP Signing EKU also had to be inserted in issuing CAs to respect EKU chaining. The security implications of these two additions together were discussed and it was agreed to only include both digitalsignature KU and OCSP Signing EKU in issuing CAs with keys under GlobalSign's sole control, without fully realizing the impact this could have on the reliability of certificate status information provided by the parent if ever keys of issuing CA that include the OCSP Signing EKU were compromised.

Knowing that this does not provide any assurance, here are some additional insights into our environment:

  • Even if they were intended to be potentially used for that purpose in the long term (OCSP response pre-generation directly from the CA), none of the affected issuing CA and associated keys have effectively been used to sign OCSP responses, we use dedicated OCSP responder systems, certificates, keys and HSMs for that purpose.
  • Our CA issuance systems are logically set up in such a way that only outbound push/pull queuing from the CA issuance segment to another network segment is possible. No other interactions between the CA issuance segment and other networks are possible.
  • CA issuance systems are stripped of any unnecessary functionality and do not contain the necessary software to generate OCSP responses (short of manual generation using pkcs11-tool or openssl, which would be captured in the audit logs)
  • Apart from logical segmentation our CA issuance systems and HSMs are physically in a separate zone (room) with multi-person access requirements, preventing the CA keys to be exposed or exported to other systems such as OCSP responders.
  • We maintain configuration records and system change logs for all systems involved in CA operations
  • Each key signing action on CA issuance systems is logged

We are currently in discussion with qualified WebTrust auditors to seek for any independent assurance an auditor can provide within the context of this incident and controls described above.

List of steps your CA is taking to resolve the situation and ensure that such situation or incident will not be repeated in the future, accompanied with a binding timeline of when your CA expects to accomplish each of these remediation steps.

Apart from removing the OCSP signing EKU from our issuing CA template profiles, we are performing following changes to our internal processes by July 31 to prevent such a compliance/security issues to be overlooked in the future:

  • Modify the CA creation and CA profile change process so that an additional PKI compliance officer needs to approve (expanding from two to three PKI compliance officers)
  • Modify the CA creation and CA profile change process so that any change to default profile requires a peer comparison with similar issuing CA from 3 other parties that acts as a WebPKI CA.

Additional to the above, as of October 2019, we have started building an internal repository of all external (root program / Bugzilla) communication so that communication, discussions and risks are captured for future consideration and decision making.

The below table provides an overview of the remediation plans for all affected CA. As introduction to the remediation details, previously described in https://bugzilla.mozilla.org/show_bug.cgi?id=1599788#c4, we have just created our future generation of roots and started embedding them within the relevant root programs. As part of remediating this incident we will be moving a considerable amount of the affected issuing CA to non-TLS roots. More details on the reasons we could not complete revocation activities within 7 days and on our future hierarchy and segregation will be discussed on the associated delayed revocation bug https://bugzilla.mozilla.org/show_bug.cgi?id=1651447. A list of untreated CA that can be added to OneCRL will be provided in the next days.

The key destruction date refers to the date on which the primary copy of the issuing CA key, loaded in the active issuance data center, will be destroyed. Each issuing CA key effectively corresponds to five copies of the key: one in the active issuance data center, one in the stand-by issuance data center, and three in cold storage for back-up, continuity & emergency revocation purposes. Since these copies are located across three different continents, multiple key destruction ceremonies will be executed around the time of the destruction of the primary copy but varying with a few days depending on locational context, logistics and availability of local destruction ceremony attendants and auditor.

ID CN crt.sh Treatment Revocation date Destruction date Additional Information
1 AbbVie AATL ICA 2020 https://crt.sh/?id=2369948023 Revoke & destroy October 21 2020 October 21 2020 We are creating and embedding new AATL CA under a non-TLS hierarchy, once embedded we will create a new CA under the new hierarchies, replace leaf certificates and revoke the current one.
2 CRB Group SMIME CA 2019 https://crt.sh/?id=2029982659 Revoke & destroy October 21 2020 October 21 2020 We are creating and cross-signing new S/MIME hierarchies under a non-TLS hierarchy, we will create a new CA under the new hierarchies, replace leaf certificates and revoke the current one.
3 DexKo Global SMIME CA 2019 https://crt.sh/?id=2029984306 Revoke & destroy November 18 2020 November 18 2020 We are creating and cross-signing new S/MIME hierarchies under a non-TLS hierarchy, we will create a new CA under the new hierarchies, replace leaf certificates and revoke the current one.
4 GlobalSign AATL Partners CA 2019 https://crt.sh/?id=1436918881 Revoke & destroy October 21 2020 October 21 2020 We are creating and embedding new AATL CA under a non-TLS hierarchy, once embedded we will create a new CA under the new hierarchies, replace leaf certificates and revoke the current one.
5 GlobalSign Issuing CA for AATL Partners 2019 https://crt.sh/?id=1703475173 Revoke & destroy October 21 2020 October 21 2020 We are creating and embedding new AATL CA under a non-TLS hierarchy, once embedded we will create a new CA under the new hierarchies, replace leaf certificates and revoke the current one.
6 GlobalSign Qualified Time Stamping CA 2019 https://crt.sh/?id=1490728721 Destroy N/A July 28 2020 This is a timestamping CA with a few TSA leafs with keys under GlobalSign control. Because revocation would affected all previously issued timestamps we will destroy the issuing CA keys, and TSU keys once they are retired. We are creating and embedding new eIDAS-related CA under a non-TLS hierarchy.
7 Qu\C3\A1litas Compa\C3\B1\C3\ADa de Seguros S.A. de C.V. https://crt.sh/?id=1814826066 Revoke & destroy October 21 2020 October 21 2020 We are creating and cross-signing new S/MIME hierarchies under a non-TLS hierarchy, we will create a new CA under the new hierarchies, replace leaf certificates and revoke the current one.
10 GlobalSign CodeSigning CA - G3 https://crt.sh/?id=157564305 Revoke & destroy April 21 2021 April 21 2021 This is one of our current primary CodeSigning CA. We are creating and cross-signing new CodeSigning hierarchies under a non-TLS hierarchy, we will create a new CA under the new hierarchies and revoke the current one. Replacing all affected leaf certificates will take time as they are partially provided on hardware tokens.
11 GlobalSign PersonalSign 1 CA - G3 https://crt.sh/?id=2369948051 Revoke & destroy July 28 2020 July 28 2020
12 GlobalSign PersonalSign 2 CA - G3 https://crt.sh/?id=2369947954 Revoke & destroy July 28 2020 July 28 2020
13 GlobalSign PersonalSign 2 CA - SHA256 - G3 https://crt.sh/?id=24592899 Revoke & destroy February 24 2021 February 24 2021 This CA contains has 400K active leaf certificates including certificates issued on hardware tokens.
14 GlobalSign PersonalSign 2 CA - SHA256 - G4 https://crt.sh/?id=2839140428 Revoke & destroy July 8 2020 July 28 2020
15 GlobalSign PersonalSign 2 ECC CA SHA 384 - G4 https://crt.sh/?id=405618313 Revoke & destroy July 28 2020 July 28 2020
16 GlobalSign PersonalSign 2 RSA CA SHA 384 - G4 https://crt.sh/?id=405618295 Revoke & destroy July 28 2020 July 28 2020
17 GlobalSign PersonalSign 3 CA - G3 https://crt.sh/?id=2369948436 Revoke & destroy July 28 2020 July 28 2020
18 GlobalSign SMIME CA 2018 https://crt.sh/?id=549505576 Revoke & destroy September 16 2020 October 21 2020 We are creating and cross-signing new S/MIME hierarchies under a non-TLS hierarchy, we will create a new CA under the new hierarchies, replace leaf certificates and revoke the current one.
20 NAESB Issuing CA - SHA384 - G3 https://crt.sh/?id=2369948019 Revoke & destroy July 28 2020 October 21 2020 Destruction timing influenced by sibling CA #21.
21 NAESB Issuing CA - SHA384 - G3 https://crt.sh/?id=18068129 Revoke & destroy September 30 2020 October 21 2020 Revocation date as detailed on https://bugzilla.mozilla.org/show_bug.cgi?id=1591005#c27
25 ATT Organization Validated CA 2019 https://crt.sh/?id=1490728430 Revoke & destroy September 30 2020 October 21 2020 TLS ICA dedicated to a single organization.
27 DPDHL Global TLS CA - I4 https://crt.sh/?id=1814823951 Revoke & destroy September 30 2020 October 21 2020 TLS Issuing CA dedicated to an organization. The organization is in the progress of setting up fully automated issuance and replacement of certificates and will need another 3 months to fully complete the process. As part of the automation excercise they also expect to mass re-issue certificates within 7 days in the future.
32 DPDHL User CA I4 https://crt.sh/?id=2369948075 Revoke & destroy November 18 2020 November 18 2020 Issuing CA dedicated to an organization with 300K leaf certificates for S/MIME and client authentication. The organization is in the progress of setting up fully automated issuance and replacement of certificates and will need another 3 months to fully complete the process.
34 GlobalSign CA 4 for AATL https://crt.sh/?id=1229139435 Revoke & destroy December 31 2020 January 20 2021 We are creating and embedding new AATL CA under a non-TLS hierarchy, once embedded we will create a new CA under the new hierarchies, replace leaf certificates and revoke the current one.
35 GlobalSign CA 4 for AATL https://crt.sh/?id=405831326 Revoke & destroy July 8 2020 January 20 2021 Key destruction date influenced by sibling #34
36 GlobalSign CA 5 for AATL https://crt.sh/?id=408789250 Revoke & destroy December 31 2020 January 20 2021 We are creating and embedding new AATL CA under a non-TLS hierarchy, once embedded we will create a new CA under the new hierarchies, replace leaf certificates and revoke the current one.
37 GlobalSign CA 6 for AATL https://crt.sh/?id=2369988390 Revoke & destroy December 31 2020 January 20 2021 We are creating and embedding new AATL CA under a non-TLS hierarchy, once embedded we will create a new CA under the new hierarchies, replace leaf certificates and revoke the current one.
38 GlobalSign CA for AATL - SHA256 - G3 https://crt.sh/?id=2369947889 Destroy N/A January 20 2021 Refer to https://bugzilla.mozilla.org/show_bug.cgi?id=1591005#c27. The sibling ICA using the same key pair has around 100 subordinate certificates in three categories: a) OCSP responder certificates, b) AATL TSA certificates, c) subordinate ICA that effectively sign leaf AATL certificates. Included in c) are the "GlobalSign CA 2 for AATL" and "GlobalSign CA 3 for AATL" which are also impacted by the incident. There are approximately 700.000 active leaf certificates issued by the ICA under c). After the leaf certificates under c) are addressed we will revoke all c) subordinate ICA but not revoke the subject ICA ("GlobalSign CA for AATL - SHA256 - G2") because of the AATL TSA certificates, which have been used to provide Timestamps for document signatures provided by any leaf of the subject ICA. If we revoke the subject ICA any signature time-stamped through this hierarchy will be rendered invalid. The subject ICA keys will hence be destroyed after all subordinate ICAs are revoked.
39 GlobalSign CodeSigning CA - SHA256 - G2 https://crt.sh/?id=1703475054 Revoke & destroy October 21 2020 October 21 2020
40 GlobalSign CodeSigning CA - SHA256 - G3 https://crt.sh/?id=26749929 Revoke & destroy April 21 2021 April 21 2021 This is one of our current primary CodeSigning CA. We are creating and cross-signing new CodeSigning hierarchies under a non-TLS hierarchy, we will create a new CA under the new hierarchies, replae all certificates and revoke the current one. Replacing all affected leaf certificates will take time as they are partially provided on hardware tokens.
41 GlobalSign ECC EV SSL CA 2019 https://crt.sh/?id=2329203344 Revoke & destroy July 8 2020 July 28 2020
42 GlobalSign Extended Validation CodeSigning CA - SHA256 - G2 https://crt.sh/?id=1703475088 Revoke & destroy October 21 2020 October 21 2020
43 GlobalSign Extended Validation CodeSigning CA - SHA256 - G3 https://crt.sh/?id=41285443 Revoke & destroy April 21 2021 April 21 2021 This is our current primary EV CodeSigning CA. We are creating and cross-signing new CodeSigning hierarchies under a non-TLS hierarchy, we will create a new CA under the new hierarchies and revoke the current one. Replacing all affected leaf certificates will take time as they are all provided on hardware tokens.
44 GlobalSign HV ECC DV SSL CA 2018 https://crt.sh/?id=970083107 Revoke & destroy July 8 2020 July 28 2020
45 GlobalSign HV RSA DV SSL CA 2018 https://crt.sh/?id=970082980 Revoke & destroy July 8 2020 July 28 2020
46 GlobalSign PersonalSign 1 CA - SHA256 - G3 https://crt.sh/?id=147619379 Revoke & destroy February 24 2021 February 24 2021 We will replace the certificates under other, non-affected hierarchies and revoke the ICA as soon as these activities are completed.
47 GlobalSign PersonalSign 3 CA - SHA256 - G3 https://crt.sh/?id=163079175 Revoke & destroy February 24 2021 February 24 2021 We will replace the certificates under other, non-affected hierarchies and revoke the ICA as soon as these activities are completed.
48 GlobalSign PersonalSign Partners CA - SHA256 - G2 https://crt.sh/?id=2369948041 Revoke & destroy January 20 2021 January 20 2021 Refer to https://bugzilla.mozilla.org/show_bug.cgi?id=1591005#c27
49 GlobalSign PersonalSign Partners CA - SHA256 - G2 https://crt.sh/?id=12721528 Revoke & destroy February 19 2020 January 20 2021 Refer to https://bugzilla.mozilla.org/show_bug.cgi?id=1591005#c27. Key destruction date impacted by sibling #48.
50 GlobalSign PersonalSign Partners CA - SHA256 - G2 https://crt.sh/?id=12715740 Revoke & destroy September 30 2020 January 20 2021 Refer to https://bugzilla.mozilla.org/show_bug.cgi?id=1591005#c27. Key destruction date impacted by sibling #48.
51 GlobalSign Qualified CA 1 https://crt.sh/?id=509714293 Revoke & destroy February 24 2021 February 24 2021 Our primary CA used for issuing eIDAS qualified certificates for electronic signatures and electronic seals. We are creating and embedding new eIDAS-related CA under a non-TLS hierarchy, as soon as they are on the EU TL we will migrate all customers. All the certificates issued by this CA or on a QSCD.
52 GlobalSign Qualified CA 2 https://crt.sh/?id=509714291 Revoke & destroy July 8 2020 July 28 2020
53 GlobalSign Qualified CA 3 https://crt.sh/?id=509714292 Revoke & destroy July 8 2020 July 28 2020
54 GlobalSign Qualified Timestamping ECC CA 2020 https://crt.sh/?id=2839140405 Revoke & destroy July 8 2020 July 28 2020
55 GlobalSign R6 Admin CA - SHA256 - G3 https://crt.sh/?id=164243753 Revoke & destroy July 8 2020 July 28 2020
56 GlobalSign R6 RSA EV SSL CA 2019 https://crt.sh/?id=1476654013 Revoke & destroy July 8 2020 July 28 2020
57 GlobalSign RSA DV SSL CA 2018 https://crt.sh/?id=970083106 Revoke & destroy January 20 2021 January 20 2021 Our primary DV TLS issuing CA on our legacy platform, used mostly to issue DV certificates to retail customers, without any current form of automation. Has 375K actife leaf certificates that need replacement.
58 GlobalSign RSA EV QWAC CA 2019 https://crt.sh/?id=1490728500 Revoke & destroy February 24 2021 February 24 2021 Our primary CA used for issuing eIDAS qualified web authentication certificates. We are embedding new eIDAS-related QWAC, as soon as it is on the EU TL we will migrate all customers.
59 GlobalSign RSA EV SSL CA 2019 https://crt.sh/?id=2220986544 Revoke & destroy July 8 2020 July 28 2020
62 LinQuest SMIME CA 2020 https://crt.sh/?id=2369948433 Revoke & destroy November 18 2020 November 18 2020 Client Authentication & S/MIME ICA dedicated to a single organization.
63 NAESB Issuing CA - SHA384 - G4 https://crt.sh/?id=2369948432 Revoke & destroy February 24 2021 February 24 2021 We almost finished moving all customers to this CA in the context of remediating https://bugzilla.mozilla.org/show_bug.cgi?id=1591005#c27. We will create a new CA under non-TLS roots, embed it into the NAESB ecosystem and re-issue the certificates.
65 Trafigura PTE Ltd S/MIME ICA 2020 https://crt.sh/?id=2369948428 Revoke & destroy July 11 2020 July 28 2020
66 GlobalSign CodeSigning CA - G2 https://crt.sh/?id=1476651569 Revoke & destroy July 28 2020 July 28 2020
70 GlobalSign CA for AATL on HV https://crt.sh/?id=1119260014 Revoke & destroy February 19 2020 July 28 2020
71 GlobalSign CA for AATL on HV https://crt.sh/?id=163322577 Revoke & destroy February 19 2020 July 28 2020
72 GlobalSign PersonalSign 1 CA - G3 https://crt.sh/?id=18068232 Revoke & destroy June 30 2020 February 24 2021 Sibling of #11
73 GlobalSign PersonalSign 2 CA - G3 https://crt.sh/?id=17569373 Revoke & destroy June 30 2020 February 24 2021 Sibling of #12
74 GlobalSign PersonalSign 3 CA - G3 https://crt.sh/?id=18068118 Revoke & destroy June 30 2020 February 24 2021 Sibling of #17
77 MSC Trustgate.com RSA AATL CA 2019 https://crt.sh/?id=1119259389 Revoke & destroy February 19 2020 July 28 2020
78 VWFS CA for AATL https://crt.sh/?id=163688490 Revoke & destroy May 20 2020 July 28 2020
93 CBMM SMIME CA 2019 https://crt.sh/?id=1596016282 Revoke & destroy November 18 2020 November 18 2020 Client Authentication & S/MIME ICA dedicated to a single organization.
94 Accenture Federal Services External CA https://crt.sh/?id=215376216 Revoke & destroy December 31 2020 January 20 2021 Client Authentication & S/MIME ICA dedicated to a single organization.
95 EY LLP SHA256 CA - G2 https://crt.sh/?id=970084237 Revoke & destroy December 31 2020 January 20 2021 Client Authentication & S/MIME ICA dedicated to a single organization.
96 GROB-WERKE GmbH und Co. KG SMIME CA https://crt.sh/?id=872293772 Revoke & destroy October 21 2020 October 21 2020 Client Authentication & S/MIME ICA dedicated to a single organization.
97 Hyperion SMIME CA 2018 https://crt.sh/?id=721305503 Revoke & destroy November 18 2020 November 18 2020 Client Authentication & S/MIME ICA dedicated to a single organization.
98 Spirit AeroSystems SHA256 CA - G2 https://crt.sh/?id=215376218 Revoke & destroy November 18 2020 November 18 2020 Client Authentication & S/MIME ICA dedicated to a single organization.
99 HERTZ SHA256 CA - G2 https://crt.sh/?id=408789253 Revoke & destroy October 21 2020 October 21 2020 Client Authentication & S/MIME ICA dedicated to a single organization.

At the time of posting, the following affected issuing CA have been addressed:

ID CN crt.sh Treatment Revocation date Destruction date Additional Information
8 Ford Motor Company - Enterprise Issuing CA01 https://crt.sh/?id=392882654 Parent revoked on July 11 2020 N/A N/A
9 Ford Motor Company - Enterprise Issuing CA01 https://crt.sh/?id=306624237 Parent revoked on July 11 2020 N/A N/A
19 JCAN Public CA1 - G4 https://crt.sh/?id=163676419 Parent revoked on July 11 2020 N/A N/A
22 SHECA DV Secure Server CA https://crt.sh/?id=1225556701 Parent revoked on July 11 2020 N/A N/A
23 SHECA EV Secure Server CA https://crt.sh/?id=1229139434 Parent revoked on July 11 2020 N/A N/A
24 SHECA OV Secure Server CA https://crt.sh/?id=1225556702 Parent revoked on July 11 2020 N/A N/A
26 CrowdStrike OV SSL Issuing CA 2020 https://crt.sh/?id=2839140453 Parent revoked on July 11 2020 N/A N/A
28 DPDHL User CA I3 https://crt.sh/?id=1596016275 Parent revoked on July 11 2020 N/A N/A
29 DPDHL User CA I3 https://crt.sh/?id=12729527 Parent revoked on July 11 2020 N/A N/A
30 DPDHL User CA I3 Not in crt.sh Parent revoked on July 11 2020 N/A N/A SHA256 BCE3A5BD8D9082636C5BFE3E0B71ACEE551E24E3BD035887D2661ADA65AFF484
31 DPDHL User CA I3 https://crt.sh/?id=329514052 Parent revoked on July 11 2020 N/A N/A
33 Giesecke and Devrient CA https://crt.sh/?id=196919504 Parent revoked on July 11 2020 N/A N/A
60 GlobalSign Timestamping CA - G3 https://crt.sh/?id=2392141070 Destroy N/A May 28 2020 Keys were already destroyed in the context of remediating https://bugzilla.mozilla.org/show_bug.cgi?id=1591005#c38
61 GlobalSign Timestamping CA - SHA256 - G3 https://crt.sh/?id=2369948437 Destroy N/A May 28 2020 Keys were already destroyed in the context of remediating https://bugzilla.mozilla.org/show_bug.cgi?id=1591005#c38
64 RNP ICPEdu OV SSL CA 2019 https://crt.sh/?id=1476651440 Parent revoked on July 11 2020 N/A N/A
67 DPDHL User CA I3 https://crt.sh/?id=12729526 Parent revoked on July 11 2020 N/A N/A
68 Ford Motor Company - Enterprise Issuing CA01 https://crt.sh/?id=215376217 Parent revoked on July 11 2020 N/A N/A
69 Ford Motor Company - Enterprise Issuing CA01 https://crt.sh/?id=215376215 Parent revoked on July 11 2020 N/A N/A
75 Liberty University External Issuing CA 01 Not in crt.sh Parent revoked on July 11 2020 N/A N/A SHA256 1F91212C6BFC333C6EB52A685525E1E5B9E3AC1EF7A5A86649F5F95C721D8898
76 Liberty University External Issuing CA 01 https://crt.sh/?id=36391364 Parent revoked on July 11 2020 N/A N/A
79 Crown Prince Court CA https://crt.sh/?id=7890405 Parent revoked on July 11 2020 N/A N/A
80 Crown Prince Court CA https://crt.sh/?id=10105729 Parent revoked on July 11 2020 N/A N/A
81 Crown Prince Court CA Not in crt.sh Parent revoked on July 11 2020 N/A N/A SHA256 F164AD5E4CE9EFC0A144CA902EA2ED46C464D2D508CA919A23095CDF30D4DC68
82 Crown Prince Court CA for AATL Not in crt.sh Parent revoked on July 11 2020 N/A N/A SHA256 DF45EEAED905C58D730EC5497B59B3AB4CCE7C6459953DF9CA5C1F031AC06DD8
83 Crown Prince Court CA for AATL Not in crt.sh Parent revoked on July 11 2020 N/A N/A SHA256 D21076207F79A9B04137D40A4FFE6DD08921CCF49E6EB60277FF4593E076D538
84 Crown Prince Court CA for AATL Not in crt.sh Parent revoked on July 11 2020 N/A N/A SHA256 70BDB19C31F5EF105B29376E35EA5ED8EEBE13CB5C0758C32DFC4C5F7230A173
85 DPDHL TLS CT CA I3 https://crt.sh/?id=786990298 Parent revoked on July 11 2020 N/A N/A
86 DPDHL TLS SHA 2 CA I3 https://crt.sh/?id=8527797 Parent revoked on July 11 2020 N/A N/A
87 DPDHL TLS SHA 2 CA I3 https://crt.sh/?id=329514048 Parent revoked on July 11 2020 N/A N/A
88 DPDHL TLS SHA 2 CA I3 https://crt.sh/?id=27823827 Parent revoked on July 11 2020 N/A N/A
89 DPDHL TLS SHA 2 CA I3 https://crt.sh/?id=9881176 Parent revoked on July 11 2020 N/A N/A
90 Southern Company External Issuing CA 1 https://crt.sh/?id=11501550 Parent revoked on July 11 2020 N/A N/A
91 Trusted Root CA G2 https://crt.sh/?id=1862521 Revoked July 11 2020 N/A N/A Had children CA that included OCSP signing EKU and with third-party operated keys
92 Trusted Root CA SHA256 G2 https://crt.sh/?id=3179271 Revoked July 11 2020 N/A N/A Had children CA that included OCSP signing EKU and with third-party operated keys

Thanks. I'm going to set this matter for a report back / next update on or before 15-Oct 2020.

Whiteboard: [ca-compliance] → [ca-compliance] Next Update 15-Oct 2020

Looking at one of the certificates,

GlobalSign HV RSA DV SSL CA 2018
https://crt.sh/?id=970082980
Revoke & destroy
Revocation Date: July 8 2020
Destruction Date: July 28 2020

After clicking through to crt.sh the cert is not revoked via OCSP or CRL.

Is there a reason why https://crt.sh/?id=970082980 wasn't revoked according to the schedule?

Flags: needinfo?(arvid.vermote)

Thanks Joe, Ben. Looks like a mistake slipped in when copying over and converting the tables into markdown format. The correct date for #45 "GlobalSign HV RSA DV SSL CA 2018" is both revocation and key destruction on October 21 2020. I'll post a new full overview of all unaddressed issuing CA after the July 28 activities when we are revoking & destroying a batch of affected issuing CA - we are also looking to accelerate and push forward the date of some other affected issuing CA.

Flags: needinfo?(arvid.vermote)

Hi Ben - please find attached a list of currently unaddressed affected CA that can be added to OneCRL. Let me know if any additional information is required to proceed - thank you.

Flags: needinfo?(bwilson)

Please proceed

Flags: needinfo?(bwilson)

(In reply to Arvid Vermote from comment #15)

Hi Ben - please find attached a list of currently unaddressed affected CA that can be added to OneCRL. Let me know if any additional information is required to proceed - thank you.

Are all of the CAs on this list issuers of TLS certificates? Also, it might be that we don't add these to OneCRL if there is no threat Firefox users. I'm still looking into this.

Hi Ben - some of them are TLS issuers, others are not. But they all contain the OCSP EKU which in case of issuing CA key compromise can be abused to manipulate the validity status of the issuing CA itself and other (TLS) issuing CA and certificates that share the same parent. Based on that we understood that adding them all to OneCRL would additionally protect Mozilla users from these issuing CA being abused as OCSP delegated responder certificates.

Those CA Certificates that you revoke and update in the CCADB will be automatically added to OneCRL. For ones that you are not revoking, then I would like for you to segregate out the TLS-capable issuers from the non-TLS-capable issuers. (It would be good to know why they are not being revoked.) Then I will have more information to help determine whether the TLS-capable issuers need to be added to OneCRL. While this exercise might be unnecessary because Firefox users are adequately protected by the way that mozilla::pkix processes CA certificates with the OCSP-signing EKU, let's take this next step of identifying TLS issuers as outlined above. (Also, adding these CAs to OneCRL doesn't provide extra protection to Thunderbird users because Thunderbird doesn't use OneCRL).

Attachment #9165342 - Attachment is obsolete: true

Hi Ben - the requested OneCRL entries that related to TLS issuers are not relevant anymore since their keys got destroyed July 28 2020. I have attached a new version of the OneCRL request list, which inlcudes issuers of non-TLS certificates that have the OCSP EKU set and for which the keys haven't been destroyed yet - in case Mozilla deems it applicable to add these. Thanks

Following affected issuing CA have been revoked on July 28 2020:

ID CN SHA256 crt.sh
11 GlobalSign PersonalSign 1 CA - G3 F068DEAA18CC02D5A8BE35CB8338327910291F6E62E7216A934764A1ABA4A800 https://crt.sh/?id=2369948051
12 GlobalSign PersonalSign 2 CA - G3 925EE7D5A22AD7FBE9BAB54D7C8D0B9A74F7E35A8AF6AF645E2E8C3519A7092F https://crt.sh/?id=2369947954
15 GlobalSign PersonalSign 2 ECC CA SHA 384 - G4 46038F6326228CDB56619C52266613DA04C8CA499E0D03B0EDCFFC110D5CFC70 https://crt.sh/?id=405618313
16 GlobalSign PersonalSign 2 RSA CA SHA 384 - G4 5CDD809CF44F5F8665EAC15055504C5B06B787AC18294505BDBAB4A77E50D776 https://crt.sh/?id=405618295
17 GlobalSign PersonalSign 3 CA - G3 B1FE3AEBF963A7880E74B0B0556681EA8B1CCCE3E69A7D3B10A68ACBE86E48A1 https://crt.sh/?id=2369948436
20 NAESB Issuing CA - SHA384 - G3 128DED1A8AD60C24B4254E31DB94FC4392BF93ED5434472AA43A0B9856106068 https://crt.sh/?id=2369948019

The copies of following issuing CA keys loaded in issuance-capable environments have been destroyed on July 28 and August 3 2020, backup copies will be destroyed in the upcoming two weeks:

ID CN SHA256 crt.sh
6 GlobalSign Qualified Time Stamping CA 2019 74ABE5E5CCEB75491FF72C4CF325405D8ADBFE390E189CF430BA60E62798878E https://crt.sh/?id=1490728721
11 GlobalSign PersonalSign 1 CA - G3 F068DEAA18CC02D5A8BE35CB8338327910291F6E62E7216A934764A1ABA4A800 https://crt.sh/?id=2369948051
12 GlobalSign PersonalSign 2 CA - G3 925EE7D5A22AD7FBE9BAB54D7C8D0B9A74F7E35A8AF6AF645E2E8C3519A7092F https://crt.sh/?id=2369947954
14 GlobalSign PersonalSign 2 CA - SHA256 - G4 27D6FDAF80297846DFEFF82E7F58B9A48AC9E3EE93A112B1BBE243EE1A97447C https://crt.sh/?id=2839140428
15 GlobalSign PersonalSign 2 ECC CA SHA 384 - G4 46038F6326228CDB56619C52266613DA04C8CA499E0D03B0EDCFFC110D5CFC70 https://crt.sh/?id=405618313
16 GlobalSign PersonalSign 2 RSA CA SHA 384 - G4 5CDD809CF44F5F8665EAC15055504C5B06B787AC18294505BDBAB4A77E50D776 https://crt.sh/?id=405618295
17 GlobalSign PersonalSign 3 CA - G3 B1FE3AEBF963A7880E74B0B0556681EA8B1CCCE3E69A7D3B10A68ACBE86E48A1 https://crt.sh/?id=2369948436
41 GlobalSign ECC EV SSL CA 2019 0D3176C58F321AA34C57C8DF7C17D1F4E76C797EC116C9F1D697748ED1FCE7D9 https://crt.sh/?id=2329203344
44 GlobalSign HV ECC DV SSL CA 2018 4B0D1392D39157353207A64CCB14683DDE9D2CED1FB58B16E038BE5707C27813 https://crt.sh/?id=970083107
52 GlobalSign Qualified CA 2 FD3A0F3DD4480092B6D450473DEB9201A0B308A8807833A3C738F8A07EB81ED3 https://crt.sh/?id=509714291
53 GlobalSign Qualified CA 3 0AA9F2E7D95C718B7D1EB7CCDBD0164E86057AE9D66922BC60F9903F94A0F0EF https://crt.sh/?id=509714292
54 GlobalSign Qualified Timestamping ECC CA 2020 C2FEACD674878C7B0C2325A2ECED0A333DB7780A86DFEC3758100EFC0101C665 https://crt.sh/?id=2839140405
55 GlobalSign R6 Admin CA - SHA256 - G3 C5B679106958152F83FB5886DDC41F0785193EF67C6975BE3E509F17F29B7A86 https://crt.sh/?id=164243753
56 GlobalSign R6 RSA EV SSL CA 2019 57264B82A864DBA1C11EF3F80ABB94CAC3660662B0C22F571FF993B3FBCF76FB https://crt.sh/?id=1476654013
59 GlobalSign RSA EV SSL CA 2019 0D6E46784F3B694E9C7506786417BC6F87F9D2F73D19B5E8081612B21137B766 https://crt.sh/?id=2220986544
65 Trafigura PTE Ltd S/MIME ICA 2020 5E7FCB9C97BDA56993B1658D120232761D665A3644534300FA6A5BEC5E0D5795 https://crt.sh/?id=2369948428
70 GlobalSign CA for AATL on HV 1C9266902A31C3941B506D44D0D4D06EC9DB7655E65F9557659FAB768B290B1B https://crt.sh/?id=1119260014
71 GlobalSign CA for AATL on HV BCBD04D4AED962C9D25AFE0CFAF8638CE1431652988EC5217329E7559AC3C671 https://crt.sh/?id=163322577
77 MSC Trustgate.com RSA AATL CA 2019 3A882530C03EA615E5EF4DADBD7C8660912FA93FAF5088716FB46A8E1FFA9218 https://crt.sh/?id=1119259389
78 VWFS CA for AATL CF89A41DFEE5F71740DEF602735DDBF1DEBE0CB816D73980D9A583C5881CE778 https://crt.sh/?id=163688490

We were able to accelerate the planned revocation of three affected issuing CA, which have been revoked on August 19 2020.

ID CN SHA256 crt.sh
1 AbbVie AATL ICA 2020 154E4834B28D4FB1F90FEE935D0DDE46C45A177FC1425A028C685C32855A85AD https://crt.sh/?id=2369948023
4 GlobalSign AATL Partners CA 2019 83FC891B350D9E0D7EBE6DD2A6BFE3D0B0F4653FCA048615A5DEEBBC039A3F66 https://crt.sh/?id=1436918881
5 GlobalSign Issuing CA for AATL Partners 2019 67C46DC17762667844F1596089375FF45E05C2B316C89499F6E7FAB78C8F0379 https://crt.sh/?id=1703475173

As per the plan the "GlobalSign SMIME CA 2018" was revoked on September 16 2020.

ID CN SHA256 crt.sh
18 GlobalSign SMIME CA 2018 C8192C32F7B49C7F32A1CA001595A7F9E36C9E72058D6EAA1BAB7752A8C16718 https://crt.sh/?id=549505576

The "NAESB Issuing CA - SHA384 - G3", "ATT Organization Validated CA 2019", "DPDHL Global TLS CA - I4" and "GlobalSign PersonalSign Partners CA - SHA256 - G2" have been revoked on September 30 2020.

ID CN SHA256 crt.sh
21 NAESB Issuing CA - SHA384 - G3 0986B5A1C7314EFB04FB648B9E2B57CF4842FD1D4345D28E52094C90A9FECBFE https://crt.sh/?id=18068129
25 ATT Organization Validated CA 2019 7AA45D6F5B14DAB1C6844C19C2804E14B5811E6EDE1F02B0AEF065A7B359C68F https://crt.sh/?id=1490728430
27 DPDHL Global TLS CA - I4 94C663E9EA5C27EE4F64127F9B425863E991A9E156C07DF1A00803AE31764162 https://crt.sh/?id=1814823951
50 GlobalSign PersonalSign Partners CA - SHA256 - G2 118262C2088EE1528E20D836D2070854707C0D8F8E80FBE396F9ECD4B9141B5B https://crt.sh/?id=12715740

Attached the ISAE3000 report on the destruction of the keys depicted in the table below.

ID CN SHA256 crt.sh
54 GlobalSign Qualified Timestamping ECC CA 2020 C2FEACD674878C7B0C2325A2ECED0A333DB7780A86DFEC3758100EFC0101C665 https://crt.sh/?id=2839140405
44 GlobalSign HV ECC DV SSL CA 2018 4B0D1392D39157353207A64CCB14683DDE9D2CED1FB58B16E038BE5707C27813 https://crt.sh/?id=970083107
56 GlobalSign R6 RSA EV SSL CA 2019 57264B82A864DBA1C11EF3F80ABB94CAC3660662B0C22F571FF993B3FBCF76FB https://crt.sh/?id=1476654013
55 GlobalSign R6 Admin CA - SHA256 - G3 C5B679106958152F83FB5886DDC41F0785193EF67C6975BE3E509F17F29B7A86 https://crt.sh/?id=164243753
59 GlobalSign RSA EV SSL CA 2019 0D6E46784F3B694E9C7506786417BC6F87F9D2F73D19B5E8081612B21137B766 https://crt.sh/?id=2220986544
41 GlobalSign ECC EV SSL CA 2019 0D3176C58F321AA34C57C8DF7C17D1F4E76C797EC116C9F1D697748ED1FCE7D9 https://crt.sh/?id=2329203344
52 GlobalSign Qualified CA 2 FD3A0F3DD4480092B6D450473DEB9201A0B308A8807833A3C738F8A07EB81ED3 https://crt.sh/?id=509714291
53 GlobalSign Qualified CA 3 0AA9F2E7D95C718B7D1EB7CCDBD0164E86057AE9D66922BC60F9903F94A0F0EF https://crt.sh/?id=509714292
14 GlobalSign PersonalSign 2 CA - SHA256 - G4 27D6FDAF80297846DFEFF82E7F58B9A48AC9E3EE93A112B1BBE243EE1A97447C https://crt.sh/?id=2839140428
65 Trafigura PTE Ltd S/MIME ICA 2020 5E7FCB9C97BDA56993B1658D120232761D665A3644534300FA6A5BEC5E0D5795 https://crt.sh/?id=2369948428
6 GlobalSign Qualified Time Stamping CA 2019 74ABE5E5CCEB75491FF72C4CF325405D8ADBFE390E189CF430BA60E62798878E https://crt.sh/?id=1490728721
77 MSC Trustgate.com RSA AATL CA 2019 3A882530C03EA615E5EF4DADBD7C8660912FA93FAF5088716FB46A8E1FFA9218 https://crt.sh/?id=1119259389
78 VWFS CA for AATL CF89A41DFEE5F71740DEF602735DDBF1DEBE0CB816D73980D9A583C5881CE778 https://crt.sh/?id=163688490
11 GlobalSign PersonalSign 1 CA - G3 F068DEAA18CC02D5A8BE35CB8338327910291F6E62E7216A934764A1ABA4A800 https://crt.sh/?id=2369948051
12 GlobalSign PersonalSign 2 CA - G3 925EE7D5A22AD7FBE9BAB54D7C8D0B9A74F7E35A8AF6AF645E2E8C3519A7092F https://crt.sh/?id=2369947954
17 GlobalSign PersonalSign 3 CA - G3 B1FE3AEBF963A7880E74B0B0556681EA8B1CCCE3E69A7D3B10A68ACBE86E48A1 https://crt.sh/?id=2369948436
15 GlobalSign PersonalSign 2 ECC CA SHA 384 - G4 46038F6326228CDB56619C52266613DA04C8CA499E0D03B0EDCFFC110D5CFC70 https://crt.sh/?id=405618313
16 GlobalSign PersonalSign 2 RSA CA SHA 384 - G4 5CDD809CF44F5F8665EAC15055504C5B06B787AC18294505BDBAB4A77E50D776 https://crt.sh/?id=405618295
70 GlobalSign CA for AATL on HV 1C9266902A31C3941B506D44D0D4D06EC9DB7655E65F9557659FAB768B290B1B https://crt.sh/?id=1119260014
71 GlobalSign CA for AATL on HV BCBD04D4AED962C9D25AFE0CFAF8638CE1431652988EC5217329E7559AC3C671 https://crt.sh/?id=163322577
Whiteboard: [ca-compliance] Next Update 15-Oct 2020 → [ca-compliance] Next Update 2020-12-01
Flags: needinfo?(arvid.vermote)

Following CA have been revoked on October 21 2020:

ID CN SHA256 crt.sh
2 CRB Group SMIME CA 2019 6A5F4C1678CA65E59F060D57CDFF665065314861D53A8E7D1450CA92D96CA102 https://crt.sh/?id=2029982659
7 Qu\C3\A1litas Compa\C3\B1\C3\ADa de Seguros S.A. de C.V. B716B089FE4E53D1A2EF7BA57AC85E68EC722CF61052C25A59626AD3B15C5F40 https://crt.sh/?id=1814826066
39 GlobalSign CodeSigning CA - SHA256 - G2 BE40813869AB27A071D12AD6A8830583EBC3B618E3F2346359F4B11A1C9434EE https://crt.sh/?id=1703475054
45 GlobalSign HV RSA DV SSL CA 2018 54C37A8E853FD1D6378D378B939307EC321A31CC1A5A89E7180633BC13F18762 https://crt.sh/?id=970082980
96 GROB-WERKE GmbH und Co. KG SMIME CA 1608BF87414CDCFAB4279102A19702D9D5996A91329E3DF2F80495473AAD86C6 https://crt.sh/?id=872293772
99 HERTZ SHA256 CA - G2 64FD7F66B805EF0FAE09DAC06EAD1A9AB5C28E6F24AD0759996D349987FEE7E2 https://crt.sh/?id=408789253

The active key pairs of following CA have been destroyed on October 21, 2020, backup copies were destroyed by October 27 2020:

ID CN SHA256 crt.sh
1 AbbVie AATL ICA 2020 154E4834B28D4FB1F90FEE935D0DDE46C45A177FC1425A028C685C32855A85AD https://crt.sh/?id=2369948023
2 CRB Group SMIME CA 2019 6A5F4C1678CA65E59F060D57CDFF665065314861D53A8E7D1450CA92D96CA102 https://crt.sh/?id=2029982659
4 GlobalSign AATL Partners CA 2019 83FC891B350D9E0D7EBE6DD2A6BFE3D0B0F4653FCA048615A5DEEBBC039A3F66 https://crt.sh/?id=1436918881
5 GlobalSign Issuing CA for AATL Partners 2019 67C46DC17762667844F1596089375FF45E05C2B316C89499F6E7FAB78C8F0379 https://crt.sh/?id=1703475173
7 Qu\C3\A1litas Compa\C3\B1\C3\ADa de Seguros S.A. de C.V. B716B089FE4E53D1A2EF7BA57AC85E68EC722CF61052C25A59626AD3B15C5F40 https://crt.sh/?id=1814826066
18 GlobalSign SMIME CA 2018 C8192C32F7B49C7F32A1CA001595A7F9E36C9E72058D6EAA1BAB7752A8C16718 https://crt.sh/?id=549505576
20 NAESB Issuing CA - SHA384 - G3 128DED1A8AD60C24B4254E31DB94FC4392BF93ED5434472AA43A0B9856106068 https://crt.sh/?id=2369948019
21 NAESB Issuing CA - SHA384 - G3 0986B5A1C7314EFB04FB648B9E2B57CF4842FD1D4345D28E52094C90A9FECBFE https://crt.sh/?id=18068129
25 ATT Organization Validated CA 2019 7AA45D6F5B14DAB1C6844C19C2804E14B5811E6EDE1F02B0AEF065A7B359C68F https://crt.sh/?id=1490728430
27 DPDHL Global TLS CA - I4 94C663E9EA5C27EE4F64127F9B425863E991A9E156C07DF1A00803AE31764162 https://crt.sh/?id=1814823951
45 GlobalSign HV RSA DV SSL CA 2018 54C37A8E853FD1D6378D378B939307EC321A31CC1A5A89E7180633BC13F18762 https://crt.sh/?id=970082980
96 GROB-WERKE GmbH und Co. KG SMIME CA 1608BF87414CDCFAB4279102A19702D9D5996A91329E3DF2F80495473AAD86C6 https://crt.sh/?id=872293772
99 HERTZ SHA256 CA - G2 64FD7F66B805EF0FAE09DAC06EAD1A9AB5C28E6F24AD0759996D349987FEE7E2 https://crt.sh/?id=408789253

Following CA have been revoked on November 18 2020, active key pairs destroyed on November 18 2020, backup copies destroyed by November 24 2020:

ID CN SHA256 crt.sh
3 DexKo Global SMIME CA 2019 ABC86706C98D6BF67372F908EC01ADF631B191D733AE89F8343EB047B108144B https://crt.sh/?id=2029984306
32 DPDHL User CA I4 C25C4EDBC36E3FB7C3D937BEE9F2D29E36AFB07CFA3188262E0D5FDC919E0D77 https://crt.sh/?id=2369948075
62 LinQuest SMIME CA 2020 113138DD7B216725840238E2D7EEECB3738DB139064B24CB853FC270A49E6057 https://crt.sh/?id=2369948433
93 CBMM SMIME CA 2019 C346D9137E05254C6EEAC99AC2F6748A0C5D3AFC6B7B9B1E00C40ADF4D85655D https://crt.sh/?id=1596016282
97 Hyperion SMIME CA 2018 DCD77E34B46D530AEF645A513389CD4FFC0F7D196A115B8F62A5FD0D557D46C6 https://crt.sh/?id=721305503
98 Spirit AeroSystems SHA256 CA - G2 8609BDCEF95E4A4D426497B5CD8ED4B001C953A5C14471CAAF58FB650DF8ABF0 https://crt.sh/?id=215376218

For the affected Code Signing CA wa have altered our approach, given that revoking these issuing CA would render any previously applied code signature through this hierarchy invalid. Destroying the keys without revoking the CA is not an option since we need to have the capability to revoke both active and expired (MS Root Program requirements) code signing certificates. Given that the OCSP EKU risk is present as long as the CA is valid and the keys are alive, we will apply following course of action:

  • April 22 2020 through April 21 2021: period-under-audit of custom OCSP EKU SOC2 report covering non-performance of OCSP signing by affected keys and protection of CA keys affected by the OCSP EKU incident, refer to https://bugzilla.mozilla.org/show_bug.cgi?id=1651447#c13
  • Januari 2021 through April 21 2021: work with customers to move all certificates to unaffected code signing hierarchies, request customers to destroy keys associated with leafs under the affected hierarches and where possible work with customers to revoke leaf certificates
  • April 21 2021: Creation of a seperate offline key storage environment equal to the protection we apply to root keys and migration of the Code Signing keys to this environment
  • April 21 2021: Include controls on the design of offline Code Signing key storage environment in the custom OCSP EKU SOC2 report.
  • April 21 2021: Destruction of all Code Signing key pair copies except those stored in the offline key storage environment (witnessed by Qualified Independent auditor)
  • April 21 until expiry of the last OCSP EKU affected Code Signing ICA: execute special Code Signing leaf revocation key ceremonies for revocation requests on certificates issued through the affected hierarchies.

Following table depicts the Code Signing CA for which we will take above approach:

ID CN SHA256 crt.sh
10 GlobalSign CodeSigning CA - G3 4047C9D69260C07213BCB8608A7EC5E2838A56B79F67847812EAC0778D0D27F1 https://crt.sh/?id=157564305
39 GlobalSign CodeSigning CA - SHA256 - G2 BE40813869AB27A071D12AD6A8830583EBC3B618E3F2346359F4B11A1C9434EE https://crt.sh/?id=1703475054
40 GlobalSign CodeSigning CA - SHA256 - G3 FB54EEA9BCE8E9EA9782154F3D414277FB709F49B947D73978AC278546C2CE03 https://crt.sh/?id=26749929
42 GlobalSign Extended Validation CodeSigning CA - SHA256 - G2 1E864278C20881B671C0C6D2E14B61150AD1F13CF92C6EC14B550DCBC47E1541 https://crt.sh/?id=1703475088
43 GlobalSign Extended Validation CodeSigning CA - SHA256 - G3 DD038E87E0B4D2C369680D3DE78638AB39FC1D7E50632996921101768DB8D4D8 https://crt.sh/?id=41285443
66 GlobalSign CodeSigning CA - G2 FFFE077503FD72F0E5338B0A7B4E218E7D1FF82E493E7E852AE51AA1C7585D17 https://crt.sh/?id=1476651569
Flags: needinfo?(arvid.vermote)

Following CA have been revoked on December 31 2020:

ID CN SHA256 crt.sh
34 GlobalSign CA 4 for AATL EBA34C7B109671614C367E1DE075124C3954CE19F85FACF61090EC319F7F1A7F https://crt.sh/?id=1229139435
36 GlobalSign CA 5 for AATL 306E9739E3458FF4546877B704B2E3905E58B235D64E32F4F026AC91B7295D15 https://crt.sh/?id=408789250
37 GlobalSign CA 6 for AATL BE1FFC0E1FF6088104F43E327E7C7DC72A9CA7B0DF05793123ABE32DEACEE76F https://crt.sh/?id=2369988390
94 Accenture Federal Services External CA A9C8E971259A2ED6E65F721E07AA967C72C7CDB47C7BE1288D87BF08D2F3580D https://crt.sh/?id=215376216
95 EY LLP SHA256 CA - G2 1557F65BA61C958B74EFA4A582BBAEBDD62A6D9B65FE95A80D5ED518F46ED87F https://crt.sh/?id=970084237
Whiteboard: [ca-compliance] Next Update 2020-12-01 → [ca-compliance]

This bug can probably be closed and progress can be tracked in Bug 1651447.

Hi Ben, if you don't mind we prefer this bug to be left open as we have referenced this ticket in communication with affected customers and relying parties, who are monitoring our remediation progress & communication with the Mozilla root program based on the information posted in this ticket.

If it is ok with Mozilla we will use https://bugzilla.mozilla.org/show_bug.cgi?id=1651447 to still further track the actions we are doing to avoid exceeding the 7 days time frame as set forth by section #4.9.1.2 of the SSL Baseline Requirements, whereas we will use this bug to track the remediation of all CA with the OCSP EKU set.

As per the plan, the next batch of revocations will be executed on January 20 2021:

ID CN SHA256 crt.sh
48 GlobalSign PersonalSign Partners CA - SHA256 - G2 4E707867946AC05343C6BA8FF121EA66A758037913257A8EE4974350D39A1034 https://crt.sh/?id=2369948041
57 GlobalSign RSA DV SSL CA 2018 9E898ED03FA46969690DAD73C7296675045FF9B5A0100A399BEB8435A98F5185 https://crt.sh/?id=970083106

Following CA have been revoked on January 20 2021:

ID CN SHA256 crt.sh
48 GlobalSign PersonalSign Partners CA - SHA256 - G2 4E707867946AC05343C6BA8FF121EA66A758037913257A8EE4974350D39A1034 https://crt.sh/?id=2369948041
57 GlobalSign RSA DV SSL CA 2018 9E898ED03FA46969690DAD73C7296675045FF9B5A0100A399BEB8435A98F5185 https://crt.sh/?id=970083106

As per the plan, the next batch of revocations will be executed on February 24 2021:

ID CN SHA256 crt.sh
13 GlobalSign PersonalSign 2 CA - SHA256 - G3 B778748A792B8F91F04B01BAFC31A31ED7EF6A712AFF80B6610D9AADEE207ADF https://crt.sh/?id=24592899
46 GlobalSign PersonalSign 1 CA - SHA256 - G3 F5D2D2BA6817A7A9AA0E21354BBF0E6F95C5E287EE88CF2F279F0FFEC4EDAC15 https://crt.sh/?id=147619379
47 GlobalSign PersonalSign 3 CA - SHA256 - G3 701B432AC0CDD4D9CF95B4B884C32BF5CCA90D44E0161ABD13B934D68E380472 https://crt.sh/?id=163079175
51 GlobalSign Qualified CA 1 F5709A2D2F68B53BF6F645BB178ADF95346F89FDA5C63BFDE08042A26492AAB2 https://crt.sh/?id=509714293
58 GlobalSign RSA EV QWAC CA 2019 EDC734C501501DC7A27448FA02C74931F8578BF297B173F34B841E82C6691926 https://crt.sh/?id=1490728500
63 NAESB Issuing CA - SHA384 - G4 C4C7C436BD88E8E68DB00297DF83ACC819E198639BA00522C8E3245876898523 https://crt.sh/?id=2369948432

Following CA have been revoked on February 24 2021:

ID CN SHA256 crt.sh
13 GlobalSign PersonalSign 2 CA - SHA256 - G3 B778748A792B8F91F04B01BAFC31A31ED7EF6A712AFF80B6610D9AADEE207ADF https://crt.sh/?id=24592899
46 GlobalSign PersonalSign 1 CA - SHA256 - G3 F5D2D2BA6817A7A9AA0E21354BBF0E6F95C5E287EE88CF2F279F0FFEC4EDAC15 https://crt.sh/?id=147619379
47 GlobalSign PersonalSign 3 CA - SHA256 - G3 701B432AC0CDD4D9CF95B4B884C32BF5CCA90D44E0161ABD13B934D68E380472 https://crt.sh/?id=163079175
51 GlobalSign Qualified CA 1 F5709A2D2F68B53BF6F645BB178ADF95346F89FDA5C63BFDE08042A26492AAB2 https://crt.sh/?id=509714293
58 GlobalSign RSA EV QWAC CA 2019 EDC734C501501DC7A27448FA02C74931F8578BF297B173F34B841E82C6691926 https://crt.sh/?id=1490728500
63 NAESB Issuing CA - SHA384 - G4 C4C7C436BD88E8E68DB00297DF83ACC819E198639BA00522C8E3245876898523 https://crt.sh/?id=2369948432

Just looking at the plan indicating these CAs are due to be revoked/destroyed April 21, 2021, if they haven't already. Please provide an update on items remaining before this bug and the other one can be closed. Thanks.

GlobalSign Extended Validation CodeSigning CA - SHA256 - G3
GlobalSign CodeSigning CA - G3

Whiteboard: [ca-compliance] → [ca-compliance] Next Update 2021-04-23

Hi Ben - we actioned the last remaining CA per the outline in Comment #27. By now the auditing activities are ongoing to produce the custom SOC2/SOC3 report on the non-performance of OCSP signing by the affected CA. We propose to keep this bug open until the audit report has been delivered and SOC3 report uploaded to this ticket. It typically takes the auditor 2-3 months to produce these reports. Thank you.

We have no further updates for now - the auditing activities are ongoing in order to produce a custom SOC2/SOC3 report on the non-performance of OCSP signing by the affected CA.

We have no further updates for now - the auditing activities are ongoing in order to produce a custom SOC2/SOC3 report on the non-performance of OCSP signing by the affected CA.

We have no further updates for now - the auditing activities are ongoing in order to produce a custom SOC2/SOC3 report on the non-performance of OCSP signing by the affected CA.

Flags: needinfo?(bwilson)

We have no further updates for now - the auditing activities are ongoing in order to produce a custom SOC2/SOC3 report on the non-performance of OCSP signing by the affected CA.

Flags: needinfo?(bwilson)
Whiteboard: [ca-compliance] Next Update 2021-04-23 → [ca-compliance] Next Update 2021-07-01

We have no further updates for now - the auditing activities are ongoing in order to produce a custom SOC2/SOC3 report on the non-performance of OCSP signing by the affected CA.

Whiteboard: [ca-compliance] Next Update 2021-07-01 → [ca-compliance] Next Update 2021-08-16

We are currently having some final clarification meetings and evidence sharing sessions with the auditor. We received feedback from the auditor that based on current progress the auditing activities would be closed around the end of August after which the reports will be submitted for further internal review (at auditor side) and CPA approval.

JIPDEC* requested to expand on a previous reference made to Kyushu Electric Power in Comment #7 and Comment #9 in the sense that Kyushu Electric Power was not using the certificates for their electricity supply system itself, but to support the communication regarding recovery in areas affected by the July 2020 Kyushu flooding disaster.

*JIPDEC is the promotor of JCAN certificate program which aims at providing electronic signature and e-mail certificates to entities in Japan.

We have no further updates for now - the feedback from the auditor is that based on current progress the auditing activities would be closed around the end of August after which the reports will be submitted for further internal review (at auditor side) and CPA approval.

Whiteboard: [ca-compliance] Next Update 2021-08-16 → [ca-compliance] Next Update 2021-09-01
Flags: needinfo?(arvid.vermote)

We are now further working with the auditor on the report sections and are going back-and-forth on additional clarifications and questions originating from the internal reviews at auditor side.

Flags: needinfo?(arvid.vermote)

We are still in same modus operandi of further working with the auditor on the report sections and going back-and-forth on additional clarifications and questions originating from the internal reviews at auditor side

Whiteboard: [ca-compliance] Next Update 2021-09-01 → [ca-compliance] Next Update 2021-10-01

We have been probing the auditor for an update on the report delivery timelines and will share it as soon as available to us.

We have received word from the auditor that they expect the final report to be available November 19 2021.

Whiteboard: [ca-compliance] Next Update 2021-10-01 → [ca-compliance] Next Update 2021-11-20
Whiteboard: [ca-compliance] Next Update 2021-11-20 → [ca-compliance] Next Update 2022-01-31

Please find attached the ISAE3000 Type II report regarding the non-performance of OCSP signing by the CAs affected by this incident, during the period of April 22 2020 through April 21 2021.

This concludes our remedial activities and unless there are any further questions, we believe this incident can now be closed.

Flags: needinfo?(bwilson)
Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Flags: needinfo?(bwilson)
Resolution: --- → FIXED
Product: NSS → CA Program
Whiteboard: [ca-compliance] Next Update 2022-01-31 → [ca-compliance] [ocsp-failure]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: