Closed Bug 2000277 Opened 8 months ago Closed 7 months ago

Sectigo: Certificate issuance by non-compliant Extant S/MIME CA

Categories

(CA Program :: CA Certificate Compliance, task)

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: martijn.katerbarg, Assigned: martijn.katerbarg)

Details

(Whiteboard: [ca-compliance] [smime-misissuance])

Attachments

(1 file)

Preliminary Incident Report

Summary

  • Incident description:

Today, November 14th, we became aware that one or more of our non-compliant Extant S/MIME CAs may have issued S/MIME certificates on and after September 15th, 2024.

We are currently investigating the total scope and impact of this incident.

  • Relevant policies: Baseline Requirements for the Issuance and Management of Publicly-Trusted S/MIME Certificates Version 1.0.12, Appendix B.

  • Source of incident disclosure: Self Reported

Assignee: nobody → martijn.katerbarg
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Whiteboard: [ca-compliance] [smime-misissuance]

Full Incident Report

Summary

  • CA Owner CCADB unique ID: A000016
  • Incident description:

We became aware that 6 of our non-compliant Extant S/MIME CAs have issued S/MIME certificates on and after September 15th, 2024.

5 out of the 6 Extant S/MIME CAs in question were previously marked as disabled within our system. However, a software bug allowed continued usage of these CAs for new issuance.

1 out of the 6 Extant S/MIME CAs was found to not previously have been identified as non-compliant with the S/MIME BRs and thus was not disabled until recently.

  • Timeline summary:
    • Non-compliance start date: 2024-09-15 – 00:00 UTC
    • Non-compliance identified date: 2025-11-14 – 15:47 UTC
    • Non-compliance end date: 2025-11-19 – 12:09 UTC
  • Relevant policies: Baseline Requirements for the Issuance and Management of Publicly-Trusted S/MIME Certificates Version 1.0.12, Appendix B.
  • Source of incident disclosure: Self Reported

Impact

  • Total number of certificates: 996
  • Total number of "remaining valid" certificates: 531 at the time the incident was identified. 0 at the time of the non-compliance end date.
  • Affected certificate types: MV, OV and SV S/MIME
  • Incident heuristic: See Appendix A.
  • Was issuance stopped in response to this incident, and why or why not?: Issuance by the affected Extant S/MIME CAs was halted due to a software bugfix prior to this issue having been identified.
  • Analysis: N/A.
  • Additional considerations: N/A

Timeline

All times are UTC.

2023-07-12:

  • 17:00 CA/B Forum ballot SMC-03 passes, introducing language related to Extant S/MIME CAs.

2023-10-12:

  • 10:30 We complete work assisting in identifying S/MIME issuance capable CA Certificates which lack either the SBR Reserved Policy OID or the anyPolicy OID.

2024-Q2:

  • We work with partners on migrating active S/MIME issuance from Extant S/MIME CAs to newly issued S/MIME Subordinate CAs.

2024-09-01:

  • We complete the removal of all account permissions to use the identified non-compliant Extant S/MIME CAs.

2025-03-07:

  • 09:23 We file an internal development ticket (Ticket #1) to refactor how Subordinate CAs are enabled for specific purposes and for specific accounts within our issuance system.

2025-10-10:

  • 14:59 Our QA team files an internal development ticket (Ticket #2) after having discovered a bug related to the enablement of non-TLS Subordinate CAs within our issuance systems. The bug may have allowed an account to request issuance by a Subordinate CA for which no explicit enablement was set up.

2025-10-11.

  • 23:00 We deploy an update to our issuance systems that incorporates refactored code for Ticket #1.

2025-11-08:

  • 23:00 We deploy an update to our issuance systems that incorporates code for Ticket #2.

2025-11-13:

  • 6:30 An internal ticket is filed requesting investigation of why a certificate request requesting “COMODO RSA Client Authentication and Secure Email CA” as Issuing CA is failing.
  • 18:43 The request is escalated to one of our senior engineers.
  • 19:03 The engineer escalates the request to compliance, requesting confirmation of whether or not the requested Subordinate CA is still allowed to issue S/MIME leaf certificates.

2025-11-14:

  • 10:29 A discussion takes place between the senior engineer and a member of the compliance department. The Subordinate does contain the allowed “anyPolicy” OID, however it does not contain an EKU extension. Both agree based on review of the S/MIME BRs that a SubCA that does not have the EKU extension, but does have the anyPolicy OID, is classed as an Extant S/MIME CA. A review of Appendix B of the S/MIME BRs further clarifies that an Extant S/MIME CA may continue issuance after 2024-09-15 only if the Subordinate CA itself is compliant with the S/MIME BRs. It is determined that the absence of the EKU extension itself makes this Subordinate CA non-compliant.
  • 10:30 We determine that the deployment of Ticket #1, which strengthened and refactored the usability of Subordinate CAs and their purpose, lead to the COMODO RSA Client Authentication and Secure Email CA Subordinate CA no longer being allowed to issue S/MIME certificates. We additionally review Ticket #2 and realize this bug may have led to the issuance of S/MIME certificates by other, previously disabled, Extant S/MIME CAs.
  • 10:35 We start a review of all Extant S/MIME CAs within our system.
  • 12:14 We complete an initial review of Subordinate CAs. We request a database report from our DBA team requesting an overview of all certificates issued by the identified Subordinate CAs as of September 15th, 2024.
  • 14:30 Our DBA team starts processing the request.
  • 15:26 The queries have completed and reports are sent to the compliance team for review.
  • 15:50 Compliance confirms the issuance of 991 S/MIME certificates by non-compliant Extant S/MIME CAs. Out of these, 421 expired prior to the discovery of this incident, 39 were revoked prior to the discovery of this incident, and 531 were unexpired and unrevoked.
  • 15:54 We trigger a revocation event. Revocation of unrevoked and unexpired certificates is scheduled for 2025-11-19 by 13:45 UTC. Notifications are sent out to affected Subscribers.

2025-11-17:

  • 14:36 An independent review by the senior engineer identified a further 5 affected S/MIME certificates, all of which expired prior to the discovery of this incident.

2025-11-19:

  • 12:09 We complete the revocation of all affected certificates.

Related Incidents

We did not identify any related incidents.

Root Cause Analysis

Contributing Factor #1: Compliance review of Extant S/MIME CAs did not take into account Subordinate CAs without the EKU extension

  • Description:

Compliance review of which Extant S/MIME CAs would not be allowed to issue S/MIME certificates on and after 2024-09-15 for the main part focused on the inclusion of the correct Policy OIDs, or the anyPolicy OID.

Sectigo had several Extant S/MIME CAs which contained the emailProtection EKU, but neither the anyPolicy OID nor any of the S/MIME BR Reserved Policy OIDs. As a result, these CAs were identified and replaced with a new generation. Customers were actively migrated to these new Subordinate CAs, and their account access to the previous Subordinate CA was disabled.

This review, in hindsight, overlooked the possible existence of an Extant S/MIME CA that did not contain the EKU extension at all, but rather focused on CAs that specifically included the emailProtection EKU.

  • Timeline: We performed this review in Q3 of 2023.
  • Detection: We detected this failure in our review during our investigation into this incident.
  • Interaction with other factors: This oversight directly led to the issuance by 1 out of 7 non-compliant Extant S/MIME CAs which issued S/MIME certificates on and after 2024-09-15.

Contributing Factor #2: Subordinate CA configuration possible at a per-account level

  • Description:

Up until the deployment of Ticket #1, Subordinate CA configuration was possible at a per-account level. Ticket #1 introduced new code which simplified, yet strengthened, the limitations posed upon a Subordinate CA.

  • Timeline: This was deployed in October 2025.
  • Detection: The improvements made here were part of continued strengthening of our issuance systems, and done outside of the scope of this incident. The strengthening this change brought did eventually lead to the discovery of this incident.
  • Interaction with other factors: This code change eventually led to the discovery of this incident.

Contributing Factor #3: Software bug allowed the usage of disabled non-TLS Subordinate CAs

  • Description:

Our QA department discovered a bug, which allowed accounts to utilize non-TLS Subordinate CAs without having explicit permissions for them.

This was resolved by the deployment of Ticket #2.

  • Timeline: This was deployed in October 2025.
  • Detection: This bug was discovered through routine testing by our QA department. Until the incident was discovered however, no connection was made that the bug might have allowed for misissuance.
  • Interaction with other factors: Even if we had not overlooked the non-existent EKU extension as described in Contributing Factor #1, this bug might very well still have led to continued issuance even on the Subordinate CA outlined above. In the end, this bug allowed 6 out of 7 Subordinate CAs affected by this bug, to issue S/MIME certificates.

Lessons Learned

  • What went well:
    • Customers were actively migrated to other Subordinate CAs before the requirements came into effect. This severely limited the number of affected certificates.
    • Unrelated, planned improvements to our issuance systems means we already halted issuance of non-compliant certificates prior to the discovery of the incident.
  • What didn’t go well:
    • A bug within our system allowed the usage of disabled Subordinate CAs, if a customer knew the internal ID of such a Subordinate CA.
    • We missed identifying 1 Extant S/MIME CA as being non-compliant due to the absence of the EKU extension itself.
  • Where we got lucky:
    • Even for the single Extant S/MIME CA that was not previously identified as non-compliant, we had actively migrated the majority of customers away from utilizing this Subordinate CA.
  • Additional: N/A

Action Items

Action Item Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Refactor Subordinate CA configuration management Prevent Contributing Factor # 1 & 2 Previously, Subordinate CA configuration management was done at the account level. Refactoring this is intended to strengthen the configuration changes, make these less complicated and less human-error-prone, by doing this at the issuance system level 2025-10-11 Completed
Bug-fix our Subordinate CA account enablement system to restrict usage for all types of Subordinate CAs Prevent Contributing Factor # 3 A bug within our system allowed previously disabled non-TLS Subordinate CAs to still be utilized 2025-11-08 Completed

Appendix A

See attachment #9529513 [details] for a list of affected certificates.

We are monitoring this bug for any questions and/or comments.

Report Closure Summary

  • Incident description:

We became aware that 6 of our non-compliant Extant S/MIME CAs have issued S/MIME certificates on and after September 15th, 2024. A total of 996 certificates had been issued since 2024-09-15, of which 531 were unexpired and unrevoked when we became aware of the incident.

  • Incident Root Cause(s):

5 out of 6 identified non-compliant Extant S/MIME CAs in this incident were previously marked as disabled within our system. However, a software bug allowed continued usage of these SubCAs for new issuance.

1 out of the 6 Extant S/MIME CAs was found to not previously have been identified as non-compliant with the S/MIME BRs and thus was not disabled until recently.

  • Remediation description:

An already completed refactoring of our SubCA management to lint configuration changes, and to make them less complicated and less human-error-prone, added technical constraints against the S/MIME SubCA profile requirements. Additionally, a bugfix has been implemented to prevent usage of unauthorized S/MIME SubCAs.

  • Commitment summary:

Sectigo is committed to putting technical controls in place wherever possible, as previously demonstrated in our Guard Rails project.

All Action Items disclosed in this report have been completed as described, and we request its closure.

Flags: needinfo?(bwilson)

This is a final call for comments or questions on this Incident Report.

Otherwise, it will be closed on approximately 2025-12-18.

(As a reminder, please needinfo this incident-reporting account to signal a report is ready for closure, as described here.)

Flags: needinfo?(incident-reporting)
Whiteboard: [ca-compliance] [smime-misissuance] → [close on 2025-12-18] [ca-compliance] [smime-misissuance]
Status: ASSIGNED → RESOLVED
Closed: 7 months ago
Flags: needinfo?(incident-reporting)
Flags: needinfo?(bwilson)
Resolution: --- → FIXED
Whiteboard: [close on 2025-12-18] [ca-compliance] [smime-misissuance] → [ca-compliance] [smime-misissuance]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: