1977253 - Sectigo: OV reuse data applied for wrong organization

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Tim Callan]

Preliminary Incident Report

Summary

• Incident description:
  Due to human error Sectigo applied OV reuse data for the wrong organization to one certificate. We have revoked the certificate and fixed the OV reuse data in question. We have created a ticket for a technical control to protect against this possibility in the future.

• Relevant policies:
  Baseline Requirements for the Issuance and Management of Publicly-Trusted TLS Server Certificates version 2.1.5 section 4.2.1

• Source of incident disclosure:
  The Subscriber detected the incorrect information and requested revocation and replacement. Because of this request, our customer service team saw the data mismatch and forwarded the incident to our compliance department.

Comment 1

[Martijn Katerbarg]

Attachment: Affected Certificates.csv

Comment 2

[Martijn Katerbarg]

Full Incident Report

Summary

• CA Owner CCADB unique ID: A000016
• Incident description:

On July 10th, 2025 at 05:03 UTC, we issued a certificate containing the wrong organization details due to a human error utilizing our OV re-use capabilities.

• Timeline summary:
  • Non-compliance start date: 2025-07-10 – 05:03 UTC
  • Non-compliance identified date: 2025-07-14 – 14:16 UTC
  • Non-compliance end date: 2025-07-10 – 15:34 UTC
• Relevant policies: Baseline Requirements for the Issuance and Management of Publicly-Trusted TLS Server Certificates version 2.1.5 Section 4.2.1.
• Source of incident disclosure: Third Party Reported

Impact

• Total number of certificates: 1
• Total number of "remaining valid" certificates: 0
• Affected certificate types: TLS - OV
• Incident heuristic: A single certificate was affected: https://crt.sh/?id=19564446351
• Was issuance stopped in response to this incident, and why or why not?:

We halted issuance of certificates utilizing the affected reuse method until further notice.

• Analysis: N/A
• Additional considerations: N/A

Timeline

All times are in UTC.

• 2025-07-09:
  • 16:33:17 We receive a certificate request for the certificate affected by this incident.
• 2025-07-10:
  • 05:00 One of our validation agents starts processing the certificate request.
  • 05:02 The validation agent determines the Subscriber is eligible for OV data-reuse based on time-valid available documentation. The validation agent utilizes one out of two available data-reuse mechanisms we have available internally. In the used method, the validation agents needs to manually enter an existing order ID in order to apply the applicable pre-validated data. Unbeknownst at the time, an incorrect ID is entered. The system updates the certificate request to match the organization data of the incorrectly provided ID.
  • 05:03 The affected certificate is issued.
  • 06:48 One of our sales representatives receives an email from our customer regarding an issued certificate with incorrect OV details. This sales rep. reaches out to one of our validation agents.
  • 10:14 The issued certificate is discussed in a meeting with several validation agents.
  • 14:16 A replacement certificate is issued at the request of the Subscriber.
  • 15:34 The affected certificate is revoked at the request of the Subscriber.
  • 19:52 A ticket is filed requesting an improvement to be made to the data re-use mechanism.
• 2025-07-11:
  • 20:39 The findings of the previous day are escalated by the validation team to the compliance team for analysis. The initial findings did not clearly point to the incorrectly issued certificate. The compliance team requests further details from the validation team in order to determine if, and if so, which, policies were violated.
• 2025-07-14:
  • 14:16 Validation provides additional information. Based on this additional information, the compliance team is able to identify this as an incident.
  • 14:30 A decision is made to stop usage of the affected, non-technically controlled data re-use mechanism effective immediately.
  • 19:30 We open this bug.
• 2025-07-22:
  • 09:30 Compliance requests further improvements to be made in order to add technical controls to the data re-use method.

Related Incidents

While we identified several other bugs pertaining to incorrect organization details ending up in certificates, we deem none of these to be related to the same root cause.

Root Cause Analysis

Contributing Factor #1: Human error and lack of technical control for to one out of two data re-use mechanisms**

• Description:

During the review of this incident, we realized that for one out of our two available data re-use mechanisms, there is a lack of technical controls in order to prevent human error.

For data re-use, we utilize two different methods:

Method 1 provides a “Search & Validate” button, in order to find a Pre-Validation that matches the data constituting organization pre-validation details. This method uses a search comparison function and the pending certificate request must match all fields that are specified in the pre-validation certificate for it to be used.

Method 2 allows entering a Pre-Validation Order ID and click on a "Search & Validate" button to assign the Pre-Validation Order ID.

Where Method 1 can only be used to assign existing documents for a request where the requested subject details already match, this is not the case for Method 2. Normally, this has been used to correct minor typos a request may contain. However, in this case, it caused the requested organizational details to be updated. While the included organizational details were valid of their own, they did belong to, nor were affiliated with, the Applicant.

• Timeline:

• 2025-07-10:

  • 05:02 The validation agent determines the Subscriber is eligible for OV data-reuse based on time-valid available documentation. The validation agent utilizes one out of two available data-reuse mechanisms we have available internally. In the used method, the validation agents needs to manually enter an existing order ID in order to apply the applicable pre-validated data. Unbeknownst at the time, an incorrect ID is entered. The system updates the certificate request to match the organization data of the incorrectly provided ID.
  • 19:52 A ticket is filed requesting an improvement to be made to the data re-use mechanism.

• 2025-07-14:

  • 14:30 A decision is made to stop usage of the affected, non-technically controlled data re-use mechanism effective immediately.

• 2025-07-22:

  • 09:30 Compliance requests further improvements to be made in order to add technical controls to the data re-use method.

• Detection: This case shows a single human error, which was discovered after we received an email from the Applicant of the certificate about incorrect organization data.

• Interaction with other factors: N/A.

Lessons Learned

• What went well:
  • The certificate was revoked within 12 hours of issuance.
  • We have stopped utilizing the affected re-use mechanism until technical controls are in place.
• What didn’t go well:
  • We did not have sufficient technical controls in order to prevent a human error.
• Where we got lucky:
  • The Subscriber reported the incorrectly issued certificate to us shortly after issuance.
• Additional: N/A.

Action Items

Action Item	Kind	Corresponding Root Cause(s)	Evaluation Criteria	Due Date	Status
We plan to enhance the affected data-reuse method with a visual, side-by-side comparison of existing data versus requested data, including string matching indicators requiring explicit approval prior to automatically updating any request.	Mitigate	Contributing Factor # 1	It is our expectation that increased visual aids for validation agents will help prevent and/or mitigate further human errors.	2025-08-31	Ongoing
Additional to the Action Item above, we plan on requiring a minimum percentage of string matching between the requested data and existing data, in order for approval to be technically possible.	Prevent	Contributing Factor # 1	While the first action item should reduce the likelihood of a human error occurring, we realize it won’t be a complete solution. To further restrict this, we are adding an automated string comparison mechanism which will allow for the correction of typos or minor changes but will put a firm block in place for completely different values.	2025-08-31	Ongoing

Appendix

The affected certificate details are listed in attachment #9502885 [details]

Comment 3

[Martijn Katerbarg]

Based on our action items, we'd like to request a next-update for 2025-09-01.

Comment 4

[chrome-root-program]

Thanks for filing this report. Related to the Action Items described in Comment 2, we have two questions in hopes of improving our understanding.

(Q1): What minimum string-matching percentage will be enforced by the new preventative control?

(Q2): How was this threshold determined sufficient?

Comment 5

[Martijn Katerbarg]

(In reply to chrome-root-program from comment #4)

(Q1): What minimum string-matching percentage will be enforced by the new preventative control?

This has been set at 90%, initially. Results from practical usage over time may lead us to adjust this number in time, if and when deemed necessary.

(Q2): How was this threshold determined sufficient?

For this we’ve done an analysis utilizing the same algorithm for comparison as the one that will be used in our actual environment. Specifically: EDIT_DISTANCE_SIMILARITY, based upon Levenshtein distance.

For this we used a dataset of roughly 30.000 organization names and matched it against itself. This resulted in computing roughly 878.000.000 similarity calculations.

Based on this outcome, it shows 97% fall between 0 and 30% in comparison. For the higher numbers, we found that:

• 0.00855% were between 70% and 80%
• 0.00152% were between 80% and 90%
• 0.00017% were between 90% and 100%

Our initial thinking landed on aiming for 85%. When comparing the results of the 80 to 100% results however, we noticed a fair number of wrongful similarities inside of the 85 to 88% range. As can be seen from the comparison, there is nearly a 90% drop between the 80-90 and the 90-100% ranges.

While even 90% shows some similarities, we noticed that the majority of these are in fact false positives. On top of that we also consider the human impact: This process will not be fully automated, and the automation here is merely a measure to reduce the likelihood of an error being made.

Comment 6

[S. Poppett]

Sectigo took 108 hours and 42 minutes from the time they were made aware of the problem with this certificate until posting the preliminary incident report, which is 36 hours and 42 minutes, more than a day and a half, past the required 72 hours required by the CCADB Incident Reporting Guidelines.

Q1: Why has Sectigo violated the requirement to post the Preliminary Incident Report within 72 hours?

Looking at the timeline I can see several possible root causes, but I don't wish to put words in the mouths of Sectigo's representatives, so that leads me to:

Q2: Does Sectigo agree that this constitutes a separate incident and they should therefore post a new bug to discuss root causes and a resolution plan.

Comment 7

[Martijn Katerbarg]

(In reply to S. Poppett from comment #6)

Sectigo took 108 hours and 42 minutes from the time they were made aware of the problem with this certificate until posting the preliminary incident report, which is 36 hours and 42 minutes, more than a day and a half, past the required 72 hours required by the CCADB Incident Reporting Guidelines.
Q1: Why has Sectigo violated the requirement to post the Preliminary Incident Report within 72 hours?
Looking at the timeline I can see several possible root causes, but I don't wish to put words in the mouths of Sectigo's representatives, so that leads me to:
Q2: Does Sectigo agree that this constitutes a separate incident and they should therefore post a new bug to discuss root causes and a resolution plan.

We do not agree with this assessment.

The BRs define the Certificate Problem Report mechanism for making CAs aware of problems. When a CPR is sent, it's reasonable to assume that the CA will receive it almost immediately and will therefore have been 'made aware'. The counter doesn’t start any sooner than that.

In this case however, no CPR was sent; rather, a single salesperson received an email from their customer contact, stating that the certificate did not contain the organization details as they had requested these. We do not consider the receipt of that email to be a CA Owner “receiving and corroborating” an issue.

When no CPR is filed, our internal policies dictates that any anomalies need to be reported to the compliance department for evaluation. This process was properly followed, as is clear from our timeline in comment #2.

We posted the Preliminary Incident Report 5 hours and 14 minutes after “becoming aware” of the problem.

Comment 8

[Martijn Katerbarg]

Both our pending action items have been completed on schedule.

Report Closure Summary

• Incident description:

Due to human error Sectigo applied OV reuse data for the wrong organization to a single certificate.

• Incident Root Cause(s):

A lack of technical controls combined with a human error, lead to an incorrect pre-validated organization to be assigned to a certificate request.

• Remediation description:

A visual control in the form of a side-by-side comparison with string matching visual indicators has been added, as well as a technical control preventing assigning an organization whose name differs more than 10% from the certificate request, based upon the Levenshtein distance algorithm.

• Commitment summary:

Sectigo is committed to keeping the technical controls in place, while monitoring its current use and effectiveness, tightening the percentage-based allowance if deemed necessary.

All Action Items disclosed in this report have been completed as described, and we request its closure.

Comment 9

[incident-reporting]

This is a final call for comments or questions on this Incident Report.

Otherwise, it will be closed on approximately 2025-09-11.