Closed Bug 1896190 Opened 2 years ago Closed 1 year ago

D-Trust: Issuance of an EV certificate containing a mixup of the Subject's postalCode and localityName

Categories

(CA Program :: CA Certificate Compliance, task)

Product:
CA Program
Component:
CA Certificate Compliance
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: enrico.entschew, Assigned: enrico.entschew)

Details

(Whiteboard: [ca-compliance] [ev-misissuance])

Incident Report

Summary

This is a preliminary report.

D-Trust issued an EV certificate containing a mixup of the Subject's postalCode and localityName. D-Trust was informed by a third party regarding this issue on 2024-05-09.

The impacted certificate will be revoked on time within the 5-day period.

Impact

One EV certificate is affected and needs to be revoked within the 5-day period.

Timeline

All times are UTC.

2024-05-09:

  • 00:49 Email with certificate problem report regarding the affected certificate
  • 05:09 Acknowledgement of receipt of certificate problem report
  • 05:30 Start of internal analysis
  • 18:31 Customer communication
  • 22:00 Communication with third party, confirmation of the incident and that a revocation of the affected will be carried out on time

Root Cause Analysis

We are still investigating and will provide detailed information with an update.

Lessons Learned

What went well

We are still investigating and will provide detailed information with an update.

What didn't go well

We are still investigating and will provide detailed information with an update.

Where we got lucky

We are still investigating and will provide detailed information with an update.
*

Action Items

Action Item Kind Due Date
We are still investigating and will provide detailed information with an update.

Appendix

Details of affected certificates

List of all affected certificates

https://crt.sh/?sha256=DE37383B8D8463C9B12980855DF4E77C690763CD99899440A1C4F4AD69FE2F3E

Based on Incident Reporting Template v. 2.0

Assignee: nobody → enrico.entschew
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Whiteboard: [ca-compliance] [ev-misissuance]

Quick update:
We revoked the affected certificate on 2024-05-13, 13:07 UTC.

Incident Report

Summary

This is an update to the preliminary report.

D-Trust issued an EV certificate containing a mix-up of the Subject's postalCode and localityName. D-Trust was informed by a third party regarding this issue on 2024-05-09.

We revoked the impacted certificate on time.

Impact

One EV certificate is affected and got revoked within the 5-day period.

Timeline

All times are UTC.

2023-02-15:

  • 14:43 Initially approved applicant data

2023-10-31:

  • 10:22 Annual check of the approved data

2024-05-09:

  • 00:49 Email with certificate problem report regarding the affected certificate
  • 05:09 Acknowledgement of receipt of certificate problem report
  • 05:30 Start of internal analysis
  • 18:31 Customer communication
  • 22:00 Communication with third party, confirmation of the incident and that a revocation of the affected will be carried out on time

2024-05-10:

  • 04:52 Change of approved data

2024-05-13:

  • 14:07 Revocation of affected certificate

2024-05-16:

  • 10:57 Conformity Assessment Body was informed about the issue.

Root Cause Analysis

The mix-up happened during the application process and was not discovered by the validation specialist. The respective fields in the application form are free text fields. There was no check on the field in the certificate management system. Because of this, the validation specialist did not receive a warning.

Lessons Learned

What went well

We informed the applicant in timely manner. The dedicated certificate got revoked in timely manner.

What didn't go well

The error was not discovered by the validation specialists during the annual check of the approved data.

Where we got lucky

Only one certificate was affected.

Action Items

Action Item Kind Due Date
Implementation of a postcode check Prevent 2024-09-30
Employee training on various validation aspects Prevent 2024-05-10 and 2024-05-13

Some questions:

  1. As part of your review were the other active certificates checked for similar issues?
  2. Do you have any idea of why this was not picked up during an annual check, and what processes will be changed to make sure this doesn't happen again?
  3. Was issuance stopped between the impact being discovered and suitable countermeasures being put in place to stop this reoccurring?
Flags: needinfo?(enrico.entschew)

Hi Wayne,
I will answer you directly under the respective question.

(In reply to Wayne from comment #3)

Some questions:

  1. As part of your review were the other active certificates checked for similar issues?

This was the first action as part of the incident analysis. All current EV certificates were checked to see whether a mix-up had also taken place there.

  1. Do you have any idea of why this was not picked up during an annual check, and what processes will be changed to make sure this doesn't happen again?

Our investigation came to the following conclusion: As part of the revalidation process, the validation specialists check the validity of all values.

The validation specialists check all attributes. For this purpose, proof is provided for the validation specialists or they obtain it themselves.
In this specific case, we suspect that the validation officers checked the attributes. The content of all attributes were correct, but in one case the fields were mixed up. The validation officers validated the attributes in the Certificate Management System form without paying attention to the field names. As these are free text fields, there was no error message.

We will remove the free text field in the zip code area. A verification rule will be added to prevent these kind of mix-ups in the future.

  1. Was issuance stopped between the impact being discovered and suitable countermeasures being put in place to stop this reoccurring?

No new certificates could be issued at the time that the existing certificates were being reviewed. The responsible validation specialists were informed about the bug and the planned changes. Two training sessions took place, where the validation specialists were made aware to pay more attention to the content and the respective field name.

Flags: needinfo?(enrico.entschew)

Summary

This is the final incident report.

D-Trust issued an EV certificate containing a mix-up of the Subject's postalCode and localityName. D-Trust was informed by a third party regarding this issue on 2024-05-09.

We revoked the impacted certificate on time.

Impact

One EV certificate is affected and needs to be revoked within the 5-day period.

Timeline

All times are UTC.

2023-02-15:

  • 14:43 Initially approved applicant data

2023-10-31:

  • 10:22 Annual check of the approved data

2024-05-09:

  • 00:49 Email with certificate problem report regarding the affected certificate
  • 05:09 Acknowledgement of receipt of certificate problem report
  • 05:30 Start of internal analysis
  • 05:30 Stop issuance of EV certificates
  • 09:20 Restart issuance of EV certificates up to the specific customer (need for change of approved data)
  • 18:31 Customer communication
  • 22:00 Communication with third party, confirmation of the incident and that a revocation of the affected will be carried out on time

2024-05-10:

  • 04:52 Change of approved data
  • 07:00 Employee training on various validation aspects

2024-05-13:

  • 14:07 Revocation of affected certificate
  • 07:00 Employee training on various validation aspects

2024-05-16:

  • 10:57 Conformity Assessment Body was informed about the issue.

2024-05-21:

  • 14:30 End of internal analysis

Root Cause Analysis

The mix-up happened during the application process and was not discovered by the validation specialist. The respective fields in the application form are free text fields. There was no check on the field in the certificate management system. Because of this, the validation specialist did not receive a warning.

Lessons Learned

What went well

We informed the applicant in timely manner. The dedicated certificate got revoked in timely manner.

What didn't go well

The error was not discovered by the validation specialists during the annual check of the approved data.

Where we got lucky

Only one certificate was affected.

Action Items

Action Item Kind Due Date
Implementation of a postcode check Prevent 2024-09-30
Validation specialist training on various validation aspects Prevent 2024-05-10 and 2024-05-13
Whiteboard: [ca-compliance] [ev-misissuance] → [ca-compliance] [ev-misissuance] Next update 2024-08-15

Quick update: On 06/17/2024 we implemented the postcode check for Germany. It is running in our test system and our production system. We are planning to expand the postcode check to all states of the European Economic Area as this is our target market. This should be finished by the end of September 2024.

Quick update: Nothing new to report. I kindly request to give the next update on 09/01/2024.

Flags: needinfo?(bwilson)
Flags: needinfo?(bwilson)
Whiteboard: [ca-compliance] [ev-misissuance] Next update 2024-08-15 → [ca-compliance] [ev-misissuance] Next update 2024-09-01

We are still on track. The updated version of the postcode check is still planned for September. Work on this is advancing quickly.

Work on the postcode check is still advancing quickly. Unfortunately, the updated version will be installed in October as the postcode check is part of a greater update. This update will be delayed due to some improvements which are not related with the postcode check.

Whiteboard: [ca-compliance] [ev-misissuance] Next update 2024-09-01 → [ca-compliance] [ev-misissuance] Next update 2024-10-21

Quick update: The planned update was successfully installed and the extended postcode check for all countries in the European Economic Area is running in our test system and our productive system.

Whiteboard: [ca-compliance] [ev-misissuance] Next update 2024-10-21 → [ca-compliance] [ev-misissuance]

We carried out all the measures mentioned under the action points. I kindly ask you to check whether this incident can be closed.

Flags: needinfo?(bwilson)

I'll look at closing this on Wed. 6-Nov-2024. Any additional questions or issues should be raised before then.

Status: ASSIGNED → RESOLVED
Closed: 1 year ago
Flags: needinfo?(bwilson)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.