1888016 - Digicert: Failure to include CPS URI in 1 certificate

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Jeremy Rowley]

Steps to reproduce:

Digicert Incident Report

Summary

Digicert issued one EV TLS certificate to an affiliate company with two issues. First, the certificate failed to include a CPS URI, which is required under the EV Guidelines. Second, the certificate was a Qualified Website Authentication Certificate (QWAC), and the certificate’s jurisdiction of incorporation information did not align with organizationID.

Impact

A single certificate was mis-issued during a demonstration of our new pre-production system designed to support the EU. The issue would affect all QWAC certificates that were also EV certificates issued through a European instance of the CA system. Issuance was limited to a single certificate issued through the system during an audit.

Timeline

In December, Digicert underwent an ETSI audit on a new system used to separate European data and services from the rest of the world and was developed to support its Qualified services. The timeline:

2023-12-12 8:22pm, Digicert issued an EV TLS certificate that matched the system’s QWAC profile.

2024-3-25 3:47pm, Digicert received notice from Ryan Dickson of Google of the mis-issued certificate.

2024-3-25 4:09pm, Digicert acknowledged the report and began investigating the root cause.

2024-3-25 4:23 pm, Digicert revoked the certificate.

2024-3-25 7:10 pm, Digicert discovered an error in its code where Qualified was skipping PKIlint despite being a TLS certificate. Investigation showed that QWACs was listed as a different certificate profile type than TLS. Our internal linting system detected the different profile and decided this was not a TLS certificate, skipping submission to PKIlint.

2024-3-25 11:16am, Digicert, while reviewing the incident report, realized the state identifier was missing from the certificate’s organizationID field despite being included in the JOI information. Digicert conducted the investigation and determined that the rules for what happens with US entities isn’t defined in EU qualified as US states aren’t part of the profile.

Root Cause Analysis

The root cause was a bug in our EU-systems code where the internal linter logic did not send non-TLS and non-SMIME certificates to PKIlint. Because the system through QWACs was neither TLS nor SMIME, the bug caused the CA to ignore PKIlint. Our US version of the same system does not issue QWACs certificates so was unimpacted by the bug. Similarly, the root cause for the JOI information not matching the organizationID resulted from QWACs being only issued to EU entities, not US entities. The system processed the certificate as an EU entity and failed to include the US state (as US states are not defined). was unsure on how to process a US entity and failed to include the state information in the organizationID used in qualified certificates. We are still researching the best way to address this issue.

Lessons Learned

We learned to double check and clarify certificate types. We also learned that we need to send all cert types through the linter, regardless of whether the certificate is classified internally as S/MIME or TLS.

What Went Well

Qualified certificates are generally not being issued through this system. They are still being issued on legacy QuoVadis systems. That limited the impact to 1 certificate.

What Did Not Go Well

The certificates did not go through PKIlint depsite PKIlint being operational and used for all non-qualified TLS certificates. Using profiles to define types instead of the requirements means some cert profiles could potentially be mis-categorized. We are modifying the code so that all certificates go through the linter, regardless of profile. We expect this code to go live on March 27th.

Another issue is that demo-ed the system with a US entity instead of a European entity. The EU qualified profiles are built around European requirements.

Where We Got Lucky

We have not started full production issuance from our new system. Most EU certificates are still issued from the legacy QuoVadis system.

Action Items

We are currently deploying three patches. The first will fix the profile for QWACs to include the CPS URI. The second will send all Qualified certificates through PKIlint. The third will address the JOI-orgID mismatch. We are still deciding what that third fix should look like. Options under consideration include prohibiting US entities or including the state information.

In addition, Digicert is contributing new lints (in June) to the open source PKIlint framework to account for the additional profile requirements of QWAC certificates (such as qcStatement extensions). Although this will not impact the Mozilla requirements for certificates, this will provide regulatory authorities and other CAs their own Qualified linting abilities.

Appendix

Details of affected certificates

https://crt.sh/?sha256=1c0c9604e8e070d96a3b49f2df3857af17e43f174fa2f725ee886d305af7c0b2

Comment 1

[Martin Sullivan]

DigiCert is monitoring this Bug for any questions

Comment 2

[Jeremy Rowley]

We disabled QWACs in the new platform pending a fix to the JOI-orgID mismatch issue. We currently do not plan to renable in the near future. If we do, we will put a patch in place to ensure the two fields match.

We are still working on sending all remaining publicly trusted certificates through our linter (called Digilint). Most certificates already go through the linter. The two exceptions are our client authentication and document signing certificate profiles, so the risk of an issue that the profiles could encompass the SMIME or TLS BRs is low. This project should be complete by the end of May.

Ben, given that the last two types of certificates are out of scope of the Mozilla policy, do you want to close this bug or do you want to wait until end of May when the project is complete? If you want to wait, could we set the next update to June 1?

Comment 3

[Martin Sullivan]

As of today all publicly trusted certificates are now being sent through our linter

if there are any questions we are happy to answer them.

Else Ben can we close this off?

Comment 4

[Ben Wilson]

I'll schedule to close this next Wed. 2024-06-05 unless there are additional comments or questions.

Comment 5

[Wayne]

The only note I would make is ensuring these action items are all complete:

Action Items

We are currently deploying three patches. The first will fix the profile for QWACs to include the CPS URI. The second will send all Qualified certificates through PKIlint. The third will address the JOI-orgID mismatch. We are still deciding what that third fix should look like. Options under consideration include prohibiting US entities or including the state information.

In addition, Digicert is contributing new lints (in June) to the open source PKIlint framework to account for the additional profile requirements of QWAC certificates (such as qcStatement extensions). Although this will not impact the Mozilla requirements for certificates, this will provide regulatory authorities and other CAs their own Qualified linting abilities.

Comment 6

[Ben Wilson]

I'll close this with the understanding that patching of linters is ongoing and taking place, as noted in Comment #2 and requested in Comment #5.