1563917 - QuoVadis: use of Organisationidentifier field in EV (Pre CABF Ballot SC17)

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Stephen Davidson]

1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.

QuoVadis had issued a number of Qualified Web Authentication Certificates (QWAC) with the EV OID that contained Organisationidentifier fields. Since QWAC are only issued to legal persons and not natural persons we referenced ETSI EN 319 412-3, which requires Organisation Identifier in addition to ETSI EN 319 412-4. We ceased this practice when the CA/Browser Forum began discussing this matter in the leadup to CABF Ballot SC17, enabling new compliant certificate profile templates in our certificate management system. However, we did not entirely remove the prior templates as we believed Ballot SC17 would shortly re-enable use of the field. The passage of Ballot SC17 took longer than anticipated.

In the intervening period, four certificates for three clients were manually issued by a QuoVadis admin/RA based upon the previous certificate profile. The certificates were identified during audit procedures.

2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.

Certificates identified in audit Verbally on 6/6/2019, in writing 6/24/2019
Clients contacted 6/7/2019
1st certificate revoked 6/7/2019
2nd certificate revoked 6/18/2019 (was replaced in use prior to that date)

The client using the 3rd & 4th certificates experienced difficulties replacing the certificate which delayed revocation. Sudden revocation of the certificates would have created significant business damage to the client. It was eventually determined that an important relying party (a regulatory entity) had pinned the original certificates.

3rd & 4th certificates revoked 7/2/2019

3. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.

The original policy settings were restored in production following the effective date of Ballot SC17 on June 21 2019. QuoVadis will enable the additional cabfOrganizationIdentifier field described in Ballot SC17 before January 31 2020.

4. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.

https://crt.sh/?id=1327452233
https://crt.sh/?id=786499026
https://crt.sh/?id=1005857079
https://crt.sh/?id=1005857057

5. The complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem.

See 4 above.

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

See 1 above. The certificates were identified in audit procedures.

7. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.

A change to the certificate management system is being worked on (with a target implementation of July 31) to enforce additional review of changes to certificate profile templates, and their proper cascading effect through other child certificate profile templates.

Comment 1

[Ryan Sleevi]

Stephen: Thanks for reporting.

I’m not seeing any discussion about the steps to prevent future delays of revocation. In particular, what steps are you, as a CA, taking to ensure compliance with the BRs, and customer knowledge of the risks? Looking through https://wiki.mozilla.org/CA/Closed_Incidents shows that this is not the first time this has happened; what is being done to ensure it is the last time?

Brenda: Can you confirm that this is part of the hierarchy DigiCert acquired and is responsible for, per https://www.digicert.com/news/digicert-completes-purchase-of-quovadis-ssl/ ?

Comment 2

[Brenda Bernal]

Ryan: In response to your question on Comment 1, yes this is part of the hierarchy that DigiCert has acquired. Stephen will provide a response to your other questions. Thanks.

Comment 3

[Stephen Davidson]

QuoVadis accepts responsibility for this revocation taking longer than the five days allowed in the BR. We made the decision to allow more time based on facts available to us that a rushed revocation would create disproportionate damage to the certificate holder. If I recall, the discussion at the CABF meeting in Mountain View recently covered this issue and it was agreed that the CA is in the best position to make these determinations. Our external auditors are informed. This is not a decision we made lightly, nor intend to allow routinely.

For the final two certificates, considerable effort was devoted to determine the issue preventing simple replacement of the certificates. The issue was eventually determined to be caused by (previously unknown) certificate pinning by the relying party of the certificates, a government regulator. These two certificates are not intended for use by the public or in a browser.

The miss-issuance here is an unusual case as Ballot SC17 formalized the use of the Organisation Identifier field in EV certs: in other words the profile of the “properly issued” certificates is functionally identical to the “miss-issued” certificates and the replacement certificates include exactly the same Subject DN info.

QuoVadis operates under its own CP/CPS at this time but is owned by DigiCert. In the process of integration, QuoVadis is leveraging the greater standards and compliance resources, practices, and tools of DigiCert. We are adhering to DigiCert's proactive stance towards reporting, while we understand that there are many similar QWACs currently valid from other CAs.

Comment 4

[Ryan Sleevi]

I want to correct the record regarding CA/Browser Forum: Mozilla's policy positions are documented at https://wiki.mozilla.org/CA/Responding_To_An_Incident#Revocation . In particular, I want to call specific attention to something that has been repeatedly clarified: However, Mozilla does not grant exceptions to the BR revocation requirements.

You can see the discussion about what the expectations are for when a CA makes a delay. This is not an acceptable response. This is further troubling because DigiCert is well aware that this is not an acceptable response, because the policy was itself a response to clarify and formalize after DigiCert's repeated failures to revoke.

While I understand there may be an organizational split of some form, DigiCert has accepted responsibility for ensuring compliance of the operations of these CAs, and such operation reflect poorly upon all DigiCert-operated CAs.

Please carefully review that policy and ensure that you have met the bare minimum of expectations in the follow-up response.

Comment 5

[Ryan Sleevi]

Jeremy/Brenda: From a compliance perspective, in light of DigiCert's delays with respect to underscores, this is not acceptable. While I can understand the organizational challenges that may exist from the integration of these systems, DigiCert's responsibility extends to any and all sub-CAs it acquires, and the same expectations regarding controls for past misissuance are extended.

From an incident reporting perspective, you need to work very closely with Stephen to ensure that it not only reflects QuoVadis, but DigiCert, adequately. I want to see clear timelines for if and when QuoVadis will be aligned with DigiCert with respect to compliance. If that is not on the roadmap, then I want to see clear timelines for what and when QuoVadis will be implementing equivalent controls. Repeated failures to revoke in a timely fashion are a serious and worrying sign about the CAs' operations, and you don't get a few mulligans for each and every CP/CPS you may operate under.

Comment 6

[Jeremy Rowley]

Thanks for bringing this to my attention Ryan. We've moved compliance control to DigiCert proper now, and will be providing training to the Quovadis team. Brenda, with Stephen's help while we integrate the two companies, will be managing all incident reports for Quovadis going forward. I'll work with Brenda to prepare a follow-up incident report that also talks about remediation on how the revocation requirements were missed and the plan on how to ensure that timeline is maintained consistently throughout the organization.

Comment 7

[Brenda Bernal]

Ryan, In response to your comment 5:
We have already started the alignment and integration of Quovadis into the Compliance process and operations. We plan to have Compliance notification and education specifically on the revocation process, no later than 29-July-2019, to ensure this is universally followed across DigiCert.

Comment 8

[Brenda Bernal]

Update on steps completed: Compliance communicated a notice regarding revocation requirements and conducted education on the revocation process with QuoVadis personnel.

Comment 9

[Wayne Thayer]

It appears that all questions have been answered and remediation is complete.