1645708 - QuoVadis: EV serialNumber with "none"

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Stephen Davidson]

1. How your CA first became aware of the problem.

QuoVadis was notified by a security researcher by email to our [e-mail] email address.

2. A timeline of the actions your CA took in response.

6/14/2020 18:14 GMT QuoVadis received notification by email from security researcher
8/19/2020 18:23 GMT QuoVadis acknowledges receipt to security researcher
8/19/2020 18:27 GMT QuoVadis revokes the certificate.

3. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem.

As part of https://bugzilla.mozilla.org/show_bug.cgi?id=1576283, QuoVadis previously implemented filters for this issue. The issuance of this certificate pre-dated that fix.

4. A summary of the problematic certificates.

QuoVadis issued a certificate with “none” in the EV serialNumber field on 6/15/2020. As the certificate holder is a Government Entity for which a registration number or date of creation is not available, the correct entry should have been a repetition of the content “Government Entity” which is also in the EV businessCategory field for the certificates.

The reported certificate had already been replaced on 6/11/2020 by a certificate with a correct entry in the EV serialNumber field. QuoVadis revoked the reported certificate even though the certificate had less than 12 hours remaining validity.

5. The complete certificate data for the problematic certificates.

https://censys.io/certificates/b0f0c77444438fccde95d08dbeb9b3caf6fe65aeb1e5f6f16b798e820498211a

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

This issue was previously correct as part of https://bugzilla.mozilla.org/show_bug.cgi?id=1576283. That bug describes the “filler” fields that were cleaned up as part of that bug; this certificate was not in that list.

7. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.

No other QuoVadis-issued certificates were found with “none” in the EV serialNumber field.

Comment 1

[Ryan Sleevi]

This appears to be incomplete remediation of Bug 1576283.

In that bug, QuoVadis was asked to provide a complete list of affected certificates, but apparently failed to find the cert in this report.

I’m also struggling to make sense of the dates, as they don’t make sense. The certificate is stated to have been misissued on 2020-06-15, but has already been replaced with the correct one on 2020-06-11, and that the issue predates the January 2020 remediation of bug 1576283. There appears to be a lack of attention to detail in the incident report, in addition to the retroactive scan for non-compliance.

Comment 2

[paul.leo.steinberg]

(In reply to Stephen Davidson from comment #0)

QuoVadis issued a certificate with “none” in the EV serialNumber field on 6/15/2020. As the certificate holder is a Government Entity for which a registration number or date of creation is not available, the correct entry should have been a repetition of the content “Government Entity” which is also in the EV businessCategory field for the certificates.

The reported certificate had already been replaced on 6/11/2020 by a certificate with a correct entry in the EV serialNumber field. QuoVadis revoked the reported certificate even though the certificate had less than 12 hours remaining validity.

This is, at least, inaccurate. The new certificate (https://crt.sh/?id=2936017371) has "serialNumber = CHE-108.244.581". Was this Registration Number assigned to the entity between June 15, 2018 (the notBefore date of the old certificate) and June 11, 2020 (the notBefore date of the new certificate)? Otherwise, why was that Registration Number not used at that time?

Comment 3

[Stephen Davidson]

Apologies, in cut-n-paste-haste the dates in the timeline were mangled:

6/14/2020 18:14 GMT QuoVadis received notification by email from security researcher
6/14/2020 18:23 GMT QuoVadis acknowledges receipt to security researcher
6/14/2020 18:27 GMT QuoVadis revokes the certificate

The noncompliant certificate was of course issued on 6/15/2018 not 6/15/2020. It was replaced in the standard course of its lifecycle on 6/11/2020. The replacement certificate included an updated serialNumber field containing a UID ( a standard business identifier that is now also being used for public administration units).

Comment 4

[Jeremy Rowley]

We are kicking off a project to dive into what happened. As Ryan noted, this was a repeat of the previous bug related to indication of the lack of information in the certificate. We're going to look at both how this one was missed and what happened in the process to detect them.

Comment 5

[paul.leo.steinberg]

(In reply to Stephen Davidson from comment #3)

The replacement certificate included an updated serialNumber field containing a UID ( a standard business identifier that is now also being used for public administration units).

Thank you.

Now, that serialNumber belongs to "Camera Commercio Industria, Artigianato Cantone Ticino, Ccia-TI" (https://www.uid.admin.ch/Detail.aspx?uid_id=CHE-108.244.581&lang=en), which is not equal to organizationName in that certificate (and neither is a direct translation, which would not be allowed anyway). Worse, a legal entity named "SIHK-CCIS Schweizer Industrie- und Handelskammer" seems to not exist at all! It seems to merely be an informal "union of the 19 cantonal and regional chambers of commerce in Switzerland and the Liechtenstein Chamber of Commerce" (https://www.sihk.ch/about?lang=en) with no legal status whatsoever. Even worse, the single chambers are not Government Entities but Private Organizations: "The individual chambers are organized as private law associations with a voluntary membership." You are even doing this correctly for https://crt.sh/?id=2041681790.

So, please revoke the new certificate (https://crt.sh/?id=2936017371) -- and this will probably be another incident report.

For this incident, the question remains: why did you not include that serialNumber in the old certificate (https://crt.sh/?id=540146459)? Why do yo think that “Government Entity” would have been the appropriate serialNumber?

Comment 6

[George [:fozzie]]

(In reply to paul.leo.steinberg from comment #5)

So, please revoke the new certificate (https://crt.sh/?id=2936017371) -- and this will probably be another incident report.

The leaf certificate for this is available at https://censys.io/certificates/ff029231d0c440d971309b2cacd0c94450730c4be1fc843b18601ed12181cea6, not currently on crt.sh.

Comment 7

[Stephen Davidson]

Relating to Comment 1
A manual search was made for the fillers described in https://bugzilla.mozilla.org/show_bug.cgi?id=1576283. It appears that search was caps sensitive (“NONE”) so this “none” was not picked up. The ongoing filter itself is not caps sensitive and is functioning as intended. A complete new search was performed using that filter and did not identify additional problem certs.

Relating to Comment 2
The number is a Swiss UID, which is an entity identifier administered by the Swiss Federal Statistical Office which aims to consolidate different identifier systems (such as registration or VAT) into a single identifier for companies and government entities. The number was adopted by our validation team seeking to minimize “generic” entries in the businessCategory field such as “Government Entity”.

Relating to Comment 5
"SIHK-CCIS Schweizer Industrie- und Handelskammer" is a joint project of the 19 different registration agencies in CH and LI, and this website is operated by them for their various users to verify documents filed with the Swiss Federal Department of Finance. This unusual situation resulted in the decision to select the “Government Entity” businessCategory.
Confirmation was performed that determined SIHK-CCIS was a essentially a DBA for “Camera Commercio Industria, Artigianato Cantone Ticino, Ccia-TI” the registration agency from whose offices it operates. However that information was not presented in the certificate in a compliant manner. The certificate has been corrected as https://crt.sh/?q=574e68d65a298f6bcd9392347bb2ce60823016c40e2c5906217ded88850ac462. Previous certificates have been revoked.
The certificate represents an unusual circumstance of trying to validate an entity that is jointly shared between different registration agencies, the exact entities that are typically used for validation. Considerable discussion took place regarding the known existence and its relationship between its registration agency and public sector participants. In retrospect, the outcome was incorrect.
From a procedural perspective, DigiCert has made and is making considerable steps to reduce leeway for such errors.

1. As noted previously, QuoVadis validation operations have been integrated into DigiCert’s team with its greater resources, global exposure, and focus on process.
2. As noted elsewhere, in anticipation of CABF Ballot SC 30, DigiCert has invested significantly in building a list of authorized registration/incorporating agency sources. The adoption of that list will prevent ad hoc decision making.
3. In the course of integration, on an ongoing basis QuoVadis is gaining access to additional automation tools from DigiCert that provide workflow enforcement through complicated validation trees such as EV and reduce the risks of ad hoc decision making.

Comment 8

[Stephen Davidson]

QuoVadis believes this matter has been properly resolved, and requests this bug be closed.

Comment 9

[Ben Wilson]

I intend to close this bug on or after 24-July-2020 unless I receive any further comments or questions.