1519260 - QuoVadis: Multiple unreported misissuances in 2018

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Stephen Davidson]

During 2018, QuoVadis commenced post-issuance linting of SSL issuance. At year end 2018, we undertook an additional re-lint of 2018 issuance to ensure that all problem certificates were revoked and issues were remediated. In some cases, changes to available lints and presentation in reports such as crt.sh provide new focus on existing errors.

For the sake of clarity, QuoVadis implemented changes to our certificate management system (CMS) to resolve issues as they were picked up through linting. However, Bugzilla disclosures were not always made on issues. As the emphasis on disclosures has increased, for the sake of transparency, we are filing now filing a recap of 2018 problem certificates we identified (ie, triggering an ERROR in zLint).

At the end of 2018, some pre-issuance lints (mainly focussed on technical aspects such as keys) were added to production. While continuing post-issuance linting using crt.sh and our own internal alerting tool, QuoVadis intends to continue working on pre-issuance lints in 2019.

A. ERROR: DNSNames must have a valid TLD

1. How your CA first became aware of the problem, and the time and date.

The certificate was identified on Mar 27 (the day of issuance) using linting.

2. A timeline of the actions your CA took in response.

The certificate was revoked on Mar 28.

3. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.

The issue that allowed the issuance was corrected in Mar 2018 concurrent with the discovery of this certificate.

4. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.

https://crt.sh/?id=367721342&opt=zlint issued on March 27, 2018.

5. The complete certificate data for the problematic certificates.

Above.

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

An issue was identified in our CMS that allowed IP addresses to be inserted in SAN dnsName fields.

7. List of steps CA is taking to resolve the situation and ensure it will not be repeated.

Above.

B. ERROR: Too many characters in Subject fields

1. How your CA first became aware of the problem, and the time and date.

We have grouped several error types together given the common cause.

ERROR: The 'Organizational Unit Name' field of the subject MUST be less than 64 characters
ERROR: The commonName field of the subject MUST be less than 64 characters
ERROR: The 'Serial Number' field of the subject MUST be less than 64 characters

QuoVadis became aware of the 10 problem certificates using post issuance linting, and those certificates were revoked at that time. However our year end check identified 3 certificates that were issued in 2018 before we commenced linting, which are in the process of being revoked now.

2. A timeline of the actions your CA took in response.

Our CMS included restrictions on field lengths for some but not all certificate fields – as some (such as OU) were considered low risk as the certificates displayed without issue in browsers. With the advent of linting, the classification of these as errors became clearer and field lengths were tightened up in our CMS in July and August 2018.

3. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.

The issue has been corrected.

4. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.

ERROR: The 'Organizational Unit Name' field of the subject MUST be less than 64 characters
Eight certificates: https://crt.sh/?zlint=934&iCAID=1683&minNotBefore=2018-01-01 and https://crt.sh/?id=379734504&opt=zlint

ERROR: The commonName field of the subject MUST be less than 64 characters
Single certificate: https://crt.sh/?id=611373213&opt=zlint
Note: this was a test certificate (as upper bounds for SAN dnsName are different than CN).

ERROR: The 'Serial Number' field of the subject MUST be less than 64 characters (note refers to the EV registration number in the Subject DN).
Single certificate: https://crt.sh/?id=402962402&opt=zlint

5. The complete certificate data for the problematic certificates.

Above.

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

Above.

7. List of steps CA is taking to resolve the situation and ensure it will not be repeated.

Above.

C. FATAL: asn1: syntax error: PrintableString contains invalid character

1. How your CA first became aware of the problem, and the time and date.

The certificate was identified via post issuance linting on February 25.

2. A timeline of the actions your CA took in response.

The certificate was revoked on February 26.

3. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.

The issue has been corrected through training.

4. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.

https://crt.sh/?id=340935621&opt=zlint

5. The complete certificate data for the problematic certificates.

Above.

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

The EV registration number field (in the subject DN) has been left relatively free form to accommodate the wide variety of registration identifiers in use which range from numbers to long names of Parliamentary Bills. In this case, the validation operator used a backslash within the field to separate elements.

7. List of steps CA is taking to resolve the situation and ensure it will not be repeated.

Above.

Comment 1

[Ryan Sleevi]

Wayne: From a process point, would you rather split this into three reports or to keep it as a single report? I'm inclined to split, given the distinct nature of each of these.

Comment 2

[Wayne Thayer]

I'm okay with leaving this as one bug. There is precedent for grouping multiple flavors of misissuance reported at the same time into a single bug, so IMO we should make it a policy if we want to change this practice.

Stephen: please provide:

1. timing for the preissuance linting implementation that will prevent future occurrences.
2. an explanation of why it was considered acceptable not to report these misissuances as they occurred, and what is being done to prevent that from happening again.

Comment 3

[Stephen Davidson]

Thanks Wayne and Ryan:

1. Timing for the preissuance linting implementation that will prevent future occurrences.

We implemented our own post-issuance linting tool in 2018 to be able to monitor external subCAs more effectively. From an internal perspective, we are already using limited preissuance linting for some technical features of keys and certificates. We are currently testing a broader pre-issuance linting tool that will be implemented across all QuoVadis TLS-issuing CAs by June 2019.

2. An explanation of why it was considered acceptable not to report these misissuances as they occurred, and what is being done to prevent that from happening again.

As linting tools have added new lints, and linting report formats have changed, existing CA issues were highlighted (sometimes in past issuance) for the first time. Some of the QuoVadis-associated 2018 errors were reported, however for right or wrong, some errors were considered to be low risk/occurrence so our focus was on remediating the issues promptly. A review of similar errors across the industry shows many cases also undisclosed across the spectrum of CAs. In the course of 2018, there was a clarification of browser requirements for disclosure of errors, so QuoVadis posted this year in review in order to be transparent. QuoVadis commits to continue this disclosure transparency.

Comment 4

[Wayne Thayer]

Stephen: please update this bug when pre-issuance linting is implemented across all QuoVadis TLS-issuing CAs.

Comment 5

[Stephen Davidson]

Pre-issuance linting has been implemented for QuoVadis trusted TLS policies effective April 16, 2019.

Comment 6

[Wayne Thayer]

It appears that remediation is complete.