1390997 - GlobalSign: Non-BR-Compliant Certificate Issuance - metadata-only subject fields

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Kathleen Wilson]

The following problems have been found in certificates issued by your CA, and reported in the mozilla.dev.security.policy forum. Direct links to those discussions are provided for your convenience.

To continue inclusion of your CA’s root certificates in Mozilla’s Root Store, you must respond in this bug to provide the following information:
1) How your CA first became aware of the problems listed below (e.g. via a Problem Report, via the discussion in mozilla.dev.security.policy, or via this Bugzilla Bug), and the date.
2) Prompt confirmation that your CA has stopped issuing TLS/SSL certificates with the problems listed below.
3) Complete list of certificates that your CA finds with each of the listed issues during the remediation process. The recommended way to handle this is to ensure each certificate is logged to CT and then attach a CSV file/spreadsheet of the fingerprints or crt.sh IDs, with one list per distinct problem.
4) Summary of the problematic certificates. For each problem listed below: number of certs, date first and last certs with that problem were issued.
5) Explanation about how and why the mistakes were made, and not caught and fixed earlier.
6) List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.
7) Regular updates to confirm when those steps have been completed.

Note Section 4.9.1.1 of the CA/Browser Forum’s Baseline Requirements, which states:
“The CA SHALL revoke a Certificate within 24 hours if one or more of the following occurs: …
9. The CA is made aware that the Certificate was not issued in accordance with these Requirements or the CA’s Certificate Policy or Certification Practice Statement; 
10. The CA determines that any of the information appearing in the Certificate is inaccurate or misleading; …
14. Revocation is required by the CA’s Certificate Policy and/or Certification Practice Statement; or 
15. The technical content or format of the Certificate presents an unacceptable risk to Application Software Suppliers or Relying Parties (e.g. the CA/Browser Forum might determine that a deprecated cryptographic/signature algorithm or key size presents an unacceptable risk and that such Certificates should be revoked and replaced by CAs within a given period of time).

However, it is not our intent to introduce additional problems by forcing the immediate revocation of certificates that are not BR compliant when they do not pose an urgent security concern. Therefore, we request that your CA perform careful analysis of the situation. If there is justification to not revoke the problematic certificates, then explain those reasons and provide a timeline for when the bulks of the certificates will expire or be revoked/replaced. 

We expect that your forthcoming audit statements will indicate the findings of these problems. If your CA will not be revoking the certificates within 24 hours in accordance with the BRs, then that will also need to be listed as a finding in your CA’s BR audit statement.

We expect that your CA will work with your auditor (and supervisory body, as appropriate) and the Root Store(s) that your CA participates in to ensure your analysis of the risk and plan of remediation is acceptable. If your CA will not be revoking the problematic certificates as required by the BRs, then we recommend that you also contact the other root programs that your CA participates in to acknowledge this non-compliance and discuss what expectations their Root Programs have with respect to these certificates.


The problems reported for your CA in the mozilla.dev.security.policy forum are as follows:

** Certificates with metadata-only subject fields (at least one subject field that only contains ASCII punctuation characters)
https://groups.google.com/d/msg/mozilla.dev.security.policy/Sae5lpT02Ng/-lsC11JnBwAJ
Prevent further issuance of certs with N/A and other metadata but revocation not necessary in this case.

Comment 1

[Jonathan Rudenberg]

Additionally, I was unable to find a public incident report sent to Mozilla for certificates with invalid dnsNames containing "double dots"  detailed and listed in the links below.

https://groups.google.com/d/topic/mozilla.dev.security.policy/5bpr9yBgaYo/discussion
https://misissued.com/batch/13/

Comment 2

[Linus Hallberg]

Thank you for for providing the report listing GlobalSign issued certificates that contain only meta data in the OU field. The report lists SSL certificates under the following GlobalSign CAs and in the following quantities:

GlobalSign Extended Validation CA - SHA256 - G2: 
12 Still valid
2 Expired

GlobalSign Extended Validation CA - SHA256 - G3: 
8 Still valid

GlobalSign Organization Validation CA - SHA256 - G2: 
120 Still valid
10 Expired

In late November 2016 we implemented checks to prohibit OU fields containing just ".","-" and " " (i.e space). The reported list of certificate confirms that no certificates with these characters were issued after this fix. The only other certificates listed on this report after November 2016 are 4, where the OU field contained just "*".

We recently identified a certificate that contained an "_" in the OU field, crt.sh id: https://crt.sh/?id=42510586. Based on this we initiated plans on the 16th of August 2017 to expand the scope of the checks beyond the 3 characters in the initial release, this is scheduled to be implemented on the 28th of this month. The full list of values we will preclude from the OU field is: ASCII 32 - 47, 58 - 64, 91 - 96,  123 – 126.

As stated in the ticket, revocations are not deemed to be necessary, as such no action was taken to revoke the certificates in question.

We feel this response should close out this incident.

Comment 3

[Ryan Sleevi]

(In reply to Linus Hallberg from comment #2)
> We feel this response should close out this incident.

Have I missed answers to Requests 1-7 on Comment #0?

Further, regarding revocation, please note Comment #0 specifically requests:

> If there is justification to not revoke the problematic certificates, then explain those reasons and provide a timeline for when the bulks of the certificates will expire or be revoked/replaced.

Comment 4

[Jonathan Rudenberg]

(In reply to Linus Hallberg from comment #2)
> We feel this response should close out this incident.

Can you also please provide a full response to each question for the issue described in comment 1?

Comment 5

[Ryan Sleevi]

Is there any follow-up or acknowledgement?

Comment 6

[douglas.beattie]

We're working on collecting the info for issue where meta-data only was included in some fields so we can provide a detailed list of the certificates including expiration dates.  We'll provide daily status updates here on out, sorry for the delay but we're on it now.  Same goes for comment 1? which should really be in a different incident so it can be tracked separately.

Comment 7

[Kathleen Wilson]

The CA has requested that we separate this into 3 bugs:
1) This bug will be for the metadata-only subject fields problem.
2) New bug for the double-dots problem (https://misissued.com/batch/13/)
3) New bug for RSA key smaller than 2048 bits (https://misissued.com/batch/17/)

I am fine with separating these out into separate bugs if the CA wants to handle it that way. So I will file the two new bugs.

Comment 8

[Kathleen Wilson]

(In reply to Kathleen Wilson from comment #7)
> The CA has requested that we separate this into 3 bugs:
> 1) This bug will be for the metadata-only subject fields problem.

> 2) New bug for the double-dots problem (https://misissued.com/batch/13/)

Bug #1393555

> 3) New bug for RSA key smaller than 2048 bits
> (https://misissued.com/batch/17/)

Bug #1393557

Comment 9

[douglas.beattie]

Hello Ryan,
Sorry, please let me expand and break it down point by point based on updated reporting and analysis.

1q) How your CA first became aware of the problems listed below (e.g. via a Problem Report, via the discussion in mozilla.dev.security.policy, or via this Bugzilla Bug), and the date.

1a) There were 2 discovery events:
9 March 2016 - We initially identified that some certificate OU fields that contained just meta data characters consisting of ".","-" and " " (i.e space) during one of our internal audits.

16 August 2017 - We identified that the character "_" had been used as the sole content for the OU field via our post issuance lint checker.

2q) Prompt confirmation that your CA has stopped issuing TLS/SSL certificates with the problems listed below.

2a) We stopped issuing certificates with OU fields containing only ".","-" and " " (i.e space) as of 25 October 2016. At the time that meta data characters in the OU field were found, it was not deamed a compliance issue due to the way the BRs defined the application of the meta data rule being applicable to "Other Subject Attributes". 

As far as the issue with "_" being included in the OU field: The fix will be installed on 28 August 2017. Operational work practices were immediately updated to prohibit the use of the identified character set and we will verify that no new certificates with metadata only in subject fields are issued.

3q) Complete list of certificates that your CA finds with each of the listed issues during the remediation process. The recommended way to handle this is to ensure each certificate is logged to CT and then attach a CSV file/spreadsheet of the fingerprints or crt.sh IDs, with one list per distinct problem.

3a) Please find a CSV with these fingerprints attached to this issue. The logging to CT should be finalized by end of the day 25 August 2017.

4q) Summary of the problematic certificates. For each problem listed below: number of certs, date first and last certs with that problem were issued.

4a) From 1 July 2012 through 24 August 2017, there were 1897 certificates issued with only meta data in a field of a certificate.

Breakdown:
- 1601 are either expired or revoked
- 296 remain active. Of the active certificates, 271 are OV with the following mix of fields with meta data only: 268 "OU" and 3 "S" Fields. The last will expire on 5 March 2020.
The remaining 25 active certificates are EV with the following mix of fields with meta data only: 1 "L", 1 "S", 17 "OU" and 6 "Address". The last will expire on 12 July 2019.

5q) Explanation about how and why the mistakes were made, and not caught and fixed earlier.

5a) The certificates were issued due to a following factors.

Up until 25 October 2016, meta data could be entered as the sole content in the OU field which included characters such as "-", " " (i.e. space) or "."

The fix applied on the 25 October 2016 covered the scope originally identified: ".","-" and " " (i.e space) and no additional certificates were issued with these characters since that date.

Meta data in the non OU field was entered incorrectly by the validation specialists, primarily due to the system requiring data to be entered into fields that were actually optional (S, L, and Address). The meta data was used to fill optional fields.

6q) List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.

6a) We have recently implemented a post issuance certificate lint checker that provides notifications if a certificate was issued that isn't compliant with our checks which include cablint plus additional checks we have added. While cablint by itself would not have detected this issue, our certlint would, and does (as demonstrated above with the notification that we issued a certificate with "_" in an OU field).

We remain focused on adding a preissuance check and have been working with our CA vendor to provide the applicable hooks to validate precertificates as soon as feasible.

7q) Regular updates to confirm when those steps have been completed.

7a) We'll confirm the date of the current fix when it's put into production for more robust set of meta data characters. The  plan is for 28th of August 2017.

Revocation:
We're processing revocations of the 11 valid (not expired or revoked) certificates with meta data in fields other than the OU field and will notify you when this is complete. We're not planning to revoke certificates with meta data in the OU field.

Comment 10

[douglas.beattie]

Attachment: Fingerprint file

This file contains the list of fingerprints for the certificates referenced in this bug.  The certificates have been posted to CT logs.

Comment 11

[Ryan Sleevi]

Thanks for responding to these questions, Doug.

I think it's a very positive sign that GlobalSign already had deployed existing controls to detect and prevent such issuance, and was already working towards pre-issuance checks.

With respect to this issue, I believe the remediation plan outlined demonstrates a sufficient understanding and analysis of the problem, an appropriate set of steps to implement, and an appropriate level of disclosure. As there is one outstanding matter to deploy, I'm flagging this NeedsInfo so the system will remind you to provide an update regarding the 2017-08-28 deployment. Provided that deployment is successful, we can close this out as Resolved/Fixed.

Comment 12

[douglas.beattie]

The fix went into place earlier today and all 11 (actually the final count was 10) of the certificates have been revoked. It seems the final count of certificates and the file with the fingerprints don't align, so we'll provide a final update on both tomorrow to hopefully close this out.

Comment 13

[douglas.beattie]

Attachment: Fingerprint file with 2203 unique certificates

This is the final list of all 2203 certificates involved in this issue and all are posted to CT logs.

Comment 14

[Ryan Sleevi]

Marking as Resolved/Fixed, per Comment #11 and Comment #12