1850091 - GlobalSign: EV TLS certificate with only metadata in JOI State field

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Christophe Bonjean]

Steps to reproduce:

GlobalSign issued one EV TLS certificate with only metadata in the jurisdictionStateOrProvinceName field: https://crt.sh/?id=10154188701.

We are in the process of working with the customer to replace the certificate, and it will be revoked by 29th of August, 09:00 UTC at the latest.

Investigation has started on the root cause for the issue and in the meantime we're monitoring new requests affected by this issue.

We will provide a detailed incident report as soon as we have concluded our analysis, but no later than Tuesday 29th of August.

Comment 1

[Eva Van Steenberge]

1. How your CA first became aware of the problem (e.g., via a problem report submitted to your Problem Reporting Mechanism, a discussion in the MDSP or CCADB public mailing list, a Bugzilla bug, or internal self-audit), and the time and date.

We became aware following the a report to our [e-mail] email address, which was received at 24/08/2023 09:01 UTC (All times in this report are in UTC) . The ticket created a corresponding ticket in our internal SOC, which escalated to the Compliance team, who started the investigation at 09:06.

2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a requirement became applicable, a document changed, a bug was introduced, or an audit was performed.

DD/MM/YYYY - Times in UTC	Description
16/08/2023 - 03:52	Issuance of the reported certificate.
24/08/2023 - 09:01	Received report
24/08/2023 - 09:05	Escalation of the report to compliance team.
24/08/2023 - 09:06	Investigation started by compliance team. Confirmed the certificate in report required revocation. Start root-cause analysis and review of outstanding requests.
24/08/2023 - 09:15	Revocation and replacement process initiated.
24/08/2023 - 09:17	Vetting notified of issue.
24/08/2023 - 09:43	Started review historically issued certificates
24/08/2023 - 09:54	Completed review of currently outstanding requests and found no pending requests affected by this issue. Set up of monitoring reports to review incoming requests.
24/08/2023 - 10:12	Responded to reporter.
24/08/2023 - 10:43	Completed review of historically issued valid certificates. No new additional certificates identified.
24/08/2023 - 14:47	Deployment of extended post-issuance linter to check for the presence of at least one letter or number from the unicode top level categories.
29/08/2023 - 07:15	Revocation of the affected certificate

3. Whether your CA has stopped, or has not yet stopped, certificate issuance or the process giving rise to the problem or incident. A statement that you have stopped will be considered a pledge to the community; a statement that you have not stopped requires an explanation.

Vetting management was put on high-alert on 24/08/2023 - 09:17 with monitoring reports for outstanding requests set-up and shared with Compliance for review on 24/08/2023 - 09:54. We confirmed there were no additional historically issued certificates or outstanding requests affected by this issue (confirmed on 24/08/2023 at 10:43). These measures will remain in place until all remedial actions have been completed.

4. In a case involving certificates, a summary of the problematic certificates. For each problem: the number of certificates, and the date the first and last certificates with that problem were issued. In other incidents that do not involve enumerating the affected certificates (e.g., OCSP failures, delayed responses, etc.), please provide other similar statistics, aggregates, and a summary for each type of problem identified. This will help measure the severity of each problem.

https://crt.sh/?sha256=FB9E4F6A0029B94B441683EF3DC3F83112C7820FC2F85AB390664BCF36D7B4B9

5. In a case involving TLS server certificates, the complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem. It is also recommended that you use this form in your list "[3]https://crt.sh/?sha256=[sha256-hash]", unless circumstances dictate otherwise. When the incident being reported involves an SMIME certificate, if disclosure of personally identifiable information in the certificate may be contrary to applicable law, please provide at least the certificate serial number and SHA256 hash of the certificate. In other cases not involving a review of affected certificates, please provide other similar, relevant specifics, if any.

See #4

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

The character is a unicode "Thai Character Phinthu" symbol (U+0E3A). The character is displayed as a quarter sized dot "." at regular font size (at 4K it is shown as a single pixel), and it is displayed slightly below text baseline. This caused the vetting operator to oversee this particular value.

The certificate linted successfully. Upon investigation, we identified a limitation in the built-in lint for metadata (https://github.com/zmap/zlint/blob/59d4dd332041087118bbbe86f7c5870c8bad980d/v3/lints/cabf_br/lint_subject_contains_noninformational_value.go#L31). To verify that a certificate subject field does not contain only metadata, it checks for the presence of at least one non-metadata character. The logic for this is "at least one alphanumeric character or a UTF8 rune outside of ascii table". However, UTF8 unicode characters consist of multiple categories, including punctuations and other characters that would also classify as metadata. The lint therefore accepted the "Phinthu" symbol as a rune outside the ascii table.

Our certificate processing flows apply common practices for sanitizing certificate field input, but this currently does not include filtering or highlighting metadata-only fields based on UTF8 runes. Consequently, the symbol was included in the JOI State field and sent for issuance.

7. List of steps your CA is taking to resolve the situation and ensure that such a situation or incident will not be repeated in the future. The steps should include the action(s) for resolving the issue, the status of each action, and the date each action will be completed.

The built-in lint has been extended to check for the presence of at least one letter or number from the unicode top level categories. This has been deployed on 24/08/2023 at 14:47 as part of our post-issuance linter, and will be deployed after additional testing to the pre-issuance linting process latest by 22/09/2023. The improved lint will be shared with the community for the benefit of the ecosystem.

Comment 2

[Eva Van Steenberge]

There were no scheduled deliverable this week. We are on track to deliver the other actions as per the schedule. We continue to monitor new requests in order to help ensure no wrong values are included in new certificates.

Comment 3

[Christophe Bonjean]

There were no scheduled deliverables this week. The pre-issuance lint will be deployed latest by 29/09/2023, one week later than initially planned, due to a shift in the release schedule. We continue to monitor new requests in order to help ensure no wrong values are included in new certificates.

Comment 4

[Eva Van Steenberge]

On 26/10/2023 at 09:20 UTC we have deployed the pre-issuance custom lint to check for the presence of at least one letter or number from the unicode top level categories.

This concludes the identified remedial activities - unless there are any further questions we believe this issue can be closed.