Closed Bug 1852404 Opened 2 years ago Closed 2 years ago

CommScope: Empty SCT extensions in certificates

Categories

(CA Program :: CA Certificate Compliance, task)

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: nicol.so, Assigned: nicol.so)

Details

(Whiteboard: [ca-compliance] [dv-misissuance])

Attachments

(6 files)

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36

Actual results:

CommScope received an certificate problem report informing us that certificates issued by several CAs we operate contained empty SCT extensions, which is not conformant to RFC 6962.

Expected results:

If no SCTs are included in a certificate, an SCT extension should not be present in the certificate.

1. How your CA first became aware of the problem (e.g., via a problem report submitted to your Problem Reporting Mechanism, a discussion in the MDSP or CCADB public mailing list, a Bugzilla bug, or internal self-audit), and the time and date.
The issue first came to our attention via an email to our address for incident reporting at 2023-09-05 03:39 (UTC-7). The issue is related to empty SCT extensions in certificates issued by our CAs.
2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a requirement became applicable, a document changed, a bug was introduced, or an audit was performed.
2023-09-05 03:39 (UTC-7) Timestamp of problem report received from external reporter via email
2023-09-05 09:24 (UTC-7) Meeting held to assess the problem report; determination made that the issue did not fall within the criteria for 24-hour revocation in CABF BRs § 4.9.1.1; preliminary action plan decided on; determined that the CA application needed an update
2023-09-05 11:35 (UTC-7) CA application updated and deployed to QA environment; QA started regression test on QA environment
2023-09-05 13:11 (UTC-7) Sample certificate from updated CA application validated using linter and by inspection
2023-09-05 13:30 (UTC-7) CA application deployed to staging system; QA started regression test on staging system
2023-09-06 09:00 (UTC-7) Recognition that a revision to the CP/CPS would need to be part of the action plan; the revision is to limit the scope of requirement to publish precertificates to transparency logs prior to certificate issuance; CP/CPS revision added to action plan; parallel task to revise CP/CPS initiated
2023-09-07 (UTC-7) Discussion on CP/CPS revision in progress
2023-09-08 13:35 (UTC-7) CA application updated on production and DR system
2023-09-08 14:21 (UTC-7) Updated CP/CPS published
2023-09-08 14:47 (UTC-7) Approximate start time that replacements for to-be-revoked problematic certificates issued
2023-09-08 15:16 (UTC-7) Problematic certificates revoked
2023-09-08 15:57 (UTC-7) Replacement certificates for test websites reported as installed
3. Whether your CA has stopped, or has not yet stopped, certificate issuance or the process giving rise to the problem or incident. A statement that you have stopped will be considered a pledge to the community; a statement that you have not stopped requires an explanation.
We have stopped issuance of certificates with the non-conformance issue. We have since corrected the underlying technical issue that had led to the issuance of the non-conformant certificates.
4. In a case involving certificates, a summary of the problematic certificates. For each problem: the number of certificates, and the date the first and last certificates with that problem were issued. In other incidents that do not involve enumerating the affected certificates (e.g., OCSP failures, delayed responses, etc.), please provide other similar statistics, aggregates, and a summary for each type of problem identified. This will help measure the severity of each problem.
Including certificates that had expired or been revoked at the point when we became aware of the issue, a total of 30 certificates with the problem have been issued. The first of the certificate was issued on 2022-07-25, the last 2023-08-14.
5. In a case involving TLS server certificates, the complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem. It is also recommended that you use this form in your list “https://crt.sh/?sha256=[sha256-hash]”, unless circumstances dictate otherwise. When the incident being reported involves an SMIME certificate, if disclosure of personally identifiable information in the certificate may be contrary to applicable law, please provide at least the certificate serial number and SHA256 hash of the certificate. In other cases not involving a review of affected certificates, please provide other similar, relevant specifics, if any.
The problematic certificates were not published to CT logs, so it is not possible to reference them using a crt.sh URL with a SHA-256 hash. The common issue with the problematic certificates is that they each contain an empty SCT extension. When parsed using the OpenSSL CLI tool, the non-conformant SCT extension has a structure that looks like the example below:

  823:d=4  hl=2 l=  18 cons: SEQUENCE
  825:d=5  hl=2 l=  10 prim: OBJECT            :CT Precertificate SCTs
  837:d=5  hl=2 l=   4 prim: OCTET STRING      [HEX DUMP]:04020000

The problematic certificates referenced by the external reporter are attached. Also attached are output files from parsing one representative certificate (parsed as an X.509 certificate in one, parsed as an ASN.1 structure in another.)
6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.
The issue was caused by an incorrect implementation of the requirements of RFC 6962 regarding the syntax of the SCT extension.
A contributing factor for the problem not having been detected before is the fact that the structure of the extnValue component of the SCT extension is defined outside RFC 5280 and not in ASN.1 syntax. The issue is not detected by zlint, nor the CA/Browser Forum lint test used by CCADB.
In the bigger context, CommScope is a CA operator seeking to enter the public CA space. In our effort to gain inclusion in major trusted root CA programs (some of which have CT requirements), we had configured our public CAs to include SCTs from two test CT logs in the certificates issued, to show that they would be able to comply with CT requirements. We used test CT logs because no production CT logs would accept our root certificates before they are included in at least one trusted root CA program. At some points, the CT logs we used stopped being available to us. We decided to suspend including SCTs in certificates until we are included in at least one trusted root CA program. It was only at this point that the incorrect implementation resulted in a non-conformant SCT extension.
7. List of steps your CA is taking to resolve the situation and ensure that such a situation or incident will not be repeated in the future. The steps should include the action(s) for resolving the issue, the status of each action, and the date each action will be completed.
We have modified our software to not generate certificates with empty SCT extensions. Certificates that include no SCTs now contain no SCT extension instead of a SCT extension containing no SCT. The software update was completed on 2023-09-08.

Assignee: nobody → nicol.so
Status: UNCONFIRMED → ASSIGNED
Type: defect → task
Ever confirmed: true
Whiteboard: [ca-compliance] [dv-misissuance]

At this point, the case has been open for 10 days. There have been no follow-up questions. There are no outstanding remediation action items. CommScope would like to request that the issue be treated as resolved and closed.

Flags: needinfo?(bwilson)

I anticipate closing this on Friday, 22-Sept-2023, unless there are comments or concerns that are raised.

Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Flags: needinfo?(bwilson)
Resolution: --- → FIXED

This is a supplement and amendment to our incident report in comment #7.

In Bug 1904402, Andrew Ayer identified 6 additional certificates with empty SCT extensions that we missed when we prepared comment #7. Helpfully, he also provided links to crt.sh for the set of 36 affected certificates. The below is a list of the affected certificates copied from Bug 1904402, separated into a group of 30 that we previously identified but not explicitly named and a group 6 that were newly identified. The 6 certificates in the latter were all issued on 2021-08-17, which is also the date on which the first of affected certificates was issued.

Certificates that were previous identified but not explicitly named:

https://crt.sh/?sha256=05c6ab8929e2227d78ebb91fc51f521308b89d6f7d696105dffcc53f3c7d149c
https://crt.sh/?sha256=0710ad73a6804085dafb7dc8e43b19f24254aee0bbd50a86472c6a2cd739e3e7
https://crt.sh/?sha256=0bcff18a9c9cd429b83caf737a639a28808039ee85daa463b4ab86926ea8647f
https://crt.sh/?sha256=0e408f87b5cfdc1f1cfc95a0a6271cee577bbf52bc3466d975557aaeab517109
https://crt.sh/?sha256=147438d30201df1759f24453d0a4f74004f78159eb92a2f42df3c33a732810e3
https://crt.sh/?sha256=14d1fc5c43b6ab71e62cc4d232033a0b2db3d2833ab5ec7aaa66aea22f5bd5a4
https://crt.sh/?sha256=23b3688eb98adad99d56e311c677e301c045c8f7ddb2ec6a85ad178102f230b7
https://crt.sh/?sha256=259a2f394d1514af5831bd0993767c73c1ec9bf822ee445fe2668b7a17fb0632
https://crt.sh/?sha256=28c4f324a54915a0ef0fda3707e40e855b14d53acdfa726a515000b8316d4793
https://crt.sh/?sha256=3a8a4ff7f0b6a4bf2d2904510110771fb0767fba8e2087ebe8cb222e1e0675a0
https://crt.sh/?sha256=3c9f6645daab08f7b2ce8abf4382aae2196d5f16637f36a46807515837da5437
https://crt.sh/?sha256=40781a09b834080b72cac38b372c6715cc7e1aa9b00712a3fa9840d618469205
https://crt.sh/?sha256=4f0b747c8a452c78a4f4c21e2cfb5af9ba94106a861a09c50bb7ca47be76d646
https://crt.sh/?sha256=59a51c29f40b04e8c717d3bbeb48020f23e53d46e9060d8b7aab40d3ad5ebb06
https://crt.sh/?sha256=5a6fa588225155f83d9ca382265d0f1e50df34a33a66b67e3ebfb2d7707eefb2
https://crt.sh/?sha256=5b7744444dac0f04f71be489d4dd31f2fe3296b6b380068511f5b3a4f60bf2d7
https://crt.sh/?sha256=6272cb47728081b48ac64c8da659c95ab9acd8b6a243909f8d82f15a6ab9f9fc
https://crt.sh/?sha256=7f8b732e783fc7e32e5eb7e639f9f1437bd1bbda7276458f72b7c41b06581e5e
https://crt.sh/?sha256=83090185395455675cd133ce408f87fe348dd74b1f93b5ba85788abb92199964
https://crt.sh/?sha256=87c47f512a80e7c2dcf8f004c09dc06b1d259daec6bd6477ada0b16f63bcfcfb
https://crt.sh/?sha256=ad1f7daa8dbf5cd8a30cdaf66c3307c41deea91081f45c6ce7bc624b8dacd5b9
https://crt.sh/?sha256=b00a2af08c3ed09ba639c3d5765517838af11fecc27b2ffb93126b37754b67af
https://crt.sh/?sha256=c3a63c74d6c209958048f46f22960c3ea8678817c4c41b0c85db4b9f5ff7378c
https://crt.sh/?sha256=d083da050cbf9d5556fe64b4d45421b4bde503b25b478847682ffc749657a574
https://crt.sh/?sha256=d2e6b7cc797a175e5c59d64d561d4ebe56d311d5582259e0a5154b50d8e00f0c
https://crt.sh/?sha256=ddf00d12805f1bbc36f84365fb8df58c63df8c71eed1b4da173d19cf55173003
https://crt.sh/?sha256=e44e5d34a68471655e3c65f382aca983b8c32e7110306ef0e7ed25501b39272b
https://crt.sh/?sha256=e5825957944bf85d35562a6b042faee7dbda55a69ed18c6f442474b6257016b3
https://crt.sh/?sha256=ed5df8d326dd6e61cf03ff99809cdd93bbdeb51380492daa44bea07f20e3ae8e
https://crt.sh/?sha256=fda73bcc2d06a2fc5d52f577d420b8459782816753e2b4070cc5cb959f550251

Additional affected certificates first reported in Bug 1904402 by Andrew Ayer:

https://crt.sh/?sha256=3664d3f9ad5aebb6fe9b3bc73b5b65a950a9908b325f0e4fc4c0fa34741ff178
https://crt.sh/?sha256=b0f18184d69790b3da328eec3d47c4611e04a2c8854b2566457b68719c58ab4d
https://crt.sh/?sha256=b73a749b14e45cd618358dec02c21cb4622467acdf6daf8ddd7aac93047dce0b
https://crt.sh/?sha256=7958567506ca778c11a6d553db50db7d9e89d8f3dc96b52322e2f564f9354fbf
https://crt.sh/?sha256=aadaa66b0fde5863bd8ac8043da39e14c1f8f3adf2fed561d2a00e3172492ba4
https://crt.sh/?sha256=38085f60623b3593e9c8dbc27fb8f2cb20b259295863948f17b8a56ea88af00f

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: