Open Bug 1910512 Opened 2 months ago Updated 3 days ago

CommScope: Certificates not logged in CT logs as stated in CP/CPS

Categories

(CA Program :: CA Certificate Compliance, task)

Tracking

(Not tracked)

ASSIGNED

People

(Reporter: nicol.so, Assigned: nicol.so)

Details

(Whiteboard: [ca-compliance] [policy-failure])

This is a preliminary incident report. The full report will be published on or before 2024-08-09.

Incident Report (Preliminary)

Summary

Certificates were issued without having the corresponding precertificates published to the requisite number of certificate transparency (CT) logs as stated in the CP/CPS.

Impact

Certificates were issued not according to the practices stated in our CP/CPS.

Timeline

All times are UTC:

2021-04-26:

  • Revision date of CommScope's combined CP/CPS revision 2.0, in which requirements about publication to CT logs first appeared.

2021-06-04:

  • 18:13 First certificate issued by a CommScope public CA. The certificate contains only 1 SCT (signed certificate timestamp). Altogether, 12 certificates containing 1 SCT each were issued by CommScope's public CA within the hour.

2021-08-17:

  • 20:08 Six certificates were issued by CommScope's public CAs containing no SCT. The certificates were issued within a span of a few minutes.

2021-11-16:

  • 19:07 Two certificates containing only 1 SCT were issued by CommScope public CAs. Between this time and 2022-02-03, 10 certificates containing only 1 SCT were issued in total.

2022-07-25:

  • 20:36 Eight certificates containing no SCT were issued by CommScope public CAs, within an hour. Between this time and 2023-11-14, 48 certificates containing no SCT were issued in total.

2023-11-16:

  • CommScope's public root CA certificates first appeared in Mozilla's NSS 3.95.

2024-02-27:

  • 21:28 First certificate issuance by CommScope's public CAs since inclusion in Mozilla's root program. The certificate (and others issued since) have 2 SCTs from production CT logs.

2024-06-10:

  • 20:34 User "Wayne" asked in Bug 1901578, comment 9 about the number of certificates issued by CommScope's public CAs that were not logged in CT logs. This prompted the start of a comprehensive review all issued certificates by our public CAs.

2024-07-26:

  • 17:04 We received a question from our external auditor about 2 samples certificates, which according to our CP/CPS should have precertificates published to 3 CT logs.
  • 20:42 We identified 4 certificates issued during the audit period that fitted the auditor's description. All 4 certificates have previously been identified in Bug 1901578 as not having been logged in CT logs. In all, 76 certificates (all expired or revoked now) have been identified as not having been published to the requisite number of CT logs.

Root Cause Analysis

(To be provided in the final report)

Lessons Learned

(To be provided in the final report)

What went well

(To be provided in the final report)

What didn't go well

(To be provided in the final report)

Where we got lucky

(To be provided in the final report)

Action Items

Action Item Kind Due Date
Publish final report Disclose 2024-08-09

Appendix

Details of affected certificates

Assignee: nobody → nicol.so
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Whiteboard: [ca-compliance] [policy-failure]

Incident Report

Summary

Certificates were issued without having the corresponding precertificates published to the requisite number of certificate transparency (CT) logs as stated in the CP/CPS.

Impact

Certificates were issued not according to the practices stated in our CP/CPS.

Timeline

All times are UTC:

2021-04-26:

  • Revision date of CommScope's combined CP/CPS revision 2.0, in which requirements about publication to CT logs first appeared.

2021-06-04:

  • 18:13 First certificate issued by a CommScope public CA. The certificate contains only 1 SCT (signed certificate timestamp). Altogether, 12 certificates containing 1 SCT each were issued by CommScope's public CA within the hour.

2021-08-17:

  • 20:08 Six certificates were issued by CommScope's public CAs containing no SCT. The certificates were issued within a span of a few minutes.

2021-11-16

  • 19:07 Two certificate containing only 1 SCT were issued by CommScope public CAs. Between this time and 2022-02-03, 10 certificates containing only 1 SCT were issued in total.

2022-07-25:

  • 20:36 Eight certificates containing no SCT were issued by CommScope public CAs, within an hour. Between this time and 2023-11-14, 48 certificates containing no SCT were issued in total.

2023-11-16:

  • CommScope's public root CA certificates first appeared in Mozilla's NSS 3.95.

2024-02-27:

  • 21:28 CommScope's public CAs started issuing certificates with 2 SCTs from production CT logs.

2024-06-10:

  • 20:34 User "Wayne" asked in Bug 1901578, comment 9 how many (more) certificates issued by CommScope's public CAs were not logged in CT logs. This prompted the start of a process to review all issued certificates by our public CAs.

2024-07-26:

  • 17:04 We received a question from our external auditor about 2 samples certificates, which according to our CP/CPS should have precertificates published to 3 CT logs.

Root Cause Analysis

CommScope wrote its CP/CPS with the intent to comply with the requirements of the several major trusted root CA programs ("root programs").
Consistent with that goal, CommScope's CP/CPS specifies that before a certificate is issued by one of its public CAs, corresponding precertificates are to be published in public certificate transparency (CT) logs.

Before CommScope's public CAs were accepted in at least one of the major root programs, no operators of CT logs intended for production use would accept precertificate publication from CommScope. For an initial period, a public CT log intended for testing was used for the purpose. At some point, the CT log became unavailable to CommScope. CommScope disabled publishing to CT logs in response to the difficulty.

In face of the difficulties in getting our precertificates accepted by CT logs intended for production, the correct action would have been to amend our CP/CPS to make publication to CT logs conditional on being required by a root program in which CommScope's public CAs have been accepted. The non-compliance issue in this case was caused by a process deficiency, in which a change in system configuration in response to operational difficulties did not trigger a review of its implications in terms of compliance.

Lessons Learned

Before a configuration or change to configuration to the CA system is adopted, if the change is not directed by approved requirements or policy, the proposed change must be reviewed for its compliance implications.

What went well

  • CommScope was able to start publishing to the prescribed number of CT logs after the inclusion of its root CAs in Mozilla's root program.

What didn't go well

  • The implication of a non-compliant configuration was not appreciated and corrected in a timely manner.

Where we got lucky

  • There was no significant impact to subscribers.

Action Items

Action Item Kind Due Date
Modify process documentation so that any proposed CA system configuration changes in response to operational difficulties (rather than directed by approved requirements or policy) will be reviewed for compliance with the CP/CPS. If the proposed CA system configuration changes are not in compliance with the CP/CPS, they must be disallowed unless and until the CP/CPS is updated accordingly. Prevent 2024-08-30

Appendix

Details of affected certificates

https://crt.sh/?sha256=7c8ae84e8b8fc8d36f505a196c40d685603a3fdcb52e6c5204791e9cf1d188ba
https://crt.sh/?sha256=e45971da54a296585e27501a9b81fa8625111e23e2d5cf98daab1368bde9652c
https://crt.sh/?sha256=9a34bf83970ba48b27f2f255a707e3dc8d17ac47cc97ac51f631600eaf62920b
https://crt.sh/?sha256=8c52633362a1788df57c620b59d6cdd8bc0706dba17a002a57f4011a78cf3669
https://crt.sh/?sha256=d2bf5f27dc3fd918cb50f78969fba0c83057ab0adfc1480bf9d5fc2d93ac449f
https://crt.sh/?sha256=ce6f476900563a95b41bd3f88abdf1855a04acfda65e2bc7101ec987a5cd80e6
https://crt.sh/?sha256=cc34c046de5e8c18c29748a1a54b89baae30ab3daefc74dcdebb06f7a8484692
https://crt.sh/?sha256=028baf16432c02835958abec0738444f1e72cf4114f2bd03981c9fa7d4f1501a
https://crt.sh/?sha256=2a3e103335bc992222ab9ec9a4e8e2f11e9abfc1beb8447bc26d251543fc79c8
https://crt.sh/?sha256=0efbf4f004daf39d88129d158774bebb0cdc4abd41825bd1da5cd3e0f77d211f
https://crt.sh/?sha256=48a71a51981b9b4e234372a34decd20bc550b456c588006943dac4aba6732830
https://crt.sh/?sha256=f78f5ba5d43c42e88264d806068f7dbf1a787676ce897ae856590c192c6d787f
https://crt.sh/?sha256=7958567506ca778c11a6d553db50db7d9e89d8f3dc96b52322e2f564f9354fbf
https://crt.sh/?sha256=b0f18184d69790b3da328eec3d47c4611e04a2c8854b2566457b68719c58ab4d
https://crt.sh/?sha256=38085f60623b3593e9c8dbc27fb8f2cb20b259295863948f17b8a56ea88af00f
https://crt.sh/?sha256=3664d3f9ad5aebb6fe9b3bc73b5b65a950a9908b325f0e4fc4c0fa34741ff178
https://crt.sh/?sha256=b73a749b14e45cd618358dec02c21cb4622467acdf6daf8ddd7aac93047dce0b
https://crt.sh/?sha256=aadaa66b0fde5863bd8ac8043da39e14c1f8f3adf2fed561d2a00e3172492ba4
https://crt.sh/?sha256=7f94b1762d7bd38cac4f572a4de90a31641e7cecb8caf1bede3d0a13128c0455
https://crt.sh/?sha256=51f5a0b7b40b84194f4633a877e16a314f62e1289515ac02b43144e92d6de7a1
https://crt.sh/?sha256=3f91c41a78442b75c99ce4ede1dd670cbfc1017c62d4934d00695be9b575f7cf
https://crt.sh/?sha256=e3dac932d6e63dde83de27bfd29ba40d0b4e83680bfc92a7e3c84749d7c40520
https://crt.sh/?sha256=9efd646f4e2ce9d17acea906f74e8c077fce358565da72749b5ed3f3ff5b8309
https://crt.sh/?sha256=1f93111477c6071060081b6a5c1a9e2a944065e595083a1ccf3633b1ed2e9e4d
https://crt.sh/?sha256=6d0d40c73aa4bfeda88e1a3c761951aefb5f3808cf56715cc1afcfa866b5cd62
https://crt.sh/?sha256=38c3cde7d553e5a5ecebedceacfc2e8cad3eedffccae77716fecf0e153ca8593
https://crt.sh/?sha256=f7f50b7169985b80d0ff1fbf714c042b8412659e5a15c731fc9e3e275d7b68b3
https://crt.sh/?sha256=cc235ca2a7956b29d0fe8c576b341b826fd92250f7e35871865d884dc45217d8
https://crt.sh/?sha256=0bcff18a9c9cd429b83caf737a639a28808039ee85daa463b4ab86926ea8647f
https://crt.sh/?sha256=87c47f512a80e7c2dcf8f004c09dc06b1d259daec6bd6477ada0b16f63bcfcfb
https://crt.sh/?sha256=b00a2af08c3ed09ba639c3d5765517838af11fecc27b2ffb93126b37754b67af
https://crt.sh/?sha256=7f8b732e783fc7e32e5eb7e639f9f1437bd1bbda7276458f72b7c41b06581e5e
https://crt.sh/?sha256=ddf00d12805f1bbc36f84365fb8df58c63df8c71eed1b4da173d19cf55173003
https://crt.sh/?sha256=05c6ab8929e2227d78ebb91fc51f521308b89d6f7d696105dffcc53f3c7d149c
https://crt.sh/?sha256=23b3688eb98adad99d56e311c677e301c045c8f7ddb2ec6a85ad178102f230b7
https://crt.sh/?sha256=5b7744444dac0f04f71be489d4dd31f2fe3296b6b380068511f5b3a4f60bf2d7
https://crt.sh/?sha256=259a2f394d1514af5831bd0993767c73c1ec9bf822ee445fe2668b7a17fb0632
https://crt.sh/?sha256=0710ad73a6804085dafb7dc8e43b19f24254aee0bbd50a86472c6a2cd739e3e7
https://crt.sh/?sha256=ad1f7daa8dbf5cd8a30cdaf66c3307c41deea91081f45c6ce7bc624b8dacd5b9
https://crt.sh/?sha256=3c9f6645daab08f7b2ce8abf4382aae2196d5f16637f36a46807515837da5437
https://crt.sh/?sha256=fda73bcc2d06a2fc5d52f577d420b8459782816753e2b4070cc5cb959f550251
https://crt.sh/?sha256=147438d30201df1759f24453d0a4f74004f78159eb92a2f42df3c33a732810e3
https://crt.sh/?sha256=28c4f324a54915a0ef0fda3707e40e855b14d53acdfa726a515000b8316d4793
https://crt.sh/?sha256=4f0b747c8a452c78a4f4c21e2cfb5af9ba94106a861a09c50bb7ca47be76d646
https://crt.sh/?sha256=d083da050cbf9d5556fe64b4d45421b4bde503b25b478847682ffc749657a574
https://crt.sh/?sha256=e44e5d34a68471655e3c65f382aca983b8c32e7110306ef0e7ed25501b39272b
https://crt.sh/?sha256=e5825957944bf85d35562a6b042faee7dbda55a69ed18c6f442474b6257016b3
https://crt.sh/?sha256=3a8a4ff7f0b6a4bf2d2904510110771fb0767fba8e2087ebe8cb222e1e0675a0
https://crt.sh/?sha256=83090185395455675cd133ce408f87fe348dd74b1f93b5ba85788abb92199964
https://crt.sh/?sha256=14d1fc5c43b6ab71e62cc4d232033a0b2db3d2833ab5ec7aaa66aea22f5bd5a4
https://crt.sh/?sha256=ed5df8d326dd6e61cf03ff99809cdd93bbdeb51380492daa44bea07f20e3ae8e
https://crt.sh/?sha256=40781a09b834080b72cac38b372c6715cc7e1aa9b00712a3fa9840d618469205
https://crt.sh/?sha256=6272cb47728081b48ac64c8da659c95ab9acd8b6a243909f8d82f15a6ab9f9fc
https://crt.sh/?sha256=0e408f87b5cfdc1f1cfc95a0a6271cee577bbf52bc3466d975557aaeab517109
https://crt.sh/?sha256=59a51c29f40b04e8c717d3bbeb48020f23e53d46e9060d8b7aab40d3ad5ebb06
https://crt.sh/?sha256=c3a63c74d6c209958048f46f22960c3ea8678817c4c41b0c85db4b9f5ff7378c
https://crt.sh/?sha256=5a6fa588225155f83d9ca382265d0f1e50df34a33a66b67e3ebfb2d7707eefb2
https://crt.sh/?sha256=d2e6b7cc797a175e5c59d64d561d4ebe56d311d5582259e0a5154b50d8e00f0c
https://crt.sh/?sha256=7788ceedca28546e3c75956d4dedd39b797e035f993bc99b3d959733d70fa8a5
https://crt.sh/?sha256=88cf33aade82f602422fbc148a86f6f21137fbe5d3a3f4309f3835ae42be735b
https://crt.sh/?sha256=53ac4d70191aad7c15793305fcdcc5b500bcff918658af7a3e48f0038e219011
https://crt.sh/?sha256=09307532ac8f7aa1681eb1ac4560d413b5025e97be3c04e046abb6cda8abeaa8
https://crt.sh/?sha256=43db16923133902d3d59233fe1d7ba23ef7d66b17d9685bd6c6214c0540ce0b5
https://crt.sh/?sha256=e3374a9e42c766be23fa796b2dbede7d58cd00285edf74d29b3a744ec5e2366a
https://crt.sh/?sha256=16b36e2ca23feab1773b60880f679c4b2372154bcf3e0a3a7c98a0a97ee2e86d
https://crt.sh/?sha256=9e0bf5b4b6c8fa4b993d62c404fe5a0332736e2471242bfbdff069dadf24390b
https://crt.sh/?sha256=85cb1e49010151799a74b627a76d29619a436064487e1a2f3cfbdb9b4c5fa513
https://crt.sh/?sha256=e337c885994d1f061a1666cc4dd7f9854e4c32b25031a9992534e6616c1e6817
https://crt.sh/?sha256=795bc793e3e2703290e0910eb499d987a00826d9eb3f4aafe985057fe8cb4c94
https://crt.sh/?sha256=bb00cbc7a770e031ea1666e0a02e67bf4f7239a876cd1e2cd1e6e9cd479f141d
https://crt.sh/?sha256=e25a16cc71549f28f5ba296913d39c076a191c3ca9b948247574c278770c304c
https://crt.sh/?sha256=c8f824df84a28546c5cbb4f8c06934056a452321823676a4a5e1431de9a63acf
https://crt.sh/?sha256=81b26f99eac6ac482d693d41e0d236140b87cbfba3988c93a800bfdffe2ac16b
https://crt.sh/?sha256=9a7a4f45705d331a4ccfd00902313458a4a586129a232495f4d7dd516f4b802f
https://crt.sh/?sha256=8100636c7f6044783390c65e7361bef17001a8d0b0e4a682a7f6ff9c4ecc677d
https://crt.sh/?sha256=c40842fe4a27e84df55df0dda67235f799db7740cf69f779cb49f1d741982cf1

(In reply to Nicol So from comment #1)

Impact

Certificates were issued not according to the practices stated in our CP/CPS.

Can you give a better summary of the affected certificates? How many were affected? When was the first one issued? When was the last certificate issued that had this problem?

(In reply to Mathew Hodson from comment #2)

Can you give a better summary of the affected certificates? How many were affected? When was the first one issued? When was the last certificate issued that had this problem?

Thank you for the comment. Here's a summary of the affected certificates:

  • Seventy-six (76) certificates are affected by the issue.
  • The first of the affected certificates was issued on 2021-06-04, the last of them 2023-11-14.

The action item in comment #1 is still being worked on. We don't have anything noteworthy to report at this time.

Status update: The action item in comment #1 (process documentation revision) is taking longer than expected. It is likely that we won't be able to complete the task by 2024-08-30 as we originally expected. We are continuing to work on the task.

Can CommScope provide insight into why the action item is taking longer than expected, and what an updated target date might be?

(In reply to Aaron Gable from comment #6)

Can CommScope provide insight into why the action item is taking longer than expected, and what an updated target date might be?

We had a revision proposal drafted and circulated for critique. We received differing opinions that we tried to resolve. Our efforts to ensure that all concerns are adequately addressed resulted in additional cycles of revisions and reviews. We expect to complete the action item by 2024-09-23.

CommScope has completed the action item named in comment #1 (process documentation revision). As of now, we do not have any outstanding action items.

You need to log in before you can comment on or make changes to this bug.