Closed Bug 1756847 Opened 3 years ago Closed 3 years ago

Sectigo: SC45 DCV Reuse Error

Categories

(CA Program :: CA Certificate Compliance, task)

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: tim.callan, Assigned: martijn.katerbarg)

Details

(Whiteboard: [ca-compliance] [ov-misissuance])

Through continuous QA testing of our systems, we recently became aware of occurrences of incorrect DCV reuse resulting from a bug in the new code we implemented for SC45 compliance with specific regard to BR 3.2.2.4.18:

For Certificates issued on or after 2021‐12‐01, the CA MUST NOT issue Certificates for other FQDNs that end with all the labels of the validated FQDN unless the CA performs a separate validation for that FQDN using an authorized method.

We have deployed a code fix. At present we have identified 40 affected certificates and revoked them within 24 hours of identification. We are continuing that investigation to ensure we’ve discovered all affected certificates.

We are working on a full writeup and will follow up shortly.

Assignee: bwilson → martijn.katerbarg
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Whiteboard: [ca-compliance]

1. How your CA first became aware of the problem

Due to continuous QA testing on our systems, we discovered a bug that could lead to the issuance of certificates relying on DCV methods not compliant with SC45 via a DCV reuse mechanism we call Sticky DCV. Sticky DCV, which we mentioned previously in bug 1718771 comment 2, allows the reuse of DCV information for the same Subscriber for 365 days. When done correctly this is perfectly appropriate. Due to this bug, however, it was possible for our Sectigo Certificate Manager (SCM) platform to create a Sticky DCV record that did not fully enforce the new restrictions on HTTP DCV checks that were introduced by SC45.

2. Timeline

June 23, 2021 - 15:10 UTC - A ticket is created to start and track updates to become compliant with the recently passed SC45 CA/B Forum ballot.
November 22, 2021 - 09:30 UTC - We deploy and activate our updates for SC45 compliance.
February 1, 2022 - 8:54 UTC - Continuous testing reveals a possible bug in our SC45 implementation. We create a ticket.
February 2, 2022 - 21:43 UTC - We deploy a hotfix to resolve the bug.
February 3, 2022 - 11:34 UTC - A script is run to identify all the problematic Sticky DCV entries in our system.
February 3, 2022 - 11:47 UTC - We analyze the results and identify 404 problematic Sticky DCV records that could have potentially led to misissuance.
February 3, 2022 - 12:30 UTC - Those 404 Sticky DCV records are deleted, stopping any possibility of further misissuance.
February 4, 2022 - 13:22 UTC - Post-Deployment testing starts on the hotfix code.
February 7, 2022 - 20:58 UTC - Post-Deployment testing complete.
February 8, 2022 - 16:58 UTC – We generate reports of possible misissued certificates. A review of these reports starts to eliminate false positives and confirm the results.
February 9, 2022 - 14:20 UTC - We complete the review and identify 40 certificates confirmed to have been misissued.
February 10, 2022 - 14:14 UTC - All 40 certificates are revoked.

3. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem.

We have deployed a fix and stopped issuing certificates with this problem.

4. Summary of the problematic certificates

40 certificates issued between 2021-12-01 and 2021-12-29.

5. Affected certificates

Serial Number Certificate Precertificate
1C093D23CC6F672151DBC0BD457A51B4 Certificate Precertificate
47866A80C892ACC13D1AF72159C23D6D Certificate Precertificate
262ED9105F1597319D2F80C35AE84EF7 Certificate Precertificate
00A84137DE1D6CA3943E6321C409B9749E Certificate Precertificate
52069372568F3072B83603DF78C083A5 Certificate Precertificate
00DB144BEBA39F4967497C228373CB2AE3 Certificate Precertificate
2F725A445981758F90228880EBD9DA83 Certificate Precertificate
150DEB7EC8B12EDD6DD890DB6390EC92 Certificate Precertificate
43C282762FFD21FDD2E622A3CE1D7A8D Certificate Precertificate
645E2EB7FE12D9E1F87128E575485B92 Certificate Precertificate
00C4AE0E6F14A162EA2E959F286FD15AF4 Certificate Precertificate
4BF8B2B98750DB275CF36D8D867EAB2B Certificate Precertificate
00D2BBAE37CEE1DE597C8BC2EE190326FE Certificate Precertificate
00F6003E5C90393A6904E3F977BB0D6342 Certificate Precertificate
00B2E06A75A1732E101ACE9742DE5327A0 Certificate Precertificate
00AB8E6B2F75BD5C46735C91718888853F Certificate Precertificate
5B17B549558712995E65D230C835D553 Certificate Precertificate
008D613CC072E4B82BD702A85A60DDA602 Certificate Precertificate
5365A393651A0FA229844718C5050D02 Certificate Precertificate
2BA8D6815CAF243EB2FB1028054870D4 Certificate Precertificate
4F6E604CC5DAA6FCE7D144F44B702288 Certificate Precertificate
00E6F60A584C12C4577F6E61E4ED0C2F73 Certificate Precertificate
483F50DA6D4A3A45958BA0BE83208A9F Certificate Precertificate
00D108B7561AEC7C20F2FD4F288E07B838 Certificate Precertificate
77DA002E521AE0C7BF8DBBA815231344 Certificate Precertificate
71B6C9FD9838108E800BDBE9FE9D9093 Certificate Precertificate
1C2AE9C202B83D3EF2FC9C67017AF49E Certificate Precertificate
00B1DBC34E7DACC67DD429EA95E55BB279 Certificate Precertificate
00FF5B1A6A42CE30BFC9BD5640E23FC4D8 Certificate Precertificate
009644B2F918A8E744269F0A0CE80CDC6B Certificate Precertificate
5FAB220A74E91F716EF0FE4D0EC69175 Certificate Precertificate
00961370220ED1EDEDA0B311C821165A0A Certificate Precertificate
3A12D8A2E1AFA35055E119F28A845CE4 Certificate Precertificate
2B064FFDE07705019C319276FD3F6AA1 Certificate Precertificate
66890479A9ACF34D109F4D38F7589350 Certificate Precertificate
00C0EACCBEB9E9C9EAE65A5E37E9D395F9 Certificate Precertificate
00922ADEA578607CE09803176F2646568B Certificate Precertificate
00BBEB8A3E7867EDA59198CD117A92CC07 Certificate Precertificate
2722E45F3406B335B0554722A0511393 Certificate Precertificate
00975A13F2D4943EDFDD66E9179D3F7B3F Certificate Precertificate

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now

In June 2021 ballot SC45 passed within the CA/B Forum. We started investigating the impact on our systems and the required updates. We identified the requirement to disallow the “Agreed-upon change to website v2” Domain Control Validation method for wildcard SAN values as well as to stop treating as valid subordinate FQDNs of a previously or newly validated FQDN with this method. In November 2021 we deployed the changes required for SC45. “Sticky DCV” flags orders for a specific domain from a specific source for reuse for the limited time of 365 days.
Sectigo Certificate Manager (SCM) includes a feature that allows the creation of Sticky DCV records. This feature, using an API to our backend that manages the Sticky DCV records, incorrectly allowed the creation of an automatically approved Sticky DCV record when the Subscriber’s account already had an approved Sticky DCV record for a parent FQDN for which “Agreed-upon change to website v2” was the DCV method. Prior to SC45 this behavior would have been compliant, but since SC45 it is no longer acceptable. This bug was newly introduced to our codebase in November 2021 when we implemented the changes required for SC45 compliance.
Since SCM is the only application to use this specific Sticky DCV backend API, only certificates requested via that system were affected.
At the highest level, this error owes itself to process complexity. SC45 imposed varying requirements based on the nature of the DCV performed and the set of domains for which it occurred. In a specific set of circumstances our code failed to capture that logic correctly.

7. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future

This specific error is eliminated.
This was a software bug that occurred where our business logic of necessity was complex. This possibility is the reason why we conduct continuous QA testing on our systems, which in this case limited the number of affected certificates and the amount of time this bug affected issuance.

We believe we have included all necessary information in comment 1. Remediation has been completed. We are monitoring this bug for any questions and/or comments.

Ben, since there appear to be no questions or comments, I’d like to request closure of this bug.

Flags: needinfo?(bwilson)

I will take a look at closing this on Wed. 23-Mar-2022, unless there are objections.

Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Flags: needinfo?(bwilson)
Resolution: --- → FIXED
Product: NSS → CA Program
Whiteboard: [ca-compliance] → [ca-compliance] [ov-misissuance]
You need to log in before you can comment on or make changes to this bug.