1639802 - DigiCert: Failure to revoke key-compromised certificate

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[mpalmer]

Steps to reproduce:

At 2020-05-08 13:57:08 UTC, a certificate problem report was sent to us-smtp-inbound-1.mimecast.com (205.139.110.181) on behalf of [e-mail], stating that a private key with SPKI 6da7c7d14adafc75d98d61021f0a606fd367050243e8fe2bf52b753362f86413 had been compromised, and requesting revocation of all certificates issued by Digicert using the specified SPKI. The URL of a CSR attesting to the compromise of the private key, signed by the compromised private key, was provided.

Actual results:

Certificates https://crt.sh/?id=2386157426 and https://crt.sh/?id=2399185578 were revoked within 24 hours of receiving the certificate problem report, based on validly signed OCSP responses. However, as of the time of submission of this bug, validly signed OCSP responses for certificate https://crt.sh/?id=2782826035 are showing the certificate as being valid.

Expected results:

All certificates to have been revoked within 24 hours of the problem report being received.

Comment 1

[Jeremy Rowley]

This is the same issue as this:
https://bugzilla.mozilla.org/show_bug.cgi?id=1624527

Just with a different key. Again, we searched, found the keys, sent out the revocation notice and before revocation but after searching, a new cert was issued. Our expected timeframe for remediation and having something that blocks key reuse between when the search ends and when we revoke is still June.

Comment 2

[Jeremy Rowley]

Could we combine this one with the other bug? I can update the incident report to include this one as well.

Comment 3

[mpalmer]

I'm ambivalent about whether or not to combine the two incidents. However, I'd like to highlight this from the previous incident report:

the short term solution is to scan the database both before and after we revoke the reported cert.

That was written more than a month before this incident occurred. I would have expected that it would take less than a month for a "short term solution" such as this to be deployed.

Comment 4

[Jeremy Rowley]

Hey Ben - Can we close this as duplicate of https://bugzilla.mozilla.org/show_bug.cgi?id=1624527?

Comment 5

[Ryan Sleevi]

I'm not Ben, but I don't think this should be combined, for the reason Matt highlights in Comment #3. There's a distinct failure mode here that short term mitigations were supposed to catch, but didn't.

Comment 6

[Jeremy Rowley]

The short term fix is/was super manual, and relies on people running a scan every time. There isn't a good reason for the miss other than people forgot to include some certs in the rescan. There's a couple dozen every other day and the rescan isn't part of any automation. It relies solely on the support staff remembering to do this for every cert after revocation occurs.

Comment 7

[Ben Wilson]

I am going to hold off on combining these for now.

Comment 8

[mpalmer]

It relies solely on the support staff remembering to do this for every cert after revocation occurs.

That... doesn't seem like a good procedure. Humans are generally not great at doing boring, repetitive work, which is why we have computers.

Comment 9

[Jeremy Rowley]

Yeah - hence the emphasis on a short term/stop gap solution. It's only there until we get the key blacklister in place.

Comment 10

[mpalmer]

It wasn't felt that it was necessary to have any sort of management oversight or control, QA, or anything like that on the process, to ensure it was actually being carried out?

Comment 11

[Jeremy Rowley]

Tracking June 30 for the blacklist key checker being roled out for DigiCert. Quovadis and non-TLS coming after.

Comment 12

[Jeremy Rowley]

See the other bug. I think this one can be closed.

Comment 13

[Ryan Sleevi]

Ben: While Bug 1624527 tracks the automated responder, I think we can mark this as Resolved/Fixed (not Resolved/Duplicate). DigiCert implemented short-term mitigations that failed/were inadequate, but followed up with longer-term mitigations. I think of this as distinct, and something more to keep in mind in future short-term mitigations, but otherwise seems resolved.