1684112 - Let's Encrypt: Failure to audit log subscriber certificate OCSP updates

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Andrew Gabbitas]

User Agent: Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:82.0) Gecko/20100101 Firefox/82.0

Comment 1

[Andrew Gabbitas]

Summary:

Let’s Encrypt logs an audit log event when OCSP is signed upon initial certificate issuance. Subsequent updates to the OCSP response throughout a certificate’s 90 day lifetime are not logged as audit log events as required in the baseline requirements section 5.4.1.2.5. Revocation logs are properly logged as audit logs and not affected by this incident.

Incident Report:

How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.

• During a quarterly internal review of the CA/B Forum baseline requirements, our team noticed that we may not be compliant with section 5.4.1.2.5. Further investigation confirmed that we were not.

A timeline of the actions your CA took in response.

• 2020-12-22: Internal audit detected that we were not compliant with the requirement that audit logs be kept on OCSP updates to subscriber certificates.
• 2020-12-22: Started incident report disclosure procedure.

Whether your CA has stopped, or has not yet stopped, certificate issuance or the process giving rise to the problem or incident.

• Issuance was not stopped for this incident.

In a case involving certificates, a summary of the problematic certificates. For each problem: the number of certificates, and the date the first and last certificates with that problem were issued.

• Scope of impact is all certificates issued from the Let’s Encrypt Intermediate Certificate Authorities, with the exception of any certificates that were revoked before the second OCSP signing update. For the purposes of this disclosure, it would take an unreasonable amount of time and effort to find the relatively small number of exceptions.

In a case involving certificates, the complete certificate data for the problematic certificates.

• The list of certificates impacted would be too long to paste here, but all certificates issued by Let’s Encrypt have been published to Certificate Transparency. The following Intermediates have been in service and issued certificates during this time in order of most recent to oldest.
  • Let’s Encrypt Authority R3: https://crt.sh/?caid=183267
  • Let’s Encrypt Authority X3: https://crt.sh/?caid=16418
  • Let’s Encrypt Authority X1: https://crt.sh/?caid=7395

Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

• No bugs or changes were introduced. This was a misunderstanding of the Baseline Requirements that was caught in a recent internal audit.

List of steps your CA is taking to resolve the situation and ensure that such situation or incident will not be repeated in the future, accompanied with a binding timeline of when your CA expects to accomplish each of these remediation steps.

• OCSP signing is a very high throughput service. This requires some consideration to our storage needs to adequately store the logs.
• A change to our Boulder CA software is needed to implement the logging.
• Our team will be working on both of these items with a target date of 2020-01-31 for completion.

Comment 2

[Andrew Gabbitas]

Correction to our timeline for remediation:

Our team will be working on both of these items with a target date of 2020-01-31 for completion.

Our team will be working on both of these items with a target date of 2021-01-31 for completion

Comment 3

[Kiel C]

We have deployed updated CA software with pertinent logging changes through our Staging infrastructure and to our Production infrastructure.

Comment 4

[Ben Wilson]

I believe this bug can be closed. I will plan on closing it next Wed. 7-Apr-2021 unless anyone notifies me that there is still remediation work to be done.