1391867 - Let's Encrypt: Non-BR-Compliant Certificate Issuance

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Kathleen Wilson]

**  Let's Encrypt (ISRG) already responded in the mozilla.dev.security.policy forum about this incident, but it was requested that I still file this bug and ask the CA to respond here.

The following problems have been found in certificates issued by your CA, and reported in the mozilla.dev.security.policy forum. Direct links to those discussions are provided for your convenience.

To continue inclusion of your CA’s root certificates in Mozilla’s Root Store, you must respond in this bug to provide the following information:
1) How your CA first became aware of the problems listed below (e.g. via a Problem Report, via the discussion in mozilla.dev.security.policy, or via this Bugzilla Bug), and the date.
2) Prompt confirmation that your CA has stopped issuing TLS/SSL certificates with the problems listed below.
3) Complete list of certificates that your CA finds with each of the listed issues during the remediation process. The recommended way to handle this is to ensure each certificate is logged to CT and then attach a CSV file/spreadsheet of the fingerprints or crt.sh IDs, with one list per distinct problem.
4) Summary of the problematic certificates. For each problem listed below: number of certs, date first and last certs with that problem were issued.
5) Explanation about how and why the mistakes were made, and not caught and fixed earlier.
6) List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.
7) Regular updates to confirm when those steps have been completed.

Note Section 4.9.1.1 of the CA/Browser Forum’s Baseline Requirements, which states:
“The CA SHALL revoke a Certificate within 24 hours if one or more of the following occurs: …
9. The CA is made aware that the Certificate was not issued in accordance with these Requirements or the CA’s Certificate Policy or Certification Practice Statement; 
10. The CA determines that any of the information appearing in the Certificate is inaccurate or misleading; …
14. Revocation is required by the CA’s Certificate Policy and/or Certification Practice Statement; or 
15. The technical content or format of the Certificate presents an unacceptable risk to Application Software Suppliers or Relying Parties (e.g. the CA/Browser Forum might determine that a deprecated cryptographic/signature algorithm or key size presents an unacceptable risk and that such Certificates should be revoked and replaced by CAs within a given period of time).

However, it is not our intent to introduce additional problems by forcing the immediate revocation of certificates that are not BR compliant when they do not pose an urgent security concern. Therefore, we request that your CA perform careful analysis of the situation. If there is justification to not revoke the problematic certificates, then explain those reasons and provide a timeline for when the bulks of the certificates will expire or be revoked/replaced. 

We expect that your forthcoming audit statements will indicate the findings of these problems. If your CA will not be revoking the certificates within 24 hours in accordance with the BRs, then that will also need to be listed as a finding in your CA’s BR audit statement.

We expect that your CA will work with your auditor (and supervisory body, as appropriate) and the Root Store(s) that your CA participates in to ensure your analysis of the risk and plan of remediation is acceptable. If your CA will not be revoking the problematic certificates as required by the BRs, then we recommend that you also contact the other root programs that your CA participates in to acknowledge this non-compliance and discuss what expectations their Root Programs have with respect to these certificates.


The problems reported for your CA in the mozilla.dev.security.policy forum are as follows:

** Failure to respond within 24 hours after Problem Report submitted
https://groups.google.com/d/msg/mozilla.dev.security.policy/PrsDfS8AMEk/w2AMK81jAQAJ
The problems were reported via your CA’s Problem Reporting Mechanism as listed here:
https://ccadb-public.secure.force.com/mozilla/CAInformationReport
Therefore, if this is the first time you have received notice of the problem(s) listed below, please review and fix your CA’s Problem Reporting Mechanism to ensure that it will work the next time someone reports a problem like this. 

** 
Improperly normalized IDNs
https://groups.google.com/d/msg/mozilla.dev.security.policy/g6_zGA2exXw/izYkdc7DBwAJ
RESOLUTION: https://groups.google.com/d/msg/mozilla.dev.security.policy/nMxaxhYb_iY/AmjCI3_ZBwAJ

Comment 1

[Josh Aas]

1) How your CA first became aware of the problems listed below (e.g. via a Problem Report, via the discussion in mozilla.dev.security.policy, or via this Bugzilla Bug), and the date.

At 11:30am PST on August 10, 2017, Let’s Encrypt was made aware of a compliance issue regarding unicode normalization of domain names. We were notified of the problem by a community member.

2) Prompt confirmation that your CA has stopped issuing TLS/SSL certificates with the problems listed below.

A fix was applied to our production infrastructure on the same day in which we were notified of the problem.

3) Complete list of certificates that your CA finds with each of the listed issues during the remediation process. The recommended way to handle this is to ensure each certificate is logged to CT and then attach a CSV file/spreadsheet of the fingerprints or crt.sh IDs, with one list per distinct problem.

03d03877cbcec666b81340ed6a39c47556d1
03758d04a7816ba893847658e636a58e1f71
03ef82538ca2e54e97ae2b180ecb32f8cee4
044568f36455d8847713adb24c69e60bf123
033e73ebfd2f270bc6109925f1ed40edca8b
038295d240613cdb9367506c0d3cf8002401
03556fbc38b13ea3a9b7f4dd59dacc350293
030cfe12721e17ca02c095b4a0c5e60ca8da
03ca6617e634f2f5ad9224ca32ca4c835909
03bd090cfe0fbd07b4fc60df07bbc5770b35
0314017b4eab87bb0f211e9e2bb329ca4297
03f48a8c02c473ce971236b6407ad7d00d89
03bfa7b8f318a30a88894523ebd2717ea9b4
032d7c46b0a815faa41a1876fed4d66a9993
039f94badc798eea44f8c81ceb0515024871
038f81a32455e41b702ffb1732186be3a007

4) Summary of the problematic certificates. For each problem listed below: number of certs, date first and last certs with that problem were issued.

The problematic code was introduced on October 20, 2016. That is when the first problematic certificates could have been issued. The last certificate with the problem was issued on August 10, 2017. All of our certificates have 90 days lifetimes. We found 16 unexpired affected certificates in total.

5) Explanation about how and why the mistakes were made, and not caught and fixed earlier.

It was simply a mistake that was not caught during CA software code review. Part of the reason for this is that the relevant RFCs are not easy to understand.

6) List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.

A fix was developed, reviewed, tested and deployed within hours of our being notified of the problem. Our staff are more clear on what the requirements for IDN encoding are now.

7) Regular updates to confirm when those steps have been completed.

This matter was resolved in its entirety on the day we were notified of the problem.