1689589 - Telia: Disallowed curve (P-521) in leaf certificate

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Rob Stradling]

https://crt.sh/?id=3979591389&opt=zlint has an ECDSA key on the P-521 curve, but https://www.mozilla.org/en-US/about/governance/policies/security-group/certs/policy/#51-algorithms only permits P-256 and P-384 to be used.

Other CAs have made the same mistake in the past, as discussed in this m.d.s.p thread:
https://www.mail-archive.com/[e-mail]/msg11131.html

Comment 1

[pekka.lahtiharju]

1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.
   Telia became aware of the problem from the Incident report email from Ben Wilson Sent: Fri 29 Jan 2021 17.48 EEST.

2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.
   Times below are EEST (UTC+2)
   [Mon 25.01.2021 12:57:18] Certificate was created using Telia ACME solution
   [Mon 25.01.2021 23:58:02] Telia lint checker found erroneus certificate and sent email alarm as specified
   [Fri 29.01.2021 17:48] Telia got normal level incident report email from Ben Wilson
   [Mon 1.2.2021 8-10] Incident was evaluated by Telia PKI team and confirmed to require actions; reasons were investigated, corrective actions were started, similar cases investigated (none was found).
   [Mon 1.2.2021 9:55:38] Illegal certificate was revoked
   [Mon 1.2.2021 9-10] Better Lint alarming rules were established in Telia – Lint alarm rules were enhanced to reveal further similar kind of incidents
   [Mon 1.2.2021 9-10] Bug fix preventing this error was initiated

3. Whether your CA has stopped, or has not yet stopped, certificate issuance or the process giving rise to the problem or incident. A statement that you have stopped will be considered a pledge to the community; a statement that you have not stopped requires an explanation.
   Telia has initiated fix in ACME server for root cause, this is currently under development. All other Telia SSL processes (SSL order, SSL self-service) were immediately tested and similar behavior was not found. In addition Telia has improved processes so that further similar incidents are found and illegal certificates are revoked swiftly.

4. In a case involving certificates, a summary of the problematic certificates. For each problem: the number of certificates, and the date the first and last certificates with that problem were issued. In other incidents that do not involve enumerating the affected certificates (e.g. OCSP failures, audit findings, delayed responses, etc.), please provide other similar statistics, aggregates, and a summary for each type of problem identified. This will help us measure the severity of each problem.
   Certificate https://crt.sh/?id=3979591389&opt=zlint had the issue. All active Telia SSL certificates were re-linted to be sure that this is the only one.

5. In a case involving certificates, the complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem. In other cases not involving a review of affected certificates, please provide other similar, relevant specifics, if any.
   https://crt.sh/?id=3979591389

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.
   This was combination of two problems: a) Telia ACME server let this happen, b) Telia lint alarming was unnotified by Telia PKI team.
   When Telia originally implemented the prevention to use P-521 keys it was tested within systems existing at the time (SSL order, SSL self-service). Recently Telia implemented a third SSL method to automate SSL certificate creation: ACME server. Specification for P-521 prevention code was adopted from other certificate software but looks like the prevention is not working for ACME. We are currently investigating why. Telia verified that the other Telia code (SSL order, SSL self-service) are still preventing P-521. Tests with ACME with P-521 were never done because same configuration prevented it in other use cases.
   Telia is using daily lint checking. It found this problem as expected and sent the alarm to persons responsible of those. Unfortunately the person in charge didn't notice this alarm because of human mistake. All previous lint errors have been noticed like the previous case in Oct 2020 https://bugzilla.mozilla.org/show_bug.cgi?id=1674536

7. List of steps your CA is taking to resolve the situation and ensure that such situation or incident will not be repeated in the future, accompanied with a binding timeline of when your CA expects to accomplish each of these remediation steps.
   Telia will do (or has done) several improvements to prevent similar incidents in the future. Telia has already verified that this was the only lint error that was unnoticed. Telia has also verified that other SSL processes prevent P-521 usage. Telia has revoked the problematic certificate. The improvements are:
   a) Telia ACME server will be fixed ASAP to prevent P-521 usage
   b) P-521 test case in ACME channel is added to regular test scenarios
   c) Lint error alarms are followed by three other persons in addition to the one in charge of those
   d) New regular monthly lint check (previously quarterly) will be done from now on to verify that no lint problems have been unnotified or if lint has been updated to find problems that were previously unnotified by it.

Comment 2

[pekka.lahtiharju]

Now Telia has fixed the bug in ACME Server component. Previously it didn't correctly read the configuration that deny using P-521 curve allowed by ACME clients. Now it is rejecting such invalid CSR and writing this to log: "ERROR: Request rejected at CA server: CSR checks not passed". Process improvements related to lint error handling are also in place. Telia is ready to close this bug.

Comment 3

[Ben Wilson]

I believe this bug can be closed. I'll schedule it for closure on or about 5-Feb-2021 unless there are additional issues to discuss.