1832093 - Asseco DS / Certum: Subordinate certificates with sequential serial number

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Wojciech Trapczyński]

Steps to reproduce:

As part of the periodic self-audit we discovered that the serial number of one of the intermediate certificate may have been generated with insufficient entropy. We will provide a full incident report within 7 days.

Comment 1

[Wojciech Trapczyński]

1. How your CA first became aware of the problem (e.g., via a problem report submitted to your Problem Reporting Mechanism, a discussion in the MDSP or CCADB public mailing list, a Bugzilla bug, or internal self-audit), and the time and date.

As part of the periodic self-audit on May 9, 2023 we detected that the serial number of two of the subordinate certificates is sequential. Further analysis showed that there are one more subordinate certificate with a sequential serial number.

2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a requirement became applicable, a document changed, a bug was introduced, or an audit was performed.

2023-05-09 11:30 CEST – As part of the periodic self-audit we found a two subordinate certificates we suspected had a sequential serial number.
2023-05-09 14:00 CEST – We confirmed that the serial numbers of these two subordinate certificates are sequential.
2023-05-09 16:46 CEST – We disclosed our findings in this bug. The initial report concerned only two subordinate certificates.
2023-05-10 10:00 CEST – We began a detailed investigation process in order to find a root cause.
2023-05-10 11:40 CEST – We revoked the first subordinate certificate from initial finding.
2023-05-11 13:30 CEST – We identified and confirmed the reason for the generation of the subordinate certificates with sequential serial numbers.
2023-05-12 12:00 CEST – We implemented a fix for software we use in process of subordinate certificates generation.
2023-05-12 13:00 CEST – We expanded a set of tests for CA certificates.
2023-05-15 10:11 CEST – We revoked the second subordinate certificate from initial finding.
2023-05-15 11:00 CEST – We started review all our database for CA certificates with a sequential serial number.
2023-05-15 13:15 CEST – We found another (the third) subordinate certificate with a sequential serial number during the comprehensive scan. The revocation of this subordinate certificate is scheduled on 2023-05-22 12:00 CEST.
2023-05-15 14:00 CEST – We completed the review process of all CA certificates. We found no other affected CA certificates.

3. Whether your CA has stopped, or has not yet stopped, certificate issuance or the process giving rise to the problem or incident. A statement that you have stopped will be considered a pledge to the community; a statement that you have not stopped requires an explanation.

We updated our systems to prevent additional issuance of CA certificates with a sequential serial number.

4. In a case involving certificates, a summary of the problematic certificates. For each problem: the number of certificates, and the date the first and last certificates with that problem were issued. In other incidents that do not involve enumerating the affected certificates (e.g., OCSP failures, delayed responses, etc.), please provide other similar statistics, aggregates, and a summary for each type of problem identified. This will help measure the severity of each problem.

Three subordinate certificates were impacted. The first one was issued 2020-09-09, the last one was issued 2023-03-16.

5. In a case involving TLS server certificates, the complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem. It is also recommended that you use this form in your list “https://crt.sh/?sha256=[sha256-hash]”, unless circumstances dictate otherwise. When the incident being reported involves an SMIME certificate, if disclosure of personally identifiable information in the certificate may be contrary to applicable law, please provide at least the certificate serial number and SHA256 hash of the certificate. In other cases not involving a review of affected certificates, please provide other similar, relevant specifics, if any.

https://crt.sh/?q=E21A784343F763F5EFE894DDC81E0B5B92A89DF65056FCA8C49AD4414CD74EB4
https://crt.sh/?q=2E198C227F93F9CF96F7B8A1986F659735502C956407DB36F0D4A3D601F42472
https://crt.sh/?q=6C834388F2A0D06949C63EFCF30D77782E25A8250135939E54960214540EEA45

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

The problem could only affect CA certificates and not the end-entity certificates. All affected certificates were generated offline with dedicated software. This software uses templates with predefined profiles of subordinate certificates. When generating serial number, the software uses the configuration file with the serial number value generated with the appropriate entropy. This config file has to be empty before a new certificate could be generated. If it was not and contained the serial number of the previously generated certificate the software generate a sequential serial number and then put it in the certificate. The task of resetting this configuration file was performed manually by a person. In these three cases this config file was not empty before the procedure started.

Each CA certificate generation ceremony follows a specific procedure and is preceded among other things, by a thorough verification of the data that goes into the certificate subject and checking compliance with the standards of the profile that will be used. All CA certificates are generated offline in a tempest chamber. Once a CA certificate is generated, before it can be used in production, it is verified in a number of ways, including tests of signatures and algorithms, once again the certificate profile conformance to standards is checked, once again the compliance of the data in the subject is checked. The length and uniqueness of the serial number is also checked. All these tests take place offline. Once all the tests performed offline have passed, the certificate is forwarded to the compliance team, which finally approves it.

We have not detected this until now for two reasons:

• The offline tests did not include checking whether the serial number of the CA certificate is not-sequential.
• The self-audit, thanks to which we detected the problem, include checking wheter the serial number of the CA Certificate is not-sequential. We added this test in Q3 2021 (unitl then we check it only for end-entity certificates). Since then we checked it but only for new CA certificates. That's why we didn't detect the earlier.

7. List of steps your CA is taking to resolve the situation and ensure that such a situation or incident will not be repeated in the future. The steps should include the action(s) for resolving the issue, the status of each action, and the date each action will be completed.

2023-05-12 12:00 CEST – We implemented a fix for software we use in process of subordinate certificates generation. We added automatic erasing of configuration file used in the process of generating serial numbers for subordinate certificates.
2023-05-12 13:00 CEST – We expanded a set of offline tests for CA certificates. We added a unit for testing similarity of the serial numbers which help us find sequential serial numbers. This check is performed before the serial number is entered in the certificate.
2023-05-15 14:00 CEST – We completed the review process of all CA certificates for which we used the similarity test mentioned above. We found no other affected CA certificates.

Comment 2

[Wojciech Trapczyński]

The certificate https://crt.sh/?q=6C834388F2A0D06949C63EFCF30D77782E25A8250135939E54960214540EEA45 was revoked as scheduled.

Comment 3

[Matthias]

That certificate shows "good" on the ocsp, according to both my local check and the crt.sh ocsp validator [0], as opposed to the "Revoked (superseded)" of the other two certificates you linked in Comment 1.

[0] https://crt.sh/?id=3380671365&opt=ocsp

Comment 4

[Wojciech Trapczyński]

We verified the OCSP status yesterday just after we updated our database and it was revoked. We noticed that the crt.sh OCSP validator had some delay in updating status, but now is revoked. It seems to me that it may have been some sort of caching issue.

Comment 5

[Wojciech Trapczyński]

We have no further updates.

Comment 6

[Ben Wilson]

I intend to close this bug on or about Friday, 2-June-2023, unless there are any questions or concerns.