1435770 - Asseco DS / Certum: Non-BR-Compliant Issuance - Debian Weak Keys

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Wayne Thayer]

Hanno Bock reported the following two certificates containing Debian weak keys to Certum on 3-Feb 2018:

https://crt.sh/?id=308392091&opt=ocsp
https://crt.sh/?id=6888863&opt=ocsp

The BRs require these certificates to be revoked within 24 hours in section 4.9.1.1(3) and the definition of Key Compromise. As of 5-Feb 2018 these certificates were not revoked.

Please provide an incident report in this bug, as described here:
https://wiki.mozilla.org/CA/Responding_To_A_Misissuance#Incident_Report

Comment 1

[Wayne Thayer]

As part of the incident investigation, please scan every active )non-expired, non-revoked) certificate issued by Certum for this problem.

Comment 2

[Arkadiusz Ławniczak]

Certum CA obtained unsigned email concerning two RSA keys on Saturday, Feb 3 2018.

Our staff started  risk and technical analyze to check reported issue.
In cooperation with our customers we have analyzed both RSA keys and confirmed that both certifacate must be revoked.
Both x509 certificates have been revoked with the revocation reason "Key Compromise" and the invalidity date Feb 03, 2018.
Then we checked all issued certificates against Debian weak keys in our database and did not find anyone more.
We have recognised this as security incident.
Our requests checking software is being analyzed at this time.
We will keep you inform.

Comment 3

[Wojciech Trapczyński]

I am writing on behalf of Arkadiusz Ławniczak. 

We have found the cause of wrong validation against Debian weak keys. We fixed it and deployed on production on Feb 8, 2018. Now, any certification requests based on Debian weak keys are rejected. 

After that we scanned our certificates database once again and we did not find any active certificates with a key vulnerable to the Debian OpenSSL bug. 

The official report will be sent by Arkadiusz.

Comment 4

[Firefox Bug Husbandry Bot]

Changing QA contact per https://bugzilla.mozilla.org/show_bug.cgi?id=1438254

Comment 5

[Arkadiusz Ławniczak]

Hello ALL
Please find our incident report below.

1.	How your CA first became aware of the problem and the time and date.

1)	3 February 2018, 12:06 CET -  Certum receives the message from [e-mail] to [e-mail].



2.	A timeline of the actions CERTUM took in response.

1)	3 February 2018, 12:06 CET -  Certum receives the message from [e-mail] to [e-mail].
2)	5 February 2018, 15:37 CET and 15:56 CET - Certum informs subscribers (owners of *.orix.pl and ftp.gavdi.pl domains) 
        about the obtained problem report.
3)	5 February 2018, 15:37 CET and 15:56 CET - Certum request subscribers to provide private keys checksums.
4)	5 February 2018, 16:03 CET - Certum informs Hanno Boeck that received the problem report.
5)	6 February 2018, 16:03 CET - Certum revokes certificates SHA1 - 6E8B5A67417FDBDE2871A28ED7A2C30FEE686390 and SHA1 - 
        882AE1C8660BB75E3311ACB0CEBCD3C3FF9167E3.
6)	6 February 2018 - Certum scans certificates database. No weak keys was found.
7)	6 February 2018 - Certum scans certificates database. The problem with validation against Debian weak key was found.
8)	8 February 2018 - Certum deploys a new version of the weak keys validation system.
9)	8 February 2018 - Certum Certum scans certificates database. The problem with validation against Debian weak key was not 
        found.


3.	A summary of the problematic certificates.

The total number of certificates with the problem is 2:
1)	https://crt.sh/?id=308392091&opt=ocsp 
2)	https://crt.sh/?id=6888863&opt=ocsp


4.	Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

The issue was caused by incorrect calculation of the SHA1 fingerprint of public key.
Public keys hashes stored in Certum's  database was calculated from the Modulo key value with the Modulus prefix and a line ending character while the value of public key from CSR was calculated and returned without these additional characters. 
So, this is the reason why the calculated fingerprint did not match the value from  Certum's database.

5.	List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future

Certum standardized formats of the validated and stored weak keys to be compliant with the following sources:
https://anonscm.debian.org/viewvc/pkg-openssl/openssl-blacklist/tags/0.5-2/blacklists/le64/blacklist-4096.db?view=co
https://anonscm.debian.org/viewvc/pkg-openssl/openssl-blacklist/tags/0.5-2/blacklists/le32/blacklist-4096.db?view=co
https://anonscm.debian.org/viewvc/pkg-openssl/openssl-blacklist/tags/0.5-2/blacklists/be32/blacklist-4096.db?view=co
https://anonscm.debian.org/viewvc/pkg-openssl/openssl-blacklist/trunk/blacklists/be32/blacklist-2048.db?view=co
https://anonscm.debian.org/viewvc/pkg-openssl/openssl-blacklist/trunk/blacklists/le32/blacklist-2048.db?view=co
https://anonscm.debian.org/viewvc/pkg-openssl/openssl-blacklist/trunk/blacklists/le64/blacklist-2048.db?view=co
https://anonscm.debian.org/viewvc/pkg-openssl/openssl-blacklist/trunk/blacklists/be32/blacklist-1024.db?view=co
https://anonscm.debian.org/viewvc/pkg-openssl/openssl-blacklist/trunk/blacklists/le32/blacklist-1024.db?view=co
https://anonscm.debian.org/viewvc/pkg-openssl/openssl-blacklist/trunk/blacklists/le64/blacklist-1024.db?view=co

Comment 6

[Wayne Thayer]

Arkadiusz: thank you for the incident report.

(In reply to Arkadiusz Ławniczak from comment #5)
 
> 4.	Explanation about how and why the mistakes were made or bugs introduced,
> and how they avoided detection until now.
> 
> The issue was caused by incorrect calculation of the SHA1 fingerprint of
> public key.
> Public keys hashes stored in Certum's  database was calculated from the
> Modulo key value with the Modulus prefix and a line ending character while
> the value of public key from CSR was calculated and returned without these
> additional characters. 
> So, this is the reason why the calculated fingerprint did not match the
> value from  Certum's database.

Please provide an explanation of the root cause of this problem, and what is being done to prevent similar problems from occurring in the future. For example, why was this bug not caught in testing? Are there adequate automated testing processes in place? Is there adequate staffing and focus to ensuring the quality of changes made to your systems? What needs to change to prevent a similar problem in the future?

Also, since this issue was originally posted to the mozilla.dev.security.policy forum (https://groups.google.com/d/msg/mozilla.dev.security.policy/g9YkWkmbZbU/MKh71_CoAwAJ), please also post your incident report there.

Comment 7

[Arkadiusz Ławniczak]

Weak keys verification is tested each time before the new version of the software is deployed and also periodically as part of the test schedule.
Unfortunately, the database of weak keys that served the tests contained keys hashes in incorrect formats, the parsed key was also in an incorrect format. Therefore we could not recognize weak key in its "original" OpenSSL form. So each test returned false positives.

Comment 8

[Wayne Thayer]

I am not completely satisfied with this answer because it does not explain how similar problems will be prevented in the future, but it has been posted to m.d.s.p. and received the following response:

> This is a reminder that just because your unit tests pass, doesn't mean
> your larger system behaves how you think the unit tests mean it does. If
> you want to be sure how the whole _system_ behaves (and for a CA we
> certainly do want that) you're going to need to explicitly test that
> whole system even if your unit tests are green. 

With that I am marking this resolved.