Closed Bug 1397954 Opened 8 years ago Closed 8 years ago

DigiCert / Siemens: Insufficient Serial Number Entropy

Categories

(CA Program :: CA Certificate Compliance, task)

Product:
CA Program
Component:
CA Certificate Compliance
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: kathleen.a.wilson, Assigned: jeremy.rowley)

References

Details

(Whiteboard: [ca-compliance] [ov-misissuance])

This bug has been separated out from Bug #1389172 to request an incident report specific to this subCA. Siemens a) Insufficient Serial Number Entropy - See Bug #1389172: Comments 0, 1, 4, 6, 9, 13, 14, 15 - Remediation - 2017-09-15 - CA certificate will be revoked For the problem listed above, please provide an incident report as described here: https://wiki.mozilla.org/CA/Responding_To_A_Misissuance#Incident_Report
Siemens is scheduled for revocation on 9/15/2017. They did not revoke their certificates within the required 24 hour time-frame and provided insufficient assurances about remediation. I've cc'd Markus who can provide any additional context from Siemens. In answer to the questions: 1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, via a discussion in mozilla.dev.security.policy, or via a Bugzilla bug), and the date. - A certificate problem report was sent to us. We promptly notified the Sub CA. 2. A timeline of the actions your CA took in response. - We have had several calls with Siemens. We are not satisfied with the results, hence the pending revocation. Siemens is no longer using this intermediate. 3. Confirmation that your CA has stopped issuing TLS/SSL certificates with the problem. - Confirmed. Siemens is issuing from Quovadis and no longer using the Verizon intermediate. We planned to simply wait for all existing certificates to expire, but Siemens failed to meet the revocation timeline required under the BRs. 4. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued. - Lots. The lack of serial number entropy appears pervasive. The last certificate was issued on Feb 2017. 5. The complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem. - This is in the other bug. Do you want me replicate it here? 6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now. - From what I gather, the mistake is in the CA software they are using. It didn't generate enough entropy. I'll leave it for Markus to provide further clarity around the CA software. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things. - Revoking the Sub CA on Sep 15th. Jeremy
Flags: needinfo?(ben.wilson)
(In reply to Jeremy from comment #1) > In answer to the questions: > 1. How your CA first became aware of the problem (e.g. via a problem report > submitted to your Problem Reporting Mechanism, via a discussion in > mozilla.dev.security.policy, or via a Bugzilla bug), and the date. > - A certificate problem report was sent to us. We promptly notified the Sub > CA. Given the lack of detail in #2 below, "promptly" is insufficiently vague. > 2. A timeline of the actions your CA took in response. > - We have had several calls with Siemens. We are not satisfied with the > results, hence the pending revocation. Siemens is no longer using this > intermediate. Note that this is not a timeline of actions taken. > 3. Confirmation that your CA has stopped issuing TLS/SSL certificates with > the problem. > - Confirmed. Siemens is issuing from Quovadis and no longer using the > Verizon intermediate. How have you confirmed this? That is, is this a procedural control, a technical control, a contractual assertion? > We planned to simply wait for all existing > certificates to expire, but Siemens failed to meet the revocation timeline > required under the BRs. > > 4. A summary of the problematic certificates. For each problem: number of > certs, and the date the first and last certs with that problem were issued. > - Lots. The lack of serial number entropy appears pervasive. The last > certificate was issued on Feb 2017. This is not an answer to the question stated. > 5. The complete certificate data for the problematic certificates. The > recommended way to provide this is to ensure each certificate is logged to > CT and then list the fingerprints or crt.sh IDs, either in the report or as > an attached spreadsheet, with one list per distinct problem. > - This is in the other bug. Do you want me replicate it here? Yes, please.
Flags: needinfo?(jeremy.rowley)
2. We contacted Siemens the same day as we received the certificate problem report. In that communication, we reminded them that the BRs require revocation of these certificates within 24 hours after receiving notice. Siemens replied that they were working on the issue with Quovadis and would post their response there. Quovadis posted their reply on Aug 18. Siemens has not yet revoked all mis-issued certs. 3. Contractual and procedural. Siemens uses a sub CA from Quovadis instead of DigiCert for issuance. The one from DigiCert was originally issued by Verizon. 4+5. Here is the list. The last one was issued on Feb 20 2017 and expires on Feb 20 2018. 45010775 45000519 45073660 40700407 48827344 40726549 45068282 48827312 48726324 48726344 48726335 45068035 45060262 41273940 179803714 48827347 48827332 48827315 48827348 48726294 45029804 45053316 45061425 43151162 43151152 43151164 45060837 54576144 48881965 48726336 44438214 47906173 45048889 45257938 81268921 45257935 133008905 132946615 48726297 47884474 46110537 47211018 42791689 44377709 91004967 47612195 47646461 47612190 48827338 48715703 48827336 48726376 48773656 47743543 48773648 48773658 48773657 54576085 48773662 47605232 48827352 48773645 49706248 48827353 179801334 48773663 49706251 54576119 54576115 47939190 48827317 54397723 47815889 53205222 54576127 50756547 48773638 50165772 50756565 54576082 95852573 102169703
Flags: needinfo?(jeremy.rowley)
I have asked Siemens for a status update and for them to file an Incident Response themselves.
Flags: needinfo?(ben.wilson)
@all: We (Siemens internal PKI) have contacted all our customers (all of the servers are operated by Siemens or our contractors) by email twice and informed them, that there certificates will be invalid after Friday, 15th of September and that they need to request new certificates. The response so far is rather disappointing. Starting today we will inform also the managers of the customers and we will call some of the administrators who own many of the certificates. @Ben: Markus will provide you with a detailed answer on the official channel.
Is the QuoVadis intermediate you are switching to on-premises or CA-managed? Gerv
(In reply to Gervase Markham [:gerv] from comment #6) > Is the QuoVadis intermediate you are switching to on-premises or CA-managed? > > Gerv The CA is hosted on-premise. The private key is stored on an Utimaco HSM. Creation of the key pair was witnessed by our ETSI auditor from DQS on 19th of June 2017. The biggest change compared to the old CA is, that we moved from an in-house developed CA software to EJBCA by PrimeKey. We are convinced, that with this change, we will avoid traps like this serial number entropy issue. This change was already planned due to the CT requirements and we increased the speed of the change over in August. EJBCA itself is operated by the Siemens PKI for IoT business since two years without external trust relationships. /Rufus
(One additional remark to comment #7 from Rufus Buschart) > (In reply to Gervase Markham [:gerv] from comment #6) > > Is the QuoVadis intermediate you are switching to on-premises or CA-managed? > > > > Gerv > > The CA is hosted on-premise. > [...] > > The biggest change compared to the old CA is, that we moved from an in-house > developed CA software to EJBCA by PrimeKey. [...] EJBCA itself is operated by the Siemens > PKI for IoT business since two years without external trust relationships. The RA is also hosted on-premise but completely separated from the CA. Communication between RA and CA is only CMP. We are not using EJBCA as RA software, but an in-house developed tool, that is highly integrated within our business processes. /Rufus
We are down to 25 certificates of this CA whose owners haven't reacted yet. We continue to call the administrators. I'll provide an update by tomorrow evening.
We are down to 15 certificates of this CA whose owners haven't reacted yet. We continue to call the administrators. I'll provide an update by tomorrow evening.
We are down to 2 certificates of this CA whose owners haven't reacted yet. We will revoke centrally all certificates which haven't revoked by the users themselves actively by tomorrow (Friday, 15th of September 2017) evening. I'll keep you updated on the weekend. /Rufus
All certificates from this CA issued after 30th of September 2016 (effective date of Ballot 164) were revoked on Friday, 15th of September 2017. DigiCert has filed a bug https://bugzilla.mozilla.org/show_bug.cgi?id=1400600 to include this CA in the OneCRL. If you have any further questions, please don't hesitate to ask. Also one member of my team will (most probably) be present at the Google Certificate Transparency Days in New York in November and can also answer all question in person. We will continue with the same procedure for the Bug https://bugzilla.mozilla.org/show_bug.cgi?id=1391063 . Anyhow I think this bug can be closed. /Rufus
Will use Bug #1400600 to track adding these intermediate certs for this SubCA to OneCRL.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Hi everyone, Let me briefly introduce myself, I am working in Siemens central Information Security Department, and represent here with Rufus our Corporate PKI. Following our statements above, we would like to inform you about our further procedures and current situation regarding the Siemens Internet CA V1.0 (chaining to Baltimore Cybertrust Root). This CA certified the following Issuing CAs: (1) Siemens Issuing CA Class Internet Server 2013 (2) Siemens Issuing CA Class Internet Code Signing 2013 (3) Siemens Issuing CA EE Enc 2013 (4) Siemens Issuing CA Multipurpose 2013 (5) Siemens Issuing CA Internet Code Signing 2011 (expired) (6) Siemens Issuing CA Class Internet Server 2011 (expired) The Siemens Internet CA V1.0 is part of the Siemens CA Hierarchy of 2011 and it issued also the CAs of the Hierarchy 2013, which was succeeded by the Siemens CA Hierarchy 2016 on November 2, 2016. Since that date, the CA Hierarchy was not used to issue new certificates any more, only to maintain the already issued certificates. Currently, our CA Hierarchies 2016 and 2017 chains to QuoVadis as external trust anchor. After the issues listed in this bug, and as stated above, we exchanged all mis-issued SSL/TLS certificates and will also exchange the remaining compliant SSL/TLS certificates and revoke (1) on October 4. Additionally, DigiCert requested to list the Siemens Internet CA 2013 on OneCRL to avoid further usage for SSL/TLS certificates, in accordance with the steps Siemens and DigiCert agreed upon to prevent any Future issues. New Siemens SSL/TLS certificates will be issued under our new CA Hierarchy chaining to QuoVadis. Siemens will not take any actions to issue any new SSL/TLS certificates chaining to the Baltimore Root CA. All still remaining valid certificates chaining to the Siemens Internet CA V1.0 were issued in compliance with all requirements, and are vital for Siemens business: (2) 55 Code Signing certificates are still in use chaining to this CA, used for Siemens software and products (3) and (4): More than 175.000 Siemens employees and contractors are using certificates chaining to these Issuing CAs for S/MIME e-mail encryption, providing trust for secure communication with external partners. All certificates chaining to (2) and (3) have a maximum standard validity of 3 years, the last certificate will expire in November 2019. Certificates chaining to (4) have a validity of 1 year, the last will expire in November 2017, the CA will be revoked afterwards. Exchange of these certificates is currently taking place upon expiration. New certificates are only issued under our new CA Hierarchy chaining to QuoVadis, all Issuing CAs are only used to maintain the issued certificates. CA Last expiring* (1) Siemens Issuing CA Class Internet Server 2013 CA to be revoked no Oct. 4, 2017, all issued certificates to be revoked by end of Sept. 2017 (2) Siemens Issuing CA Class Internet Code Signing 2013 2019-10-11 (3) Siemens Issuing CA EE Enc 2013 2019-11-30 (4) Siemens Issuing CA Multipurpose 2013 2017-11-02 *exclusive OCSP signer To keep trust in the certificates issued by (2), (3) and (4), considering our actions to prevent any further issuing of (1), with the support of DigiCert, we want to inform the community about this situation and to ask for your support to maintain the Siemens Internet CA 2013 until the expiration of the last certificate chaining to it in November 2019. What assurances would the browsers like that no additional certificates are being issued from these intermediates, to prevent any negative action against these roots during the transition? \Markus
Hi everyone, I would like to keep you updated on our actions, as announced in my previous post. Today, we revoked the last remaining compliant SSL/TLS certificates chaining to (1). The CRL is in the publishing process, it will be available on the Internet during this evening. We only left valid our OCSP responder signer certificates and the SSL test site certificate: DN Serial number CN=Siemens PKI OCSP Signer ZZZZZZY9,O=Siemens,C=DE 1332183347 CN=2013-internet-valid.catestsite.siemens.com, L=Muenchen,SP=Bayern,C=DE,O=Siemens-Internet,OU=GS IT HR 74,SN=ETSI0002 1784355851 These certificates will be revoked together with (1) on Oct. 4. I will keep you informed. \Markus
Hi everyone, Update on our actions. Today, Oct. 4, we revoked (1) Siemens Issuing CA Class Internet Server 2013, as announced above. The revocation lists are in the publishing process, will be available on the Internet during this evening. \Markus
(In reply to Markus Wichmann from comment #14) > What assurances would the browsers like that no additional certificates are > being issued from these intermediates, to prevent any negative action > against these roots during the transition? We accept either a statement from auditors, or addition to OneCRL (which I understand is planned). Gerv
Thanks, Gerv - and yes, (1) Siemens Issuing CA Class Internet Server 2013 is already on OneCRL, added on Oct. 2, please refer to https://bugzilla.mozilla.org/show_bug.cgi?id=1400600 \Markus
Product: NSS → CA Program
Whiteboard: [ca-compliance] → [ca-compliance] [ov-misissuance]
You need to log in before you can comment on or make changes to this bug.