1391864 - Staat der Nederlandend / PKIoverheid: Non-BR-Compliant Certificate Issuance

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Kathleen Wilson]

** Staat der Nederlandend / PKIoverheid already responded in the mozilla.dev.security.policy forum about this incident, but it was requested that I still file this bug and ask the CA to respond here.

The following problems have been found in certificates issued by your CA, and reported in the mozilla.dev.security.policy forum. Direct links to those discussions are provided for your convenience.

To continue inclusion of your CA’s root certificates in Mozilla’s Root Store, you must respond in this bug to provide the following information:
1) How your CA first became aware of the problems listed below (e.g. via a Problem Report, via the discussion in mozilla.dev.security.policy, or via this Bugzilla Bug), and the date.
2) Prompt confirmation that your CA has stopped issuing TLS/SSL certificates with the problems listed below.
3) Complete list of certificates that your CA finds with each of the listed issues during the remediation process. The recommended way to handle this is to ensure each certificate is logged to CT and then attach a CSV file/spreadsheet of the fingerprints or crt.sh IDs, with one list per distinct problem.
4) Summary of the problematic certificates. For each problem listed below: number of certs, date first and last certs with that problem were issued.
5) Explanation about how and why the mistakes were made, and not caught and fixed earlier.
6) List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.
7) Regular updates to confirm when those steps have been completed.

Note Section 4.9.1.1 of the CA/Browser Forum’s Baseline Requirements, which states:
“The CA SHALL revoke a Certificate within 24 hours if one or more of the following occurs: …
9. The CA is made aware that the Certificate was not issued in accordance with these Requirements or the CA’s Certificate Policy or Certification Practice Statement; 
10. The CA determines that any of the information appearing in the Certificate is inaccurate or misleading; …
14. Revocation is required by the CA’s Certificate Policy and/or Certification Practice Statement; or 
15. The technical content or format of the Certificate presents an unacceptable risk to Application Software Suppliers or Relying Parties (e.g. the CA/Browser Forum might determine that a deprecated cryptographic/signature algorithm or key size presents an unacceptable risk and that such Certificates should be revoked and replaced by CAs within a given period of time).

However, it is not our intent to introduce additional problems by forcing the immediate revocation of certificates that are not BR compliant when they do not pose an urgent security concern. Therefore, we request that your CA perform careful analysis of the situation. If there is justification to not revoke the problematic certificates, then explain those reasons and provide a timeline for when the bulks of the certificates will expire or be revoked/replaced. 

We expect that your forthcoming audit statements will indicate the findings of these problems. If your CA will not be revoking the certificates within 24 hours in accordance with the BRs, then that will also need to be listed as a finding in your CA’s BR audit statement.

We expect that your CA will work with your auditor (and supervisory body, as appropriate) and the Root Store(s) that your CA participates in to ensure your analysis of the risk and plan of remediation is acceptable. If your CA will not be revoking the problematic certificates as required by the BRs, then we recommend that you also contact the other root programs that your CA participates in to acknowledge this non-compliance and discuss what expectations their Root Programs have with respect to these certificates.


The problems reported for your CA in the mozilla.dev.security.policy forum are as follows:

** Failure to respond within 24 hours after Problem Report submitted
https://groups.google.com/d/msg/mozilla.dev.security.policy/PrsDfS8AMEk/w2AMK81jAQAJ
The problems were reported via your CA’s Problem Reporting Mechanism as listed here:
https://ccadb-public.secure.force.com/mozilla/CAInformationReport
Therefore, if this is the first time you have received notice of the problem(s) listed below, please review and fix your CA’s Problem Reporting Mechanism to ensure that it will work the next time someone reports a problem like this. 

** Short / sequential-looking serial numbers
https://groups.google.com/d/msg/mozilla.dev.security.policy/vl5eq0PoJxY/uD-Li1w1BgAJ
https://groups.google.com/d/msg/mozilla.dev.security.policy/YequotPYLdc/J0K3lUyzBwAJ
RESOLUTION: https://groups.google.com/d/msg/mozilla.dev.security.policy/vl5eq0PoJxY/W1D4oZ__BwAJ

Comment 1

[Mark Janssen]

Hi All,

As stated earlier at the MSDP group, this was te timeline of the incident:


Timeline (all times are UTC): 

19 July 2017 00:27: A posting on mozilla.dev.security.policy stating a non-compliant certificate issued by DDY. 

https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/vl5eq0PoJxY 

20 July 2017 16:45: Mozilla (Gerv) notifies the Policy Authority (PA) PKIoverheid on non-compliant certificates from DDY 

20 July 2017 16:45: START INCIDENT 

20 July 2017 17:27: PA PKIoverheid starts investigating the issue and almost immediately raises an internal incident. 

21 July 2017 09:08: PA PKIoverheid issues DDY to postpone further issuing of certificates and requests an action plan from DDY how they will resolve the issue by revoking and reissuing all certificates involved. 

21 July 2017 09:50: DDY confirms postponing the issuing of certificates. 

21 July 2017 09:50: FURTHER CERTIFICATE ISSUING SUSPENDED 

24 July 2017 08:53: DDY delivers action plan including two newly generated and compliant test certificates as proof that they fixed the issue. 

24 July 2017 16:25: Based on the provided certificates the PA PKIoverheid requests DDY to start executing the action plan including the approval to restart issuing certificates. 

24 July 2017 16:25: ISSUING RESTARTED 

25 July 2017 14:37: DDY installs first production certificate on website (https://www.digidentity.eu/nl/home/) 

25 July 2017 14:37: DDY starts revoking and replacing certificates 

25 July 2017 21:20: PA PKIoverheid post a message on mozilla.dev.security.policy stating that DDY has proven to be able to generate compliant certificates and is allowed to restart the issuing of new (compliant) certificates. A link to the compliant new DDY certificates is included in this post as evidence. 

26 July 2017 17:40: PA PKIoverheid requests Mozilla to honor the request to extend the 
24-hour revocation time. 

27 July 2017 10:31: Mozilla grants the PA PKIoverheid the request to extend the revocation time (that is, Mozilla will accept an audit qualified by this event). 

27 July 2017 13:21: PA PKIoverheid requests the other Application Software Suppliers to agree with Mozilla. 

28 July 2017 12:54: PA PKIoverheid informs DDY that it requests an audit statement regarding the Ballot 164 incident. 

28 July 2017 12:54: PA PKIoverheid starts monitoring the resolving progress but keeps the incident open until all involved certificates are revoked or replaced. 
INCIDENT UNDER CONTROL 

+++

1) How your CA first became aware of the problems listed below (e.g. via a Problem Report, via the discussion in mozilla.dev.security.policy, or via this Bugzilla Bug), and the date.

See timeline at 20 July 2017 16:45. 

2) Prompt confirmation that your CA has stopped issuing TLS/SSL certificates with the problems listed below.

See timeline at 21 July 2017 09:50.  

3) Complete list of certificates that your CA finds with each of the listed issues during the remediation process. The recommended way to handle this is to ensure each certificate is logged to CT and then attach a CSV file/spreadsheet of the fingerprints or crt.sh IDs, with one list per distinct problem.

The list will be logged on 8/30/2017 at the latest. 

4) Summary of the problematic certificates. For each problem listed below: number of certs, date first and last certs with that problem were issued.

After investigation it emerged that a total of 777 certificates were issued from September 30th 2016 up to July 21 2017 that were not compliant with BR ballot 164 (https://cabforum.org/2016/07/08/ballot-164/) echoed by the same requirement in version 2.4 (Compliance date: February 28, 2017) from the Mozilla CA Certificate Policy. Digidentity (DDY) will revoke and replace these non-compliant certificates. This wil take place on or before 31 August 2017.

5) Explanation about how and why the mistakes were made, and not caught and fixed earlier.

DDY concluded wrongly that ballot 164 was not applicable for them since the use of sequential serial numbers is not a security risk when used in conjunction with the SHA-256 with RSA encryption certificate signing scheme.

Non-compliance with this requirements wasn’t noticed by the auditor because DDY didn’t include the specific requirement in their Statement of Applicability (reason: see the answer on question 2). ETSI EN 319 403 (which determines the requirements for conformity assessment bodies) is not clear about who determines the scope of an audit. The auditor’s interpretation was that the client (DDY) had to determine the scope of the audit (based on their Statement of Applicability). This will be mitigated for future audits with new measure 4.  

6) List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.

NEW MEASURES: 
The following measures are newly introduced to mitigate risks that have been identified during the evaluation of this incident. 
1. The implementation of Certificate Transparancy (CT) will reduce the detection-time of non-compliant certificates to hours instead of days/weeks/months. Implementation date is set on October 1st. 
2. The PA PKIoverheid will increase the rate of random certificate checks to weekly as long as CT has not been implemented. Effective immediately. 
3. After a ballot has been adopted by the CABForum the PA PKIoverheid will require of it TSP’s to show proof of conformance to the ballot before the effective date. If a TSP isn't able to show proof of conformance as of the effective date, certificate issuance will be suspended until the TSP can supply evidence to the contrary. Effective 09/01/2017. 
4. PA PKIoverheid will review and/or determine the scope for the TSP in question before any type of audit. Without clearance/approval of the PA PKIoverheid no audit or issuance of certificates will take place. Effective 09/01/2017. 
5. Research into possible implementation of (automated) Certlint checks in the issuance process. 

7) Regular updates to confirm when those steps have been completed.

We will update this bug on a regular basis. 


Best Regards,
Mark Janssen

Comment 2

[Ryan Sleevi]

As per the mailing list, I'm going to mark this Resolved.

The responses here are illustrative of the expectation of all CAs, and demonstrate a holistic awareness of the issues and mitigations. In particular, the New Measures proposed represent a comprehensive plan that attempt to address the systemic failures via a combination of both technical and non-technical controls, to ensure the appropriate transparency and visibility, and with further redundancy for compliance.

I'm going to mark this issue as Resolved, with the expectation of continued updates regarding the process on these timelines as proposed. Thanks for your thorough response to these issues.

Comment 3

[Mark Janssen]

Attachment: 20170901DDY_Certs_CRTSH.xlsx

Comment 4

[Mark Janssen]

As of now, 9/1/2017, all 777 certificates in question have been logged (list of crt.sh IDs and CNs attached). All of them have been revoked minus 18 certificates for the domain *.ind.nl. The certificates in question who are not revoked are for the Dutch Immigration Office and cannot be revoked without severe disturbance to their primary processes. As stated in my post in MSDP on July 25, our intent was to replace and revoke all certificates on or before August 31, with the disclaimer that this would be difficult because of the holiday season. We expect that the last few certificates will be replaced and revoked next week.

Today we also published new requirements for our TSPs with regard to the new measures (nr 3 & 4) that were announced in my MDSP post of August 11th and in this bug. These new requirements can be found at https://www.logius.nl/ondersteuning/pkioverheid/aansluiten-als-tsp/programma-van-eisen/actuele-wijzigingen/ (scroll down for English text). 
	
Please let me know if you have any questions. 

Best regards,
Mark