Open Bug 1900654 Opened 1 month ago Updated 14 days ago

Certigna: ARL without reasoncode for recent revoked CA certificates

Categories

(CA Program :: CA Certificate Compliance, task)

Product:
CA Program
Component:
CA Certificate Compliance
Type:
task

Tracking

(Not tracked)

Status:
ASSIGNED

People

(Reporter: j.allemandou, Assigned: j.allemandou)

Details

(Whiteboard: [ca-compliance] [crl-failure] [external])

Incident Report

ARL without reasoncode for recent revoked CA certificates

Summary

In March 2024, CERTIGNA generates new intermediate CA certificates to enable automation of certificate issuance. It was decided to change the profiles of these certificates and they were therefore revoked and replaced with new ones. However, the reason for revocation was not positioned in the ARL produced after their revocation.

Impact

No impact on end-entity certificates: No certificates was issued by these CA certificates.

Non-compliance with the Baseline Requirements which specified:

  • Section 1.2.2: 2020-09-30, 7.2 and 7.3 All OCSP and CRL responses for Subordinate CA Certificates MUST include a meaningful reason code.
  • Section 7.2.2: CRL Entry Extension: MUST be present unless the CRL entry is for a Certificate not technically capable of causing issuance.

Timeline

All times are UTC.

2024-06-04: Receiving and processing the alert

  • 00:39 Reception of a message from a third party informing us that records in our ARL are non-compliant with the Baseline Requirements for TLS server certificates.
  • 09:00 Message qualification and incident analysis by the security/compliance team.
  • 09:30 Notification of the incident to all employees involved in issuing certificates.
  • 09:45 Planning a key ceremony for the generation of new ARLs.
  • 10:00 Diagnosis and control of ARL:
    • Checks of all requirements applicable to ARL generation.
    • Checks of all records impacted by the non-conformity.
  • 11:30 Review of the “ARL generation procedure”
  • 14:30 Validation of the new version of the “ARL generation procedure”.
  • 15:00 Key ceremony for ARL generation.
    • 15:30: Generation of new ARLs with reason code for the affected records.
    • 15:48: Publication of new ARLs.
  • 16:45 Notification to the Supervisory body (ANSSI) and the Assessment body (LSTI).

Root Cause Analysis

Unlike operations involving final certificates based on automated practices and tested before deployment, the generation of ARLs relies on the keeping of a Key Ceremony carried out manually. This operation is based on an “ARL generation Procedure” that has not been used since March 2020 to revoke a CA certificate and has not been updated to consider this requirement linked to the personalization of the revocation reason.

Lessons Learned

Our ARLs are not pre-generated, and are generated each year during a key ceremony in the presence of the secret holders, to ensure that the process is carried out correctly. The check of records before and after ARL generation did not explicitly include verification of the presence of the revocation reason.

Our “ARL generation procedure” requires more details to ensure that all applicable requirements are met during this operation.

Action Items

| Review of all applicable requirements to ensure the completeness of the guidelines and controls described in the “ARL generation procedure”. | Prevent | 2024-06-06 |
| Planning of a periodic review of the ARL generation procedure to verify compliance with applicable requirements. | Prevent | 2024-06-06 |
| Update of the compliance management procedure to stipulate that any new requirement that affects a manual process such as ARL management implies an update of the associated operational directives and contributors’ awareness. | Prevent | 2024-06-06 |

Details of affected certificates

No affected end-entity affected certificates.
No affected valid intermediate certificates.

Based on Incident Reporting Template v. 2.0

Summary: ARL without reasoncode for recent revoked CA certificates → Certigna: ARL without reasoncode for recent revoked CA certificates

(In reply to Josselin Allemandou from comment #0)

Impact

No impact on end-entity certificates: No certificates was issued by these CA certificates.

The impact section is for describing what and how many things were affected. How many intermediate certs were created and then revoked? Are they on crt.sh? The full certificate details weren't provided as required.

The timeline is also incomplete. It should include events from before the issue was reported. The most basic events are missing. When were these certificates created? When were they revoked?

(In reply to Mathew Hodson from comment #1)
Hello Mathew, below you will find the information regarding the CA certificates designated in the ARLs.

Details of affected certificates

No affected end-entity affected certificates.
No affected valid intermediate certificates.

4 revoked intermediate CA certificates were concerned:
• CN = Certigna Server Authentication ACME CA | crt.sh ID = 12396132906
• CN = Certigna Server Authentication ACME CA 2024 | crt.sh ID = 12396132907
• CN = Certigna Server Authentication ACME FR CA | crt.sh ID = 12396132908
• CN = Certigna Server Authentication ACME FR CA 2024 | crt.sh ID = 12396132909
These CA certificates were not used, were generated on March 13, 2024 and revoked on March 26, 2024.

Type: defect → task
Whiteboard: [ca-compliance] [crl-failure] [external]

Community member comments in Comment 1 are similar in nature to incident reporting expectations reminders shared in response to https://bugzilla.mozilla.org/show_bug.cgi?id=1883416 (i.e., Comment 1). Consequently, this report falls short of expectations.

Opportunities for improvement:

  • The timeline is incomplete.
  • There are opportunities for formatting improvements (i.e., table Markdown is broken).
  • The reporting template was not followed (e.g., there’s no “What didn’t go well?” list).
  • Root cause is, again, interpreted as “human error", which is considered insufficient.
  • The remediation defaults to additional human processing and manual processes to prevent this issue from repeating. This is not considered a complete or systemic solution, especially considering over-reliance on humans was a contributing factor for this incident. For example, are there opportunities for linting, possibly to include use in a test environment, to prevent future profile non-conformance?

Also, can you help us understand why reminders shared in https://bugzilla.mozilla.org/show_bug.cgi?id=1883416 were not considered when responding to this incident as a demonstration of continuous improvement?

Flags: needinfo?(j.allemandou)
Assignee: nobody → j.allemandou

Thank you for your feedback and suggestions for improvement. You will find below additional information.

Timeline

2024-02-06: Generation of our annual ARL without new CA certificates ;
2024-03-13 : Four new intermediate CA certificates are generated :

  • CN = Certigna Server Authentication ACME CA | crt.sh ID = 12396132906
  • CN = Certigna Server Authentication ACME CA 2024 | crt.sh ID = 12396132907
  • CN = Certigna Server Authentication ACME FR CA | crt.sh ID = 12396132908
  • CN = Certigna Server Authentication ACME FR CA 2024 | crt.sh ID = 12396132909

2024-03-26 : We decided to revoke these certificates to replace them. The certificates are revoked and replace during an Key ceremony. No end-entity certificates were issued by these authorities.

2024-06-04: Receiving and processing the alerts

  • 00:39 Reception of a message from a third party informing us that records in our ARL are non-compliant with the Baseline Requirements for TLS server certificates.
  • 09:00 Message qualification and incident analysis by the security/compliance team.
  • 09:30 Notification of the incident to all employees involved in issuing certificates.
  • 09:45 Planning a key ceremony for the generation of new ARLs.
  • 10:00 Diagnosis and control of ARL:
    • Checks of all requirements applicable to ARL generation.
    • Checks of all records impacted by the non-conformity.
  • 11:30 Review of the “ARL generation procedure”
  • 14:30 Validation of the new version of the “ARL generation procedure”.
  • 15:00 Key ceremony for ARL generation.
    • 15:30: Generation of new ARLs with reason code for the affected records.
    • 15:48: Publication of new ARLs.
  • 16:45 Notification to the Supervisory body (ANSSI) and the Assessment body (LSTI).

2024-06-06 : Review of requirements for ARL generation and update of our “ARL Generation Procedure” and planification of an annual review of this document;
2024-06-06 : Update of our “Compliance Management Procedure” with requirements on procedure to stipulate that any new requirement that affects a manual process such as ARL management implies an update of the associated operational directives and contributors’ awareness.

What went well

Although the operation is currently based on a manual process, the emergency LAR generation was successfully completed.

What didn't go well

The checks carried out after the issuance of a LAR did not include verification of the Reason code field for new revoked CA certificates.

Where we got lucky

This non-compliance did not impact any final certificates.

Action Items

Action Item Kind Due Date
Review of all applicable requirements to ensure the completeness of the guidelines and controls described in the “ARL generation procedure”. Prevent 2024-06-06
Planning of a periodic review of the ARL generation procedure to verify compliance with applicable requirements. Prevent 2024-06-06
Update of the compliance management procedure to stipulate that any new requirement that affects a manual process such as ARL management implies an update of the associated operational directives and contributors’ awareness. Prevent 2024-06-06
Study for the definition of a script to check the compliance of the generated LAR profile. Prevent 2024-07-30
Flags: needinfo?(j.allemandou)
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
You need to log in before you can comment on or make changes to this bug.