1931683 - eMudhra emSign PKI Services : Key Blocking Mechanism Fails to Validate Historical Public Key Reuse.

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Naveen Kumar ML]

Incident Report

Summary

The Google team identified an issue with eMudhra’s key blocking mechanism, implemented on September 20, 2024. The mechanism was designed to block certificate issuance for public keys with a history of revocation due to "Key Compromise." However, it only applied to certificates issued after deployment and did not account for previously issued certificates. The issue affected three certificates: two reissued within the same order using the same CSR after revocation of the original certificate, and one issued before revocation without properly flagging related certificates during subsequent revocation. The affected certificates have been revoked, and enhanced controls have been implemented across all issuance channels, including ACME, REST API, and the enrollment system. Additionally, eMudhra is adopting ACME ARI to enhance automation and improve renewal handling during revocation events.

Impact

Three certificates were issued using previously compromised public keys. Two certificates were reissued under the same order after revocation (first scenario), and one certificate was issued before revocation without proper linkage checks (second scenario). There is no evidence of misuse for any of these certificates. Customers with affected certificates were notified, and their certificates were revoked to mitigate potential risks.

Timeline

All times are IST.

2024-06-04 23:53:35: The initial certificate containing the public key (SHA-256: de39228476fc6bf74f5267488ff57454c0bd19b84174ea887c692e1c3808cb6a) was issued.
2024-07-01 07:19:14: The certificate was revoked with a “Key Compromise” reason code.
2024-07-08 08:03:06: A certificate containing the same public key was reissued under the same order reference, bypassing the key blocking mechanism.
2024-11-13 19:31:00: The Google team reported the issue, identifying the reuse of the compromised key in the reissued certificate.
2024-11-14 03:30:00: eMudhra completed an internal review, identified two additional certificates affected by the same issue, and raised an internal incident.
2024-11-14 05:15:00: eMudhra initiated the revocation process for the two identified certificates and notified the affected customers.

Root Cause Analysis

The issue arose due to gaps in the key blocking mechanism:

1. Lack of Retroactive Application: The key blocking mechanism applied only to certificates issued after its deployment and did not validate previously issued certificates.
2. Reissue Validation Gap: The system failed to validate public key revocation history in two distinct scenarios:
   * First Scenario: Two certificates were reissued within the same order using the same CSR after revocation of the original certificates. The key blocking mechanism did not validate the public key’s revocation history during the reissuance process within the same order and domain.
   * Second Scenario: One certificate was issued using the same CSR within the same order without revoking the original certificate. When the original certificate was later revoked, the mechanism failed to check and revoke the newly issued certificate associated with the same public key.

Lessons Learned

Verification mechanisms must validate public key reuse across all issuance scenarios, including reissuance within the same order reference.

Enhanced mechanisms must retroactively validate previously issued certificates when new controls are implemented.

While we can track certificates revoked with "Key Compromise" within our own systems, this incident highlights a broader industry challenge where certificates revoked with "Key Compromise" by other CAs remain invisible to us. We are open to discussions with the community to explore solutions, such as shared repositories or distributed ledgers, to address this issue collaboratively.

Automation tools like ACME ARI can improve communication and handling of revocations, reducing manual intervention and associated delays.

What went well

• The issue was contained to certificates issued within the same order reference, as existing controls for preventing the reuse of compromised keys across different orders and domains were effective. The Google team’s timely report allowed eMudhra to quickly address the issue and revoke the affected certificates, minimizing potential impact.

What didn't go well

• The key blocking mechanism did not retroactively validate certificates issued before its deployment, leaving a gap in coverage. Additionally, the absence of cross-reference checks for public key reuse within the same order allowed reissuance in scenarios where revocation history should have been validated.

Where we got lucky

• There is no evidence that the compromised certificates were misused before revocation. The Google team’s timely intervention ensured the issue was identified and resolved proactively, mitigating potential security risks.

Action Items

Action Item	Kind	Due Date
Enhance key blocking mechanism to validate public key reuse across all issuance scenarios, including retroactive application.	Prevent	2024-11-15
Add verification layers in the PKI validation and core certificate generation processes to revalidate public key history.	Mitigate	2024-12-08
Update the CP/CPS to explicitly define the enhanced key blocking mechanism and ensure compliance with Baseline Requirements.	Prevent	2024-11-30
Process implementation to conduct regular audits and retrospective analyses to identify and address gaps in issuance and revocation processes.	Corrective	2024-11-30

Appendix

Details of affected certificates

Below are impacted certificate that were revoked:
Scenario One
● https://crt.sh/?id=14343260000
● https://crt.sh/?id=14470656858
Scenario Two
● https://crt.sh/?id=13864882835

Remediation Steps:

1. All three affected certificates have been revoked, and impacted customers were notified.
2. The key blocking mechanism has been updated to retroactively include certificates issued before its deployment. Cross-reference checks now validate public key revocation history for certificates reissued within the same order reference.
3. Begin integrating ACME ARI (Automated Renewal Information) to streamline certificate lifecycle management and improve handling of revocations through automated notifications. This will enhance the system’s ability to proactively identify and address compromised keys.
4. The PKI validation system now performs a detailed check of public key revocation history before submission to the core certificate generation system. A final verification layer ensures that public key revocation history is revalidated during certificate generation in the core system before issuing the pre-certificate and proceeding with linting.
5. Updates are being made to the CP/CPS to define the enhanced key blocking mechanism and include support for ACME ARI adoption.

Based on Incident Reporting Template v. 2.0

Comment 1

[Ben Wilson]

Please provide updates on the status of your remediation efforts. By the way, updates are supposed to be filed on a weekly basis unless a "Next Update" has been set in the whiteboard.

Comment 2

[Ben Wilson]

I’d like to take a moment to highlight the importance of prompt and consistent communication regarding incidents. Root stores expect CAs to provide timely updates in Bugzilla to help ensure transparency and trust in the Web PKI ecosystem. We appreciate eMudhra’s efforts in this space and encourage continued diligence in responding to compliance matters. Delays in communication can slow the resolution of issues and may invite additional scrutiny from root programs. Proactive updates help mitigate risks and demonstrate a CA's commitment to best practices.

Could eMudhra please let us know what internal measures are being implemented to improve its responsiveness to incident reports?

We welcome any insights you can share.

Comment 3

[Naveen Kumar ML]

Please provide updates on the status of your remediation efforts. By the way, updates are supposed to be filed on a weekly basis unless a "Next Update" has been set in the whiteboard.

As part of our ongoing remediation efforts for the Key Blocking Mechanism Fails to Validate Historical Public Key Reuse incident, we have made quick progress in implementing corrective actions. Below is the status of our remediation efforts, along with completion timelines:
Remediation Description:
. The key blocking mechanism now applies to both newly issued and previously issued certificates. It ensures that public key revocation history is thoroughly validated, including for reissued certificates within the same order and across different orders. This prevents the issuance of any certificate using a public key that has been previously revoked for key compromise, regardless of when the original certificate was issued.
Enhanced controls are now in place across all issuance channels, including ACME, REST API, and the enrolment system. emSign has also adopted ACME ARI to improve automation and renewal handling during revocations.
The PKI validation system checks public key revocation history before certificate generation by referencing eMudhra’s internal revocation database, which maintains records of previously revoked certificates. This ensures that no certificate is issued using a public key that has been revoked for key compromise, whether within the same order or across different orders. Currently, this validation is based solely on eMudhra's revocation history, as there is no industry-wide centralized database for cross-CA key revocation tracking. A final verification step ensures proper enforcement before certificate issuance. Additionally, CP/CPS Update – Version 1.17 now explicitly defines the enhanced key blocking mechanism and its application scope.
Completed Remediation Actions

1. Enhanced Key Blocking Mechanism (Completed on 2024-11-15) – The system now checks public key history for revocation with "Key Compromise" across all issuance channels (ACME, REST API, and manual enrolment). This update ensures that no certificate is issued with a public key previously revoked for compromise.
2. Retroactive Public Key Validation (Completed on 2024-11-20) – We conducted a full scan of past certificate issuance data and validated historical public key reuse cases. Any certificate that should have been blocked under the updated mechanism has been flagged and addressed.
3. Implementation of ACME ARI (Automated Renewal Information) (Completed on 2025-01-07) – ACME ARI has been integrated to enhance automation for revocation handling and key reuse prevention, reducing manual intervention and improving security.
4. PKI Validation System Enhancements (Completed on 2024-12-06) – We have added additional verification layers into our P KI validation and core certificate generation system to check revocation history before issuing a certificate.
5. Updated CP/CPS (Completed on 2024-11-30) – The revised CP/CPS now explicitly defines our enhanced key blocking mechanism and ensures alignment with CA/B Forum Baseline Requirements.

Comment 4

[Naveen Kumar ML]

No action items are pending and no further action

Comment 5

[Naveen Kumar ML]

No action items are pending and no further action.

Comment 6

[Naveen Kumar ML]

No action items are pending and no further action.

Comment 7

[Naveen Kumar ML]

No action items are pending and no further action.

Comment 8

[Ben Wilson]

If there are no action items remaining, and you believe that this case can be closed, then please submit a Closure Summary:
https://www.ccadb.org/cas/incident-report#how-are-reports-closed
https://www.ccadb.org/cas/incident-report#closure-report
https://www.ccadb.org/cas/incident-report#incident-closure-summary
Thanks,
Ben

Comment 9

[Naveen Kumar ML]

Report Closure Summary

• Incident description:
  On September 20, 2024, eMudhra implemented a key blocking mechanism to prevent certificate issuance for public keys previously revoked due to "Key Compromise." However, Google identified an issue where this mechanism only applied to certificates issued after deployment of the mechanism, failing to account for previously issued certificates.
  As a result, three certificates were affected:

1. Two certificates were reissued within the same order using the same CSR after the original certificate was revoked.
2. One certificate was issued before the revocation process but was not properly flagged during subsequent revocation handling.

• Incident Root Cause(s):
  Lack of Retroactive Validation – The key blocking mechanism was only applied to new certificate issuances post-deployment, leaving previously issued certificates unvalidated.
  Reissue Validation Gap – The system did not properly check public key revocation history in two cases:
  Case1: Two certificates were reissued within the same order using the same CSR after the original certificates were revoked, but the mechanism failed to flag the compromised public key.
  Case2: One certificate was issued before the original was revoked, but the system did not retroactively check and revoke the newly issued certificate when the original key was later flagged as compromised.

• Remediation description:

1. The key blocking mechanism now applies to both newly issued and previously issued certificates. It ensures that public key revocation history is thoroughly validated, including for reissued certificates within the same order and across different orders. This prevents the issuance of any certificate using a public key that has been previously revoked for key compromise, regardless of when the original certificate was issued.
2. Enhanced controls are now in place across all issuance channels, including ACME, REST API, and the enrolment system. emSign has also adopted ACME ARI to improve automation and renewal handling during revocations.
   The PKI validation system checks public key revocation history before certificate generation by referencing eMudhra’s internal revocation database, which maintains records of previously revoked certificates. This ensures that no certificate is issued using a public key that has been revoked for key compromise, whether within the same order or across different orders.
3. A final verification step ensures proper enforcement before certificate issuance.

• Commitment summary:
  eMudhra is committed to maintaining the highest security standards and ensuring compliance with industry best practices. To address the identified issue, we have implemented a robust key blocking mechanism that now applies retroactively and proactively to all issuance channels. Additionally, we have enhanced our PKI validation system, integrated ACME ARI for automated revocation handling.

All Action Items disclosed in this report have been completed as described, and we request its closure.

Comment 10

[Ben Wilson]

Last call. I'll pull this up for a review and closure later next week (Wed-Fri). Please provide any additional concerns or questions before then.