1974435 - eMudhra emSign PKI Services : Delayed Revocation of TLS Certificates due to Policy Inconsistency.

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Naveen Kumar ML]

Preliminary Incident Report

Summary

• Incident description:
  We issued publicly trusted TLS certificates with RSA key sizes exceeding 2048 bits. While these key sizes were technically compliant with the CA/Browser Forum Baseline Requirements (BR 6.1.5), which allow RSA key sizes ≥2048 bits, they did not align with the CA’s published CP/CPS, which explicitly referenced only "RSA 2048" in the end-entity certificate profile.
  On June 19, 2025, an external report highlighted this misalignment. The certificates were initially assessed as compliant with the Baseline Requirements and revocation was not immediately initiated given that any such revocation event introduces risk to the ecosystem. However, following feedback from the broader CA community on June 26, 2025 local time citing the applicability of BR 4.9.1.1(12), it was decided that revocation be initiated immediately to be on the side of being absolutely compliant with this section.
  The five-day revocation window, which began on June 19, 2025, was exceeded during this reassessment and revocation planning process. Revocation of all 449 unexpired certificates was completed on June 26, 2025.
  Internal guidance has since been updated to favor and act based on absolute interpretations of applicable sections of Baseline Requirements.
• Relevant policies:
  • CA/Browser Forum Baseline Requirements – Section 4.9.1.1(12): Revocation for non-compliance with CP/CPS
  • eMudhra CP/CPS v1.14 and v1.19 – Section 11, Appendix B: End-entity Certificate Profile
  • Internal CA issuance and revocation procedures
• Source of incident disclosure:
  The non-compliance was reported by an external researcher via email to [e-mail] on June 19, 2025. The issue was initially tracked under Bugzilla Bug 1973341, which addressed the policy documentation inconsistency. The delayed revocation due to this misalignment was later identified as a separate reportable issue under BR 4.9.1.1(12).

Comment 1

[incident-reporting]

Please be aware of the following in https://www.ccadb.org/cas/incident-report:

In the case of Incident Reports with a Whiteboard field of “revocation-delay”, reports SHOULD be updated every 3 days and MUST be updated no less frequently than every 7 days to describe a summary of:

the number of certificates that have been revoked;
the number of certificates that have not yet been revoked;
the number of certificates planned for revocation that have expired; and
an estimate for when all remaining revocations will be completed.

In the Analysis section: "[provide] the factors and rationales behind the decision to delay revocation (including detailed and substantiated explanations of how extensive harm would result to third parties–such as essential public services or widely relied-upon systems–and why the situation is exceptionally rare and unavoidable)."

In the Timeline section, provide: "The time(s) at which the CA Owner is expected to complete revocation of affected certificates [AND]
The time(s) at which the CA Owner actually completed revocation of affected certificates".

Also, "Action Items list MUST include steps reasonably calculated to prevent or reduce future revocation delays. Note that it is not sufficient for these action items to simply stop this incident, they MUST create additional protections to prevent future incidents."

Comment 2

[Naveen Kumar ML]

Full Incident Report

Summary

• CA Owner CCADB unique ID:
  A005678

• Incident description:
  We issued publicly trusted TLS certificates with RSA key sizes exceeding 2048 bits. While these key sizes were technically compliant with the CA/Browser Forum Baseline Requirements (BR 6.1.5), which allow RSA key sizes ≥2048 bits, they did not align with the CA’s published CP/CPS, which explicitly referenced only "RSA 2048" in the end-entity certificate profile.
  On June 19, 2025, an external report highlighted this misalignment. The certificates were initially assessed as compliant with the Baseline Requirements and revocation was not immediately initiated given that any such revocation event introduces risk to the ecosystem. However, following feedback, from the broader CA community on June 26th, 2025 local time, citing the applicability of BR 4.9.1.1(12), it was decided that revocation be initiated immediately to be on the side of being absolutely compliant with this section.
  The five-day revocation window, which began on June 19, 2025, was exceeded during this reassessment and revocation planning process. Revocation of all 449 unexpired certificates was completed on June 26, 2025.
  Internal guidance has since been updated to favor and act based on absolute interpretations of applicable sections of CP/CPS and Baseline Requirements.

• Timeline summary:

  • Non-compliance start date: June 19, 2025 12:06 UTC
  • Non-compliance identified date: June 25, 2025 19:44 UTC
  • Non-compliance end date: June 26, 2025 07:00 UTC

• Relevant policies:

  • CA/Browser Forum Baseline Requirements – Section 4.9.1.1(12): Revocation for non-compliance with CP/CPS
  • eMudhra CP/CPS v1.14 and v1.19 – Section 11, Appendix B: End-entity Certificate Profile
  • Internal CA issuance and revocation procedures

• Source of incident disclosure:
  The non-compliance was reported by an external researcher via email to [e-mail] on June 19, 2025. The issue was initially tracked under Bugzilla Bug 1973341, which addressed the policy documentation inconsistency. The delayed revocation due to this misalignment was later identified as a separate reportable issue under BR 4.9.1.1(12).

Impact

• Total number of certificates:
  449

• Total number of "remaining valid" certificates:
  0

• Affected certificate types:
  End-entity TLS certificates

• Incident heuristic:
  CP/CPS documentation misalignment

• Was issuance stopped in response to this incident, and why or why not?:
  No. The certificate met Baseline Requirements and internal minimum RSA key size practice. Issuance processes continued with 2048 bit keys, and the incident pertained to a policy documentation inconsistency rather than a technical control lapse.

• Analysis:
  At the time of initial assessment, the affected certificates were found to be technically compliant with BR 6.1.5, which allows RSA key sizes of 2048 bits or greater. Since no security weakness was introduced and the key sizes exceeded the required minimum, revocation was not immediately pursued.
  However, following further review and feedback from the broader CA community on June 26, 2025, it became evident that BR 4.9.1.1(12) was applicable, as the certificates had been issued in a manner not fully aligned with the CA’s published CP/CPS, which only specified "RSA 2048" under the end-entity profile. Once the applicability of this section was confirmed, the CA proceeded to revoke all 449 unexpired certificates immediately.
  The delay was a result of this period of reassessment and interpretation balancing the introduction of ecosystem risk with the obligation to remain consistent with published documentation. In hindsight, a more conservative approach would have favored early revocation while the policy gap was under evaluation. Internal guidance has now been updated to ensure revocation is prioritized in similar cases, even when technical compliance exists, if policy alignment is not there or is ambiguous.

• Additional considerations:
  All revoked certificates had key sizes ≥2048 bits and had passed Debian weak key screening.
  Some of the customers included Financial Institutions, eGovernment platforms who had service disruptions as a result of the revocation.

Timeline

(All Time in UTC)
2024-05-28 06:17 : Certificate issued with a 4048-bit RSA key.
2025-06-19 12:06 : External report received highlighting key size misalignment with CP/CPS.
2025-06-19 12:30 : Report acknowledged; internal review initiated
2025-06-19 15:30 : Certificate initially assessed as technically compliant with Baseline Requirements; revocation not initiated.
2025-06-20 06:30 : Response sent to reporter indicating plan to update CP/CPS documentation.
2025-06-21 10:33 : Preliminary incident report submitted under Bugzilla Bug 1973341 for CP/CPS documentation inconsistency.
2025-06-21 11:30 : Internal discussions on whether BR 4.9.1.1(12) applies; compliance and PKI teams begin reassessment.
2025-06-25 19:44 : Community feedback highlights revocation obligations under BR 4.9.1.1(12) despite BR compliance; consensus formed that delayed revocation applies.
2025-06-26 03:00 : Internal decision made to proceed with revocation due to CP/CPS non-conformance.
2025-06-26 07:00 : All 449 unexpired certificates revoked.
2025-06-27 12:39 : Preliminary incident report for delayed revocation submitted.

Related Incidents

Bug	Date	Description
1885568	2024-03-15	Delayed revocation of TLS certificates.
1896053	2024-05-09	Delayed revocation exceeding 5 days.

Root Cause Analysis

Contributing Factor 1: CP/CPS certificate profile wording gap

• Description:
  The end-entity certificate profile referenced only “RSA 2048” and did not clarify acceptance of larger key sizes. This led to a perceived inconsistency between the certificate issuance and the documented policy.
• Timeline:
  2023-08-30: CP/CPS v1.14 published
  2025-06-06: CP/CPS v1.19 published (issue persisted)
  2025-06-19: Issue reported externally
  2025-06-26: Revocation triggered upon community clarification
• Detection:
  External report
• Interaction with other factors:
  Lack of internal policy interpretation consistency and limited clarity in documentation
• Root Cause Analysis methodology used:
  5-Whys

Lessons Learned

• What went well:
  The incident was acknowledged promptly upon receipt of the report. Operations team acted quickly to revoke affected and unexpired certificates in a timely manner once the decision was taken to revoke. The team also worked in close coordination with the customers to reissue certificates quickly.
• What didn’t go well:
  The lack of specificity in the CP/CPS regarding acceptance of RSA key sizes beyond 2048 bits created a documentation gap. Initially, the team assessed the certificates as compliant based on technical grounds, without identifying the underlying policy misalignment as a trigger for mandatory revocation. This led to us seeking community feedback and inaction for a delayed period. The process of reconciling technical compliance with documentation obligations, and the absence of pre-established escalation procedures for such policy interpretation ambiguities, contributed to the delay.
• Where we got lucky:
  The certificates, while not explicitly authorized by the CP/CPS, did not pose any security risk, as all used strong RSA key sizes well above the minimum required threshold.
• Additional:
  The incident reinforces the need for exact alignment between technical practices and published policy.

Action Items

Action Item	Kind	Corresponding Root Cause(s)	Evaluation Criteria	Due Date	Status
Update CP/CPS to clarify RSA key sizes	Prevent	Root Cause # 1	Updated policy published	2025-07-15	In Progress
Improve internal guidance on revocation interpretation	Prevent	Root Cause # 1	Training completion and procedural update	2025-07-08	In Progress
Document and enforce revocation escalation SOP for ambiguous policy cases	Reduce Impact	Root Cause # 1	SOP documented and staff trained	2025-07-08	In Progress

Appendix

Affected certificates are published in the attachment.

Comment 3

[Naveen Kumar ML]

Attachment: Annexure - Cert List Revoked.xlsx

Comment 4

[Naveen Kumar ML]

Weekly Status Update

We’re actively working on the action items identified in the incident report and anticipated to be completed on as per the below table.

Action Item	Kind	Corresponding Root Cause(s)	Evaluation Criteria	Due Date	Status
Update CP/CPS to clarify RSA key sizes	Prevent	Root Cause # 1	Updated policy published	2025-07-15	In Progress
Improve internal guidance on revocation interpretation	Prevent	Root Cause # 1	Training completion and procedural update	2025-07-08	In Progress
Document and enforce revocation escalation SOP for ambiguous policy cases	Prevent	Root Cause # 1	SOP documented and staff trained	2025-07-08	In Progress

Comment 5

[Naveen Kumar ML]

Weekly Status Update

We are pleased to report that all action items identified in the incident report have been successfully completed as per the schedule. The details are as follows:

Action Item	Kind	Corresponding Root Cause(s)	Evaluation Criteria	Due Date	Status
Update CP/CPS to clarify RSA key sizes	Prevent	Root Cause #1	Updated policy published	2025-07-15	Completed (CP/CPS V1.20 Published on 2025-07-08
Improve internal guidance on revocation interpretation	Prevent	Root Cause #1	Training completed, and procedural update finalized	2025-07-08	Completed
Document and enforce revocation escalation SOP for ambiguous policy cases	Prevent	Root Cause #1	SOP documented, and staff trained	2025-07-08	Completed

All measures have been implemented to address the root cause effectively.

Comment 6

[Naveen Kumar ML]

Weekly Update

No further action required at this time.

Comment 7

[Naveen Kumar ML]

Weekly Update

No further action required at this time.

Comment 8

[Naveen Kumar ML]

Weekly Update

No further action required at this time.

Comment 9

[Naveen Kumar ML]

Report Closure Summary

• Incident description:
  eMudhra issued publicly trusted TLS certificates with RSA key sizes greater than 2048 bits, which complied with CA/Browser Forum Baseline Requirements (BR 6.1.5) but were not explicitly aligned with the CA’s CP/CPS, which referenced only "RSA 2048" in the end-entity certificate profile. This misalignment between documented policy and operational practice was reported externally on June 19, 2025. While initial assessment concluded no BR violation, broader CA community feedback on June 26, 2025, highlighted BR 4.9.1.1(12) as applicable, prompting a decision to revoke the affected certificates. Although this action occurred beyond the five-day BR revocation window, revocation of all 449 unexpired certificates was completed the same day. Internal procedures have since been updated to enforce stricter alignment between documented policies and operational interpretations in future revocation decisions.

• Incident Root Cause(s):
  The end-entity certificate profile within the CP/CPS referenced only “RSA 2048” without explicitly clarifying acceptance of larger RSA key sizes. This omission resulted in a perceived inconsistency between certificate issuance practices and documented policy.

• Remediation description:
  The CP/CPS has been updated to explicitly clarify the acceptable RSA key sizes for end-entity certificates, eliminating ambiguity and aligning documentation with established practices. Internal guidance on interpreting revocation requirements has been enhanced to ensure consistent and unambiguous application in accordance with the CA/Browser Forum Baseline Requirements. Additionally, a formal revocation escalation Standard Operating Procedure (SOP) has been documented and is now enforced to guide timely and policy-aligned decision-making in cases involving unclear or evolving policy interpretations.

• Commitment summary:
  All preventive actions disclosed in the incident report have been completed. These include updating the CP/CPS to explicitly clarify acceptable RSA key sizes for end-entity certificates, enhancing internal guidance on revocation interpretation, and implementing a formal revocation escalation Standard Operating Procedure (SOP) to ensure consistent handling of ambiguous policy scenarios. We respectfully request the closure of this incident.

All Action Items disclosed in this report have been completed as described, and we request its closure.

Comment 10

[Aaron Gable]

I don't believe that the root cause analysis presented in the full incident report (Comment #2) and in the report closure summary (Comment #9) appropriately addresses the incident described in this ticket. It seems to address the question of why the certificates were misissued (the topic of the other incident report), rather than addressing the question of why revocation was delayed.

The root cause analysis needs to be an analysis of why the CA's staff and procedures did not realize that the certificates needed to be revoked. There are many similar incidents in the last few years where CAs have had to revoke due to violations of their own CPS rather than violations of the BRs; the root cause analysis should examine why the CA still needed to have the requirement clarified.

Comment 11

[Naveen Kumar ML]

(In reply to Aaron Gable from comment #10)

I don't believe that the root cause analysis presented in the full incident report (Comment #2) and in the report closure summary (Comment #9) appropriately addresses the incident described in this ticket. It seems to address the question of why the certificates were misissued (the topic of the other incident report), rather than addressing the question of why revocation was delayed.

The root cause analysis needs to be an analysis of why the CA's staff and procedures did not realize that the certificates needed to be revoked. There are many similar incidents in the last few years where CAs have had to revoke due to violations of their own CPS rather than violations of the BRs; the root cause analysis should examine why the CA still needed to have the requirement clarified.

Thank you for your previous input. To better address the question relating to the internal process and the need to reach out to the community, we have now uploaded an updated Root Cause Analysis (RCA) that addresses the delay in revocation with greater clarity, focusing specifically on internal decision-making, why we sought to reach out to the community and escalation pathways.
Clarification on Internal Decision Process

1. Assessment of Precedents and Documentation Interpretation
   Upon receiving the report, our team initiated an internal assessment based on past precedents, applicable Baseline Requirements (BR), and our CP/CPS.
   While the certificates were compliant with BR 6.1.5 (which permits RSA key sizes ≥2048 bits), ambiguity arose as the certificate profile in section specifically referenced “RSA 2048,” while the section in CP/CPS Section 6.1.5 states that the minimum acceptable RSA key length is 2048 bits, without explicitly excluding longer key sizes.
   Additionally, one of the applicable sections, BR Section 2.2, states that “in the event of any inconsistency between this document and those Requirements, those Requirements take precedence over this document.”
   Based on this, there was an initial interpretation that BR compliance may take precedence over the narrower CP/CPS certificate profile. This contributed to the uncertainty over whether revocation was mandatory under BR 4.9.1.1(12), or if corrective action via CP/CPS updates would suffice.
2. Ambiguity in Interpretation and Impact on Relying Parties
   Given the potential operational and risk impact of revoking hundreds of certificates, including those used by financial and eGovernment platforms, the matter was escalated internally to our Policy Authority. Given the slight subjective interpretation our CP/CPS introduced it was suggested that the community be consulted for input.
3. Community Engagement and Prompt Action Upon Feedback
   Upon receiving feedback that BR 4.9.1.1(12) unequivocally applies in cases of CP/CPS non-conformance, we proceeded with revocation immediately.
4. Addressing the Root Cause More Clearly
   In an effort to address the root cause more clearly, we have updated the Root Cause analysis to reflect the facts below
   • A thorough internal review was conducted immediately becoming aware of the issue; the delay was not due to inaction.
   • The delay resulted from interpretative ambiguity regarding CP/CPS profile wording and BR applicability.
   • Internal discussions were focused on understanding obligations, and minimizing disruption.

Comment 12

[incident-reporting]

This is a final call for comments or questions on this Incident Report.

Otherwise, it will be closed on approximately 2025-08-18.