Closed Bug 2058294 Opened 1 month ago Closed 6 days ago

SDAIA: Delayed Revocation related to Bugzilla #2056942

Categories

(CA Program :: CA Certificate Compliance, task)

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: Asofyani, Assigned: Asofyani)

Details

(Whiteboard: [ca-compliance] [leaf-revocation-delay])

Attachments

(2 files, 2 obsolete files)

Preliminary Incident Report

Summary

  • Incident description: Following the audit-coverage gap disclosed in Bug 2056942 (SDAIA: Missing S/MIME WebTrust audit coverage, affecting the Saudi National Root CA and Government CA 2), SDAIA identified that the S/MIME certificates issued under this hierarchy without the required audit coverage must be revoked.

    Of the affected population, 839 certificates have been revoked to date. 172 certificates belonging to 21 government agencies have not yet been revoked. These certificates may still be actively used by government entities for critical services/processes, and immediate revocation is assessed as carrying a material risk of operational/financial impact to the affected organizations.

    SDAIA is therefore unable to complete revocation of the remaining certificates within the timeline required (i.e., 5 days) by the S/MIME Baseline Requirements and is filing this preliminary report to disclose the delay. SDAIA is continuously monitoring and following-up with the affected organizations on the potential impact of revocation and possible mitigation activities.

  • Impact and rationale for delayed revocation

    • Most certificates were revoked in due time. For a limited subset of certificates, which may be supporting critical governmental services, the revocation must be delayed to allow coordination of transition/mitigation.

    SDAIA consider the associated risks to the be limited because:

    • Certificate issuance was tightly controlled through an Enterprise RA model with two-step approval, reducing the risk of unauthorized issuance
    • The number of affected certificates is limited. Several certificates have been issued as part of 3-certificate bundles (1 client authentication certificate, and 2 certificates with S/MIME EKU). This means only 105 subscribers are affected.
    • There is no indication that any certificates have been compromised or misused.
  • Relevant policies:

    • CA/Browser Forum S/MIME Baseline Requirements, Section 4.9.1 (Reasons for Revoking a Subscriber Certificate);
    • Mozilla Root Store Policy, Section 6.2 (S/MIME) and Section 2.3 (Baseline Requirements Conformance);
    • CCADB Incident Reporting Guidelines v3.2.
  • Source of incident disclosure: Self Reported.

Assignee: nobody → Asofyani
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Whiteboard: [ca-compliance] [leaf-revocation-delay]

Status Update

We have continued the revocation of the affected S/MIME certificates. The remaining number of certificates to be revoked is 45.
The attached tracking list includes the current revocation status of each affected certificate together with supporting information (as required by the Incident Reporting Guidelines).
Subscriber personal information, associated with S/MIME certificates, has not been included in this public incident report.
For the remaining certificates subject to delayed revocation, the respective Enterprise RAs have confirmed that these certificates are currently used by critical internal services and that immediate revocation would result in significant operational impact. We are actively coordinating with the affected subscribers to replace these certificates before revocation. Due to the complexity of the subscriber environments and the required change coordination, this process requires additional time.
The latest expected date for revocation is 2026-08-16. It was confirmed that these certificates are only used for internal purposes.
We will continue to update this incident as revocation activities progress and will keep the incident open until all remaining affected certificates have been revoked.

Full Incident Report

Summary

  • CA Owner CCADB unique ID: A011197

  • Incident description: Delayed revocation of a subset of publicly trusted S/MIME certificates that could not be revoked within the applicable revocation timeline following the incident reported under Bugzilla 2056942.

  • Timeline summary:

    • Non-compliance start date: 2026-07-25
    • Non-compliance identified date: 2026-07-25
    • Non-compliance end date: Ongoing
  • Relevant policies:

    • CA/Browser Forum S/MIME Baseline Requirements, Section 4.9.1 (Reasons for Revoking a Subscriber Certificate)
    • CCADB Incident Reporting Guidelines v3.2.
  • Source of incident disclosure: CA Self-Reported

Impact

  • Total number of certificates: 331 (as of 2026-07-25)

  • Total number of "remaining valid" certificates: 45

  • Affected certificate types: S/MIME subscriber certificates issued under Government CA 2

  • Incident heuristic: All certificates identified in the Appendix are affected by this incident.

  • Was issuance stopped in response to this incident, and why or why not?:

    • Yes. Issuance and renewal of S/MIME certificates under the affected hierarchy were suspended immediately following discovery of the original incident.
  • Analysis:
    The affected certificates could not all be revoked within the applicable revocation timeline because immediate revocation would have caused significant operational impact to a limited number of government entities.

    SDAIA engaged individually with each affected Enterprise RAs to assess operational dependencies. The remaining certificates were confirmed by the respective Enterprise RAs to be supporting critical internal processes/services. Immediate revocation may have resulted in significant operational impact before replacement certificates could be deployed.

    Several factors were considered before delaying revocation:

    • The remaining population represents a small subset of the overall affected certificates.
    • The certificates were issued through an Enterprise RA model with two-step approval, reducing the risk of unauthorized issuance.
    • The certificates are used exclusively for internal government services/processes. The remaining certificates belong to 2 government entities only.

    The delay is expected to remain limited to the minimum period necessary to complete certificate replacement and coordinate revocation.

  • Additional considerations: None

Timeline

All times are GMT+3 unless otherwise noted.

  • 2026-07-20 - 12:14 PM - Internal compliance incident raised and incident response team established
  • 2026-07-22 - between 2:30 PM & 5:03 PM - Communications to Enterprise RA informing about the upcoming revocation of S/MIME certificates
  • 2026-07-22 - between 6:06 PM & 6:35 PM - First batch of certificate revocation (499)
  • 2026-07-23 - between 8:01 PM & 9:57 PM - Second batch of certificate revocation (115)
  • 2026-07-26 - between 1:40 PM & 5:07 PM - Third batch of certificate revocation (42)
  • 2026-07-27 - 11:00 PM - Public disclosure of the preliminary incident report on Bugzilla for Delayed revocation
  • 2026-07-27 - between 3:37 PM & 5:11 PM - Fourth batch of certificate revocation (114)
  • 2026-07-28 - between 8:21 PM & 10:01 PM - Fifth batch of certificate revocation (46)
  • 2026-07-29 - between 11:56 AM & 5:32 PM - Sixth batch of certificate revocation (34)
  • 2026-07-30 - between 4:46 PM & 6:25 PM - Seventh batch of certificate revocation (48)

Related Incidents

Bug Date Description
Bugzilla 2056942 2026-07-22 SDAIA: Missing S/MIME WebTrust audit coverage

Root Cause Analysis

This incident does not introduce new root causes or contributing factors. The delayed revocation is a direct consequence of the operational impact associated with revoking certificates identified in the original incident.

Lessons Learned

What went well

  • Most certificates were revoked within the planned timeline.
  • Certificate replacement activities began immediately after subscriber confirmation.

What didn't go well

  • Several critical internal services required additional coordination for certificate replacement.
  • Replacement requires more time than initially anticipated due to the complexity of the affected environments.

Where we got lucky

  • The remaining certificates represent a limited subset of the affected population.
  • The certificates are restricted to controlled government environments and are not used for public-facing services.

Additional

  • SDAIA will continue reviewing subscriber transition planning to reduce future delayed revocations.

Action Items

# Action Item Kind Root Cause Evaluation Criteria Due Date Status
1 Revoke all affected S/MIME certificates and notify impacted subscribers. Mitigate N/A All affected certificates revoked. 2026-08-16 Ongoing

Appendix

The tracking list will be updated as new certificate are revoked.

The lack of Root Cause analysis and treating this as a non-incident is alarming.

The purpose of a Delayed Revocation incident is to explain why you missed the baseline requirements deadline, and to show the steps you have taken to ensure it will not happen again. None of that it here.

If your problem has been issuance to subscribers who should be on a private root, then put that forward as your plan. That this is taking weeks to resolve is a different, and more serious matter.

Your entire analysis is flawed as well, you have 5 days upon being made aware to revoke every certificate. You have a huge chunk of certificates revoked outside of this window that are not in this incident. The timeline claims first awareness internally was 2026-07-20 - 12:14 PM, but there are 286 certificates revoked on 2026-07-26 or afterwards in your own data. Those are part of this delayed revocation incident.

Try to re-read the CCADB Reporting Guidelines and ensure you are compliant going forward.

(In reply to Wayne from comment #5)

The lack of Root Cause analysis and treating this as a non-incident is alarming.

The purpose of a Delayed Revocation incident is to explain why you missed the baseline requirements deadline, and to show the steps you have taken to ensure it will not happen again. None of that it here.

If your problem has been issuance to subscribers who should be on a private root, then put that forward as your plan. That this is taking weeks to resolve is a different, and more serious matter.

Your entire analysis is flawed as well, you have 5 days upon being made aware to revoke every certificate. You have a huge chunk of certificates revoked outside of this window that are not in this incident. The timeline claims first awareness internally was 2026-07-20 - 12:14 PM, but there are 286 certificates revoked on 2026-07-26 or afterwards in your own data. Those are part of this delayed revocation incident.

Try to re-read the CCADB Reporting Guidelines and ensure you are compliant going forward.

Thank you for your feedback.

We acknowledge your comments and agree that this (any) delayed revocation incident should contain its own root cause analysis and corrective actions. It was not our intention to treat this as a non-incident or to minimize the importance of missing the applicable revocation deadline.

The delayed revocation was not the result of an operational failure of the CA but resulted from a risk-based (as per the CCADB incident report guidelines - impact analysis for revocation delay) decision to avoid significant operational impact while affected certificates are being replaced. We recognize that this rationale and the associated preventive measures should have been better explained in the incident report. We will update this incident to clearly explain why the revocation deadline could not be met, the factors that contributed to the delay, and the measures taken to prevent a similar situation in the future.

Regarding the affected population, the scope of this incident already includes all certificates (286) revoked after the applicable revocation deadline, including those that have not been revoked (45). The reported population (331) is therefore not limited to the certificates that remain active today. We will ensure that the root cause analysis extends to the already revoked (but delayed) certificates.

Full Incident Report (Updated)

This version of the full incident report now includes the updated root cause analysis and action items. This also includes clarifications under the impact/analysis section regarding the affected vs remaining certificates.

Summary

  • CA Owner CCADB unique ID: A011197

  • Incident description: Delayed revocation of a subset of publicly trusted S/MIME certificates that could not be revoked within the applicable revocation timeline following the incident reported under Bugzilla 2056942.

  • Timeline summary:

    • Non-compliance start date: 2026-07-25
    • Non-compliance identified date: 2026-07-25
    • Non-compliance end date: Ongoing
  • Relevant policies:

    • CA/Browser Forum S/MIME Baseline Requirements, Section 4.9.1 (Reasons for Revoking a Subscriber Certificate)
    • CCADB Incident Reporting Guidelines v3.2.
  • Source of incident disclosure: CA Self-Reported

Impact

  • Total number of certificates: 331 (as of 2026-07-25)

  • Total number of "remaining valid" certificates: 45

  • Affected certificate types: S/MIME subscriber certificates issued under Government CA 2

  • Incident heuristic: All certificates identified in the Appendix are affected by this incident.

  • Was issuance stopped in response to this incident, and why or why not?:

    • Yes. Issuance and renewal of S/MIME certificates under the affected hierarchy were suspended immediately following discovery of the original incident.
  • Analysis:
    The affected certificates could not all be revoked within the applicable revocation timeline because immediate revocation would have caused significant operational impact to a limited number of government entities.

    SDAIA engaged individually with each affected Enterprise RAs to assess operational dependencies. The affected certificates (331) were confirmed by the respective Enterprise RAs to be supporting critical internal processes/services and that immediate revocation may have resulted in significant operational impact before replacement certificates could be deployed.

    Several factors were considered before delaying revocation:

    • The remaining population represents a small subset of the overall affected certificates.
    • The certificates were issued through an Enterprise RA model with two-step approval, reducing the risk of unauthorized issuance.
    • The certificates are used exclusively for internal government services/processes. The "remaining active" certificates (45) belong to 2 government entities only.

    The delay is expected to remain limited to the minimum period necessary to complete certificate replacement and coordinate revocation.

  • Additional considerations: None

Timeline

All times are GMT+3 unless otherwise noted.

  • 2026-07-20 - 12:14 PM - Internal compliance incident raised and incident response team established
  • 2026-07-22 - between 2:30 PM & 5:03 PM - Communications to Enterprise RA informing about the upcoming revocation of S/MIME certificates
  • 2026-07-22 - between 6:06 PM & 6:35 PM - First batch of certificate revocation (499)
  • 2026-07-23 - between 8:01 PM & 9:57 PM - Second batch of certificate revocation (115)
  • 2026-07-26 - between 1:40 PM & 5:07 PM - Third batch of certificate revocation (42)
  • 2026-07-27 - 11:00 PM - Public disclosure of the preliminary incident report on Bugzilla for Delayed revocation
  • 2026-07-27 - between 3:37 PM & 5:11 PM - Fourth batch of certificate revocation (114)
  • 2026-07-28 - between 8:21 PM & 10:01 PM - Fifth batch of certificate revocation (46)
  • 2026-07-29 - between 11:56 AM & 5:32 PM - Sixth batch of certificate revocation (34)
  • 2026-07-30 - between 4:46 PM & 6:25 PM - Seventh batch of certificate revocation (48)

Related Incidents

Bug Date Description
Bugzilla 2056942 2026-07-22 SDAIA: Missing S/MIME WebTrust audit coverage

Root Cause Analysis

SDAIA identified one primary root cause supported by three contributing factors that collectively resulted in the delayed revocation.

The delayed revocation resulted from a risk-based decision to avoid significant operational impact while affected certificates are being replaced. A subset of the affected S/MIME certificates supported critical internal government services. Immediate revocation within the applicable Baseline Requirements timeline would have disrupted those services before replacement certificates could be completed.

SDAIA therefore coordinated certificate replacement (where necessary) with affected enteprise RAs. Certificates were revoked as soon as replacement certificates had been deployed and/or the subscriber confirmed no potential impact on their services if certificates were revoked. This applies for the 331 certificates affected by the delayed revocation.

As part of the original incident, SDAIA has stopped issuing S/MIME certificates under the affected hierarchy and is proceeding with the removal of the S/MIME trust bit with Microsoft. Once completed, certificates intended solely for internal government use will no longer be issued from a publicly trusted S/MIME hierarchy, preventing similar delayed revocation scenarios from occurring in the future.

Contributing Factor 1: Public trust configuration and subcriber deployment model

  • Description:
    The affected certificates were deployed within critical internal government services. Because the certificates originated from a publicly trusted S/MIME hierarchy, revocation became subject to public trust S/MIME timelines that were not aligned with the operational constraints of these environments.

    This factor established the conditions that made the delayed revocation necessary. Without the hierarchy publicly trusted for S/MIME, the public trust revocation timelines would not have applied.

  • Timeline:

    • 2012-04-29: multi-purpose CA hierarchy established.
    • 2021-08-24: TLS trust removed from the hierarchy; S/MIME trust configuration remained unchanged from this point.
    • 2023-09-15: S/MIME Baseline Requirements introduced (effective date).
    • 2023-09-01: Microsoft Root Program audit requirements for S/MIME took effect
  • Detection:
    The condition was identified during the investigation of the original incident. Prior to that, the certificates were considered part of an internal government service and the implications of the public trust configuration were not fully recognized.

  • Interaction with other factors: N/A

  • Root Cause Analysis methodology used: 5 Whys

Contributing Factor 2: Enterprise RA coordination and impact confirmation

  • Description:
    Following discovery of the original incident, SDAIA coordinated with each Enterprise RA to identify operational dependencies and plan certificate replacement before revocation.

  • Timeline:

    • 2026-07-22 - between 2:30 PM & 5:03 PM - Communications to Enterprise RA informing about the upcoming revocation of S/MIME certificates
    • Between 2026-07-22 - Now - Daily communications with remaining affected subscribers to support transition and certificate replacement.
  • Detection:
    The need for coordinated replacement became apparent once Enterprise RAs confirmed that some certificates supported critical internal services. Earlier engagement may have reduced the number of delayed revocations; however, several certificates would still have required additional transition time.

  • Interaction with other factors: CF3

  • Root Cause Analysis methodology used: 5 Whys

Contributing Factor 3: Subscriber certificate replacement complexity

  • Description:
    Some subscriber environments required coordinated changes across multiple systems before replacement certificates could be deployed and the affected certificates revoked.

  • Timeline:

    • 2026-07-22 - between 2:30 PM & 5:03 PM - Communications to Enterprise RA informing about the upcoming revocation of S/MIME certificates
    • Between 2026-07-22 - Now - Daily communications with remaining affected subscriber to support transition and certificate replacement.
  • Detection: Operational dependencies could only be fully identified through discussions with the affected Enterprise RAs during the transition planning.

  • Interaction with other factors: CF2 (This factor extended the delay required before revocation could safely occur)

  • Root Cause Analysis methodology used: 5 Whys

Action Items

# Action Item Kind Root Cause Evaluation Criteria Due Date Status
1 Remove the S/MIME trust capability from the publicly trusted hierarchy - Microsoft Root Program Action Prevent CF1 Root Program trust configuration updated. 2026-08-27 Ongoing
2 Revoke all affected S/MIME certificates and notify impacted subscribers. Mitigate RC All affected certificates revoked. 2026-08-16 Ongoing
3 Continuous support with Enterprise RA/Subscribers to replace remaining affected certificates Mitigate CF2,CF3 Confirmation to revoke from affected subsribers 2026-08-16 Ongoing
4 Stop issuance of S/MIME certificates under affected CA hierarchy Prevent CF1 S/MIME issuance no longer possible 2026-07-20 Complete

Appendix

The tracking list will be updated as new certificate are revoked.

lesson learned were missing in the perviously updated incident report.

Full Incident Report (Updated)

This version of the full incident report now includes the updated root cause analysis and action items. This also includes clarifications under the impact/analysis section regarding the affected vs remaining certificates.

Summary

  • CA Owner CCADB unique ID: A011197

  • Incident description: Delayed revocation of a subset of publicly trusted S/MIME certificates that could not be revoked within the applicable revocation timeline following the incident reported under Bugzilla 2056942.

  • Timeline summary:

    • Non-compliance start date: 2026-07-25
    • Non-compliance identified date: 2026-07-25
    • Non-compliance end date: Ongoing
  • Relevant policies:

    • CA/Browser Forum S/MIME Baseline Requirements, Section 4.9.1 (Reasons for Revoking a Subscriber Certificate)
    • CCADB Incident Reporting Guidelines v3.2.
  • Source of incident disclosure: CA Self-Reported

Impact

  • Total number of certificates: 331 (as of 2026-07-25)

  • Total number of "remaining valid" certificates: 45

  • Affected certificate types: S/MIME subscriber certificates issued under Government CA 2

  • Incident heuristic: All certificates identified in the Appendix are affected by this incident.

  • Was issuance stopped in response to this incident, and why or why not?:

    • Yes. Issuance and renewal of S/MIME certificates under the affected hierarchy were suspended immediately following discovery of the original incident.
  • Analysis:
    The affected certificates could not all be revoked within the applicable revocation timeline because immediate revocation would have caused significant operational impact to a limited number of government entities.

    SDAIA engaged individually with each affected Enterprise RAs to assess operational dependencies. The affected certificates (331) were confirmed by the respective Enterprise RAs to be supporting critical internal processes/services and that immediate revocation may have resulted in significant operational impact before replacement certificates could be deployed.

    Several factors were considered before delaying revocation:

    • The remaining population represents a small subset of the overall affected certificates.
    • The certificates were issued through an Enterprise RA model with two-step approval, reducing the risk of unauthorized issuance.
    • The certificates are used exclusively for internal government services/processes. The "remaining active" certificates (45) belong to 2 government entities only.

    The delay is expected to remain limited to the minimum period necessary to complete certificate replacement and coordinate revocation.

  • Additional considerations: None

Timeline

All times are GMT+3 unless otherwise noted.

  • 2026-07-20 - 12:14 PM - Internal compliance incident raised and incident response team established
  • 2026-07-22 - between 2:30 PM & 5:03 PM - Communications to Enterprise RA informing about the upcoming revocation of S/MIME certificates
  • 2026-07-22 - between 6:06 PM & 6:35 PM - First batch of certificate revocation (499)
  • 2026-07-23 - between 8:01 PM & 9:57 PM - Second batch of certificate revocation (115)
  • 2026-07-26 - between 1:40 PM & 5:07 PM - Third batch of certificate revocation (42)
  • 2026-07-27 - 11:00 PM - Public disclosure of the preliminary incident report on Bugzilla for Delayed revocation
  • 2026-07-27 - between 3:37 PM & 5:11 PM - Fourth batch of certificate revocation (114)
  • 2026-07-28 - between 8:21 PM & 10:01 PM - Fifth batch of certificate revocation (46)
  • 2026-07-29 - between 11:56 AM & 5:32 PM - Sixth batch of certificate revocation (34)
  • 2026-07-30 - between 4:46 PM & 6:25 PM - Seventh batch of certificate revocation (48)

Related Incidents

Bug Date Description
Bugzilla 2056942 2026-07-22 SDAIA: Missing S/MIME WebTrust audit coverage

Root Cause Analysis

SDAIA identified one primary root cause supported by three contributing factors that collectively resulted in the delayed revocation.

The delayed revocation resulted from a risk-based decision to avoid significant operational impact while affected certificates are being replaced. A subset of the affected S/MIME certificates supported critical internal government services. Immediate revocation within the applicable Baseline Requirements timeline would have disrupted those services before replacement certificates could be completed.

SDAIA therefore coordinated certificate replacement (where necessary) with affected enteprise RAs. Certificates were revoked as soon as replacement certificates had been deployed and/or the subscriber confirmed no potential impact on their services if certificates were revoked. This applies for the 331 certificates affected by the delayed revocation.

As part of the original incident, SDAIA has stopped issuing S/MIME certificates under the affected hierarchy and is proceeding with the removal of the S/MIME trust bit with Microsoft. Once completed, certificates intended solely for internal government use will no longer be issued from a publicly trusted S/MIME hierarchy, preventing similar delayed revocation scenarios from occurring in the future.

Contributing Factor 1: Public trust configuration and subcriber deployment model

  • Description:
    The affected certificates were deployed within critical internal government services. Because the certificates originated from a publicly trusted S/MIME hierarchy, revocation became subject to public trust S/MIME timelines that were not aligned with the operational constraints of these environments.

    This factor established the conditions that made the delayed revocation necessary. Without the hierarchy publicly trusted for S/MIME, the public trust revocation timelines would not have applied.

  • Timeline:

    • 2012-04-29: multi-purpose CA hierarchy established.
    • 2021-08-24: TLS trust removed from the hierarchy; S/MIME trust configuration remained unchanged from this point.
    • 2023-09-15: S/MIME Baseline Requirements introduced (effective date).
    • 2023-09-01: Microsoft Root Program audit requirements for S/MIME took effect
  • Detection:
    The condition was identified during the investigation of the original incident. Prior to that, the certificates were considered part of an internal government service and the implications of the public trust configuration were not fully recognized.

  • Interaction with other factors: N/A

  • Root Cause Analysis methodology used: 5 Whys

Contributing Factor 2: Enterprise RA coordination and impact confirmation

  • Description:
    Following discovery of the original incident, SDAIA coordinated with each Enterprise RA to identify operational dependencies and plan certificate replacement before revocation.

  • Timeline:

    • 2026-07-22 - between 2:30 PM & 5:03 PM - Communications to Enterprise RA informing about the upcoming revocation of S/MIME certificates
    • Between 2026-07-22 - Now - Daily communications with remaining affected subscribers to support transition and certificate replacement.
  • Detection:
    The need for coordinated replacement became apparent once Enterprise RAs confirmed that some certificates supported critical internal services. Earlier engagement may have reduced the number of delayed revocations; however, several certificates would still have required additional transition time.

  • Interaction with other factors: CF3

  • Root Cause Analysis methodology used: 5 Whys

Contributing Factor 3: Subscriber certificate replacement complexity

  • Description:
    Some subscriber environments required coordinated changes across multiple systems before replacement certificates could be deployed and the affected certificates revoked.

  • Timeline:

    • 2026-07-22 - between 2:30 PM & 5:03 PM - Communications to Enterprise RA informing about the upcoming revocation of S/MIME certificates
    • Between 2026-07-22 - Now - Daily communications with remaining affected subscriber to support transition and certificate replacement.
  • Detection: Operational dependencies could only be fully identified through discussions with the affected Enterprise RAs during the transition planning.

  • Interaction with other factors: CF2 (This factor extended the delay required before revocation could safely occur)

  • Root Cause Analysis methodology used: 5 Whys

Lessons Learned

What went well

  • Most certificates were revoked within the planned timeline.
  • Certificate replacement activities began immediately after subscriber confirmation.

What didn't go well

  • Several critical internal services required additional coordination for certificate replacement.
  • Replacement requires more time than initially anticipated due to the complexity of the affected environments.

Where we got lucky

  • The remaining certificates represent a limited subset of the affected population.
  • The certificates are restricted to controlled government environments and are not used for public-facing services.

Additional

  • SDAIA will continue reviewing subscriber transition planning to reduce future delayed revocations.

Action Items

# Action Item Kind Root Cause Evaluation Criteria Due Date Status
1 Remove the S/MIME trust capability from the publicly trusted hierarchy - Microsoft Root Program Action Prevent CF1 Root Program trust configuration updated. 2026-08-27 Ongoing
2 Revoke all affected S/MIME certificates and notify impacted subscribers. Mitigate RC All affected certificates revoked. 2026-08-16 Ongoing
3 Continuous support with Enterprise RA/Subscribers to replace remaining affected certificates Mitigate CF2,CF3 Confirmation to revoke from affected subsribers 2026-08-16 Ongoing
4 Stop issuance of S/MIME certificates under affected CA hierarchy Prevent CF1 S/MIME issuance no longer possible 2026-07-20 Complete

Appendix

The tracking list will be updated as new certificate are revoked.

Status Update

Revocation-related activities continue to progress.

We expect to revoke an additional 9 certificates today, further reducing the number of outstanding affected certificates to 36 certificates.
The remaining certificates are associated with a single entity that has confirmed significant operational impact should the certificates be revoked before replacement. We continue to work closely with the subscriber to complete the necessary transition activities.

A coordinated implementation is currently planned for Sunday (2026-08-16). Based on the outcome of this activity, we expect to revoke all remaining certificates on the same day.

Attachment #9621275 - Attachment is obsolete: true

Status Update

We confirm that 9 additional certificates have now been revoked.

Please find attached the update certificate tracking list, reflecting the latest revocation status and associated certificate information.

Attachment #9626334 - Attachment is obsolete: true

Status Update

We confirm that all remaining affected certificates have now been revoked, completing the revocation activities associated with this incident. The only remaining open action is the removal of the S/MIME trust capability from the affected hierarchy, which continues to also be tracked under the primary incident.
Please note that two certificates were revoked on 13 August 2026 using a different revocation reason than initially planned. For these certificates, the subscribers submitted revocation requests directly through the RA Portal before SDAIA completed the scheduled revocation. The tracking sheet has been updated accordingly to reflect the actual revocation reason, date and status.

Status Update

SDAIA confirms that August 2026 Microsoft Trusted Root Program deployment notice correctly includes the SDAIA Root CA for S/MIME distrust, with a release date of August 27, 2026 and an effective date of September 15, 2026.
Action Item 1 will be considered complete once Microsoft Trusted Root Program has deployed its root store update, which continues to be tracked under the primary incident.

Updated Timeline

All times are GMT+3 unless otherwise noted.

  • 2026-07-20 - 12:14 PM - Internal compliance incident raised and incident response team established
  • 2026-07-22 - between 2:30 PM & 5:03 PM - Communications to Enterprise RA informing about the upcoming revocation of S/MIME certificates
  • 2026-07-22 - between 6:06 PM & 6:35 PM - First batch of certificate revocation (499)
  • 2026-07-23 - between 8:01 PM & 9:57 PM - Second batch of certificate revocation (115)
  • 2026-07-26 - between 1:40 PM & 5:07 PM - Third batch of certificate revocation (42)
  • (*) 2026-07-28 - 12:00 AM - Public disclosure of the preliminary incident report on Bugzilla for Delayed revocation
  • 2026-07-27 - between 3:37 PM & 5:11 PM - Fourth batch of certificate revocation (114)
  • 2026-07-28 - between 8:21 PM & 10:01 PM - Fifth batch of certificate revocation (46)
  • 2026-07-29 - between 11:56 AM & 5:32 PM - Sixth batch of certificate revocation (34)
  • 2026-07-30 - between 4:46 PM & 6:25 PM - Seventh batch of certificate revocation (48)
  • 2026-08-13 - between 3:37 PM & 5:59 PM - Eighth batch of certificate revocation (9) + certificate revocation initiated by subscriber (2)
  • 2026-08-16 - between 10:21 AM & 6:18 PM - Nineth batch of certificate revocation (34)
  • 2026-08-16 - between 6:30 PM - Confirmation that all affected certificates were revoked

(*) Timeline correction: Due to time zone conversion mistake, the identified activity time was recorded as GMT+2 rather than GMT+3. It has now been corrected and SDAIA confirms it did not affect the sequence of events, the reported actions, or any other conclusions presented in this report.

Updated Action Items

# Action Item Kind Root Cause Evaluation Criteria Due Date Status
1 Remove the S/MIME trust capability from the publicly trusted hierarchy - Microsoft Root Program Action Prevent CF1 Root Program trust configuration updated. 2026-08-27 Ongoing
2 Revoke all affected S/MIME certificates and notify impacted subscribers. Mitigate RC All affected certificates revoked. 2026-08-16 Complete
3 Continuous support with Enterprise RA/Subscribers to replace remaining affected certificates Mitigate CF2,CF3 Confirmation to revoke from affected subscribers 2026-08-16 Complete
4 Stop issuance of S/MIME certificates under affected CA hierarchy Prevent CF1 S/MIME issuance no longer possible 2026-07-20 Complete

Status Update

Final revocation summary for this incident:

  • Certificates revoked: 331
  • Certificates not yet revoked: 0
  • Certificates planned for revocation that expired before revocation: 0
  • Non-compliance end date: 2026-08-16

Correction to the Action Items disclosed in the Full Incident Report. Action Item 1 (Remove the S/MIME trust capability from the publicly trusted hierarchy - Microsoft Root Program Action) is owned and tracked under the primary incident, Bugzilla 2056942, where it is Action Item 1. It was duplicated into this report because it addresses Contributing Factor 1. It is not an Action Item of this incident and is withdrawn from this report's Action Items. It will be restated as an ongoing commitment in the closure report. Progress and completion will be reported in Bugzilla 2056942.

# Action Item Kind Root Cause Evaluation Criteria Due Date Status
1 Remove the S/MIME trust capability from the publicly trusted hierarchy - Microsoft Root Program Action Prevent CF1 Root Program trust configuration updated. 2026-09-15 Withdrawn from this report. Tracked as Action Item 1 in Bugzilla 2056942
2 Revoke all affected S/MIME certificates and notify impacted subscribers. Mitigate RC All affected certificates revoked. 2026-08-16 Complete
3 Continuous support with Enterprise RA/Subscribers to replace remaining affected certificates Mitigate CF2, CF3 Confirmation to revoke from affected subscribers 2026-08-16 Complete
4 Stop issuance of S/MIME certificates under affected CA hierarchy Prevent CF1 S/MIME issuance no longer possible 2026-07-20 Complete

Report Closure Summary

  • Incident description: Following the S/MIME audit coverage gap reported in Bugzilla 2056942, SDAIA was required to revoke the S/MIME subscriber certificates issued under Government CA 2. 331 of those certificates were not revoked within the 5 day period required by CA/Browser Forum S/MIME Baseline Requirements section 4.9.1. Revocation of the last of these certificates completed on 2026-08-16 and no affected certificate remains valid.

  • Incident Root Cause(s): The delay was the result of a risk based decision to avoid disrupting critical internal government services before replacement certificates could be deployed. Three factors contributed. CF1: the certificates were issued from a hierarchy publicly trusted for S/MIME although they served exclusively internal government use, which made the public trust revocation timelines applicable to environments not built for them. CF2: operational dependencies could only be established through individual engagement with each Enterprise RA after discovery. CF3: replacement in several subscriber environments required coordinated changes across multiple systems, which extended the time before revocation could safely occur.

  • Remediation description: S/MIME issuance and renewal under the affected hierarchy were suspended on 2026-07-20 and have not resumed. SDAIA engaged each affected Enterprise RA individually, supported certificate replacement daily, and revoked certificates in batches as each subscriber confirmed readiness. The final batch was revoked on 2026-08-16, setting the non-compliance end date. Tracking lists reflecting the revocation status, date and reason of every affected certificate were published as attachments to this report.

  • Commitment summary:

    • Removal of the S/MIME trust capability from the publicly trusted hierarchy, a Microsoft Trusted Root Program action tracked as Action Item 1 in Bugzilla 2056942. Microsoft has published the restriction with an effective date of 2026-09-15. Once in effect, certificates intended solely for internal government use will no longer be issued from a hierarchy publicly trusted for S/MIME, which removes the condition described in CF1.
    • Continued review of subscriber transition planning, so that replacement lead times are established before revocation is required rather than after.

All Action Items disclosed in this report have been completed as described, and we request its closure.

Flags: needinfo?(incident-reporting)

This is a final call for comments or questions on this Incident Report.

Otherwise, it will be closed on approximately 2026-09-07.

Flags: needinfo?(incident-reporting)
Whiteboard: [ca-compliance] [leaf-revocation-delay] → [close on 2026-09-07] [ca-compliance] [leaf-revocation-delay]
Status: ASSIGNED → RESOLVED
Closed: 6 days ago
Resolution: --- → FIXED
Whiteboard: [close on 2026-09-07] [ca-compliance] [leaf-revocation-delay] → [ca-compliance] [leaf-revocation-delay]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: