Closed Bug 1983274 Opened 1 year ago Closed 9 months ago

PKIoverheid: TSP KPN Findings in 2025 ETSI Audit - Incident Report #14 – Back-up

Categories

(CA Program :: CA Certificate Compliance, task)

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: pkioverheid, Assigned: pkioverheid)

Details

(Whiteboard: [ca-compliance] [audit-finding])

Preliminary Incident Report

Summary

  • Incident Description:
    • Minor Non-conformity: Back-up Plan(s) Not Documented
  • Relevant Policies:
    • ETSI 319 401 (REQ-7.11.2-02X)
  • Source of incident disclosure:
    • Annual ETSI Audit
Summary: TSP KPN Findings in 2025 ETSI Audit - Incident Report #14 – Back-up → PKIoverheid: TSP KPN Findings in 2025 ETSI Audit - Incident Report #14 – Back-up
Assignee: nobody → pkioverheid
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Whiteboard: [ca-compliance] [audit-finding]

Full Incident Report - ETSI Finding #14 - Back-up

Summary

  • CA Owner CCADB unique ID: A000068

  • Incident description: The CAB noted that a backup plan (or plans) weren’t fully documented. The explicit testing of a backup plan was a new requirement from ETSI EN 319 401 v3.1.1. This resulted in a minor non-conformity during the annual ETSI audit.

  • Timeline summary:

    • Non-compliance start date: 28-Feb-2025

    • Non-compliance identified date: 11-Jul-2025

    • Non-compliance end date: Ongoing.

  • Relevant policies:

    • ETSI 319 401 REQ-7.11.2-02X: The TSP shall define backup plans taking into account at least the following: a) recovery times; b) assurance of the backup copies' completeness and accuracy (including configuration data and information stored in cloud service environment); c) storage of backup copies at a safe location or locations which are outside the network of the system backed up and are at sufficient distance to escape any damage from a disaster at the main site; d) physical/environmental and logical controls for backup copies in accordance with their information classification level; and e) processes for restoring information from backup copies (including approval processes).
  • Source of incident disclosure: Finding by CAB during annual ETSI audit.

Impact

  • Total number of certificates: N/A

  • Total number of "remaining valid" certificates: N/A

  • Affected certificate types: N/A

  • Incident heuristic: N/A

  • Was issuance stopped in response to this incident, and why or why not?: N/A (see point below)

  • Analysis: N/A

  • Additional considerations: KPN only operates legacy S/MIME-capable CAs, which have not issued any S/MIME certificates since 1 August 2023. At that time this was changed due to updated S/MIME regulations, under which email addresses and the EKU emailProtection were no longer included in publicly trusted certificates.

  • Incident heuristic: N/A

Timeline

  • 28-Feb-2025: Effective date of version 3.1.1 of ETSI EN 319 401 which included updated and new requirements.

  • 11-Jul-2025: Auditor identifies finding

  • 17-Jul-2025: Created Corrective Action Plan

  • 12-Aug-2025: Corrective Action Plan Approved by auditor

Related Incidents

Bug Date Description
1983262 15-Aug-2025 Similar root cause(s)
1983271 15-Aug-2025 Similar root cause(s)
1983273 15-Aug-2025 Similar root cause(s)

Root Cause Analysis

Contributing Factor 1: Compliance management not fully effective

  • Description: Not all requirements were implemented in time, as the changes were not fully tracked or prioritized during the transition period, because there was no structured process in place to monitor updates. Responsibility for tracking and prioritizing these changes was not clearly assigned, which contributed to delays in implementation.

  • Timeline: See main timeline.

  • Detection: Audit finding.

  • Interaction with other factors: No.

  • Root Cause Analysis methodology used: N/A

Lessons Learned

  • What went well: Backup systems are in place and configured correctly.

  • What didn’t go well: The documentation is not fully in line with the requirement.

  • Where we got lucky: N/A

  • Additional: N/A

Action Items

Action Item Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Improve backup plan Mitigate Root Cause #1 Check if implemented 2025-10-11 In progress
Execute gap analyses and implement remaining requirements Prevent Root Cause #1 Check content 2025-10-11 In progress
Improve the structured process for tracking and implementing new requirements, including deadlines, periodic reviews and timely escalation to management. This includes assigning responsible persons for monitoring changes, and integrating updates into operational procedures. Prevent Root Cause #1 Check content 2025-10-11 In progress
Discuss compliance requirements and the necessary resources in a recurring meeting with management Detect Root Cause #1 Check content 2025-10-11 Completed

Appendix

N/A

In the Action items above an error had occurred while pasting the information from internal systems to markdown, so an updated version with the right Evaluation Criteria is provided below. In the meantime, PKIoverheid is monitoring this bug and we're open for additional questions or remarks people might have.

Action Item Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Improve backup plan to conform to ETSI requirements Mitigate Root Cause #1 Update backup plan and report back 2025-10-11 In progress
Execute gap analysis and implement remaining requirements Mitigate Root Cause #1 Analysis performed, use output for Action item #1 and report back 2025-10-11 In progress
Improve the structured process for tracking and implementing new requirements, including deadlines, periodic reviews and timely escalation to management. This includes assigning responsible persons for monitoring changes, and integrating updates into operational procedures. Prevent Root Cause #1 Implement process and report back 2025-10-11 In progress
Discuss compliance requirements and the necessary resources in a recurring meeting with management Detect Root Cause #1 Recurring meeting has been planned and has taken place at least twice 2025-10-11 In progress

With regards to this audit finding we have some updates:

  • As indicated earlier Action item #3 had been completed already.
  • Action items #2 & #4 (which are also an action items in several other bugs) have been completed as well.
  • KPN has indicated that due to unforeseen complexities action item #1 will take a bit longer (three weeks delay).

With that, the status is:

Action Item Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Improve backup plan to conform to ETSI requirements Mitigate Root Cause #1 Update backup plan and report back 2025-11-01 In progress
Execute gap analysis and implement remaining requirements Mitigate Root Cause #1 Analysis performed, use output for Action item #1 and report back 2025-10-11 Completed
Improve the structured process for tracking and implementing new requirements, including deadlines, periodic reviews and timely escalation to management. This includes assigning responsible persons for monitoring changes, and integrating updates into operational procedures. Prevent Root Cause #1 Implement process and report back 2025-10-11 Completed
Discuss compliance requirements and the necessary resources in a recurring meeting with management Detect Root Cause #1 Recurring meeting has been planned and has taken place at least twice 2025-10-11 Completed

KPN has reported that action item #1 has been completed. With that, remediation is complete. We'll be submitting a closure request shortly, but in the meantime we'll be monitoring this bug and are open to comments and/or questions the community might have. Thanks.

Report Closure Summary

  • Incident description: The CAB noted that a backup plan (or plans) weren’t fully documented. The explicit testing of a backup plan was a new requirement from ETSI EN 319 401 v3.1.1. This resulted in a minor non-conformity during the annual ETSI audit.
  • Incident Root Cause(s): Not all requirements were implemented in time, as the changes were not fully tracked or prioritized during the transition period, because there was no structured process in place to monitor updates. Responsibility for tracking and prioritizing these changes was not clearly assigned, which contributed to delays in implementation.
  • Remediation description: KPN has taken steps to improve its backup and compliance processes. These include an improved and updated backup plan which complies with the updated 401 requirements and the execution of a gap analysis for the remaining new or updated 401 requirements. The process for tracking and implementing new compliance requirements has been improved, including clear deadlines, assigned responsibilities, and escalation paths and compliance requirements and resource needs are now discussed in a recurring management meeting to ensure ongoing alignment and oversight.
  • Commitment summary: KPN commits to maintaining a documented and testable backup strategy. To support this, restore tests for offline backups will be scheduled and monitored regularly. KPN is also implementing automated backup-restore testing using Nagios to ensure recoverability is continuously validated.

All Action Items disclosed in this report have been completed as described, and we request its closure.

This is a final call for comments or questions on this Incident Report.

Otherwise, it will be closed on approximately 2025-11-19.

Flags: needinfo?(incident-reporting)
Whiteboard: [ca-compliance] [audit-finding] → [close on 2025-11-19] [ca-compliance] [audit-finding]
Status: ASSIGNED → RESOLVED
Closed: 9 months ago
Flags: needinfo?(incident-reporting)
Resolution: --- → FIXED
Whiteboard: [close on 2025-11-19] [ca-compliance] [audit-finding] → [ca-compliance] [audit-finding]
You need to log in before you can comment on or make changes to this bug.