1936741 - Telia: Findings in 2024 Audit

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Antti Backman]

Audit Incident Report

This is Telia CA's audit incident report of the 2024 annual ETSI Audit. All findings have been remedied and verified by Auditors before issuance of the respective audit attestation.

Finding #1, S/MIME

ETSI EN 319 401, 7.7 Operation security

Test documentation for Cygate secure mail, and software acceptance process
shall be improved. [REQ-7.7-03, REQ-7.7-04]

Root Cause Analysis

RA Change management documentation was found lacking information on how testing in relation to change management of RA Software changes should be conducted as part of change management control procedures. The RA Software quality assurance practice was not part of the RA Change managent documentation and practices, thus audit identified this as a non-conformant gap in the documentation.

Root cause for this is that in our understanding of the requirements, the software deployment and related testing practices of the released sofware version should be part of the RA Change management documentation, but RA Software quality assurance before particular release is made available for deployment is a separate process not applicable to the said requirements. Therefore the incorporation of the development stage quality assurance was not part of the documentation.

Telia RA Software development is based on SAFe Agile model and the practice and supporting documentation was not linked and/or incoprorated in the Change management documentation resulting the relevant documentation to be non-conformant by auditors.

Action Items

Action Item	Kind	Due Date
RA software testing process and practices duly documented to change management control procedures and documentation.	Prevent	2024-12-02

Finding #2 S/MIME

ETSI EN 319 411-1, 6.3.12 Key escrow and recovery

The security of any duplicated subject's private keys shall be improved. [SDP6.3.12-02]

Root Cause Analysis

Internal back-up procedure documentation was found lacking of explicit statement addressing the requirement referenced for the non-conformity, thus defining the number of copies required to meet the said requirement. Although the practice was implemented to ensure what is required, auditors interpretation and conclusion was that the explicit statement missing in the documentation would be a non-conformity.

Our own interpretation of the requirement did not recognize such explicit documentation to be needed consequently resulted in the missing details recorded in the relevant back-up practice documentation causing the relevant documentation to be non-conformant by auditors.

Action Items

Action Item	Kind	Due Date
Internal documentation for backup requirements supplemented to document the number of keys required.	Prevent	2024-11-19

Finding #3

ETSI EN 319 411-1, 6.6.3 OCSP profile

OCSP service monitoring shall be improved. [OVR-6.6.3-03]

Root Cause Analysis

OCSP responder service was identified lacking "The CA should monitor such requests concerning non-issued certificates on the responder as part of its security response procedures to check if this is an indication of an attack." capability.

OCSP monitoring solution was changed in late August 2024 and in that transition monitor to detect and alert OCSP requests made for certificates not found in Telia CA's database was not migrated to the new solution.

In part of this transition we reviewed the TLS and S/MIME BRs effective at the time and neither of the BRs required OCSP requests to be recorded, but the review did not cover ETSI EN 319 411-1, consequently caused non-conformant transition to the new monitoring solution that was only detected by the annual audit.

Action Items

Action Item	Kind	Due Date
Monitoring and alerting of OCSP request of non-issued certificates deployed.	Prevent	2024-12-02

Comment 1

[chrome-root-program]

Thanks for filing this report, and for taking the time to use Markdown formatting features. We consider doing so a good practice and it goes a long way in improving the readability of these reports!

A few questions below, based on Audit Attestation Letters 1 and 2 disclosed to the CCADB:

Question 1) Both letters contain the statement: “All major non-conformities have been closed before the issuance of this attestation.”

a) Should we interpret this to mean “major non-conformities” were detected during the audit?
b) If yes, can you describe which findings in the Audit Attestation Letters or this report correspond to those “major non-conformities?”

Question 2) In the “Standards Considered” listing, we note the Chrome Root Program Policy is listed. This seems great, but for our own knowledge, are you able to share details concerning how Telia was evaluated against our program policy?

Question 3) Additionally, in the “Standards Considered” listing, we note the following ETSI standards listed:

• ETSI EN 319 411-1 V1.3.1 (2021-05)
• ETSI EN 319 401 V2.3.1 (2021-05)

It’s our understanding:

• EN 319 411-1 V1.3.1 was superseded by EN 319 411-1 V1.4.1 in October 2023.
• EN 319 401 V2.3. was superseded by EN 319 401 V3.1.1 in June 2024.

When considered against the “audit dates” listed in the attestation letters (beginning 2024-11-12), that means 319 411-1 V1.3.1 had been superseded for approximately 13 months and EN 319 401 V2.3 was superseded for approximately 5 months.

Can you help us understand the rationale for relying on superseded versions of the ETSI standards (especially in the case of 319 411-1 which was superseded prior to the audit period beginning)?

General comment: Both the descriptions of the findings and the corresponding Action Items lack specificity, making it difficult to understand the nature of the findings or to help us understand whether the Action Items described will meaningfully prevent future recurrence of the issues. For example, we’re not sure what “The security of any duplicated subject's private keys shall be improved. [SDP6.3.12-02]” means in practice. Adding more detail would be appreciated.

Comment 2

[Antti Backman]

Hi chrome-root-program,

Thank you for your comments and questions in comment #1.

As some of the topics require us to work with our auditor, we'll come back with full response to all the questions and comments no later than the 17th of December 16:00 EET.

Comment 3

[Antti Backman]

We thank you your questions, observations and comments in comment #1. This is our response to your request to and in conjuction to this response, updated audit reports have been uploaded to the CCADB to remedy issues identified by chrome-root-program. We hope you find the response satisfactory.

Question 1) Both letters contain the statement: “All major non-conformities have been closed before the issuance of this attestation.”
a) Should we interpret this to mean “major non-conformities” were detected during the audit?
b) If yes, can you describe which findings in the Audit Attestation Letters or this report correspond to those “major non-conformities?”

No major non-conformities were detected in the audit. More clarification added to the newly updated audit reports to bring further clarity to this issue. ACAB-c templates provide predefined statements for the audit results and the selected option was deemed the most appropriate to represent our audit result. Additional clarification statement supplements the standard predefined phrase. Hope this explanation and supplemental information provided on the reports remove any ambiquity about findings severity.

Question 2) In the “Standards Considered” listing, we note the Chrome Root Program Policy is listed. This seems great, but for our own knowledge, are you able to share details concerning how Telia was evaluated against our program policy?

We've agreed with our auditors to provide us feedback on our adherence to Root Program policies / requirements to give us furhter assurance of our compliance. While this type of consideration would not be part of the actual audit criterion, having external review as part of our annual audit also for the Root Program requirements have been considered a value-adding practice by us. Further as we've seen over time that there might be requirements that may be more detailed in the Root Program requirements / policies than in the actual audit framework, additional verification by auditors is found valuable to us.

We keep our auditors updated on the Root Program requirement updates over time and during the audits in addition to the audit they also consider the Root Program requirements as part of the scope they work through with us during the audit. The detailed practice auditor's employ for this is at their discretion. Some of the means typicalle would be detailed questions made during the audit session to verify selected topics and "homework" done by auditors against the Root Program requirements. Any observations they make, gives us futher possibility to improve our compliance status with the ever extending requirement landscape.

And while we have this practice, we've requested this to be disclosed through the audit reports. As explained we have found this to be value-adding to us and while we're doing it we've decided to make that available in the audit reports as well.

Question 3) Additionally, in the “Standards Considered” listing, we note the following ETSI standards listed:

• ETSI EN 319 411-1 V1.3.1 (2021-05)
• ETSI EN 319 401 V2.3.1 (2021-05)

It’s our understanding:

• EN 319 411-1 V1.3.1 was superseded by EN 319 411-1 V1.4.1 in October 2023.
• EN 319 401 V2.3. was superseded by EN 319 401 V3.1.1 in June 2024.

When considered against the “audit dates” listed in the attestation letters (beginning 2024-11-12), that means 319 411-1 V1.3.1 had been superseded for approximately 13 months and EN 319 401 V2.3 was superseded for approximately 5 months.

Can you help us understand the rationale for relying on superseded versions of the ETSI standards (especially in the case of 319 411-1 which was superseded prior to the audit period beginning)?

We agree with your interpretation of the versions and thank you for your prompt observation. The audit was conducted based on the latest versions. We missed in our final review that the reports had incorrect version numbers representing the requirement standards. That issue was discussed with our auditor and has now been remedied on the newly updated and published reports. We shall take this as lesson learned and will ensure in the future audits that we verify version information also before passing the preliminary AALs for publication.

General comment: Both the descriptions of the findings and the corresponding Action Items lack specificity, making it difficult to understand the nature of the findings or to help us understand whether the Action Items described will meaningfully prevent future recurrence of the issues. For example, we’re not sure what “The security of any duplicated subject's private keys shall be improved. [SDP6.3.12-02]” means in practice. Adding more detail would be appreciated.

Thank your for this valuable feedback, we'll consider this as opportunity for improvement. We've also reviewing the proposed CCADB Incident Reporting Guidelines to be able to provide better and detailed reports in the future.

Below table provides further details on the findings:

Finding	Details
Finding #1	Observation details: There is no test documentation for Cygate secure mail, and the software acceptance process is not formal enough. Consequently, secure software development life cycle is not followed. Remediation details: Our S/MIME RA software development process and related documentation was not integral part of operational change management process and documentation. Auditors consequently interpreted this as there is no documentation for testing for the agile software development. S/MIME RA software development is following Scaled Agile Framework (SAFe) fitted Telia's purposes. This process is well documented including testing during the agile cycles before the any software development is released for deployment. Deployment of new releases through the operational change management is separate process including testing of the ready made software release. Auditors required us to incorporate SAFe processes also to be part of the operational change management to suffice their interpretation of the requirement. This is now updated to our documentation through the Action Item so that the full end-to-end cycle is integral part of the operational change management and satisfying the requirements. Evidence provided for the auditors to verify the implementation.
Finding #2	Observation details: We could not find evidence that the number of copies of a subject's private keys is well known to the CA. Therefore, it could not be decided whether it exceeds the minimum needed to ensure continuity of the service. Remediation details: Our back up procedure documentation was lacking explicit statement of how many copies of private keys is needed to meet the requirement. During the audit we explained the procedure and it was determined to be sufficient to meet the requirement. Basically we had documented what is needed to ensure continuity of the services based on rotational back up procedure, that documentation did not consider explicitly if the procedure would be sufficient to ensure continuity from the requirement perspective. Auditors recommended that information to be included in the documentation. The remedation Action Item completed and evidence provided for the auditors to verify the implementation..
Finding #3	Observation details: We could not find evidence that the CA monitors requests concerning nonissued certificates on the responder as part of its security response procedures to check if this is an indication of an attack. Remediation details: As remediation Action we deployed the monitor and alarms to detect and notify of possible malicous / suspicious request made towards our OCSP responder end-points. Monitor was adequately documented and evidence provided for the auditors to verify the implementation.

All action items have been completed, Telia continues to follow-up this incident for comments and questions.

Comment 4

[Ben Wilson]

Hi Antti,

Could you please provide a very brief closing summary?

A closing summary should briefly:

• describe the incident, its root cause(s), and remediation;
• summarize any ongoing commitments made in response to the incident; and
• attest that all Action Items have been completed.

Here is a markdown template:

Incident Report Closure Summary

• Incident Description: [Two or three sentences summarizing the incident.]
• Incident Root Cause(s): [Two or three sentences summarizing the root cause(s).]
• Remediation Description: [Two or three sentences summarizing the incident's remediation.]
• Commitment Summary: [A few sentences summarizing ongoing commitments made in response to this incident.]

All Action Items disclosed in this Incident Report have been completed as described, and we request its closure.

Thanks!

Comment 5

[Antti Backman]

Incident Report Closure Summary

Finding #1

• Incident Description: There is no test documentation for Cygate secure mail, and the software acceptance process is not formal enough. Consequently, secure software development life cycle is not followed. Reference ETSI EN 319 401, 7.7 Operation security, [REQ-7.7-03, REQ-7.7-04]
• Incident Root Cause(s): Telia CA's interpretation of the requirements was prior this finding, that RA Software quality assurance and related documentation before particular release is made available for deployment is a separate process not applicable to the said requirements. Therefore the incorporation of the RA software development process and in particular quality assurance was not part of the operation change management documentation, thus leading to the non-conformity identified.
• Remediation Description: Documentation updated through the Action Item so that the full end-to-end software development cycle is integral part of the operational change management and satisfying the requirements.
• Commitment Summary: Immediate remediation done as described in this summary. Telia shall maintain continuously documentation as deemed necessary based on the auditor's finding.

Finding #2

• Incident Description: Auditor could not find evidence that the number of copies of a subject's private keys is well known to the CA. Therefore, it could not be decided whether it exceeds the minimum needed to ensure continuity of the service. Reference ETSI EN 319 411-1, 6.3.12, [SDP6.3.12-02]
• Incident Root Cause(s): Telia CA's interpretation of the requirement did not recognize that explicit documented statement to be needed in the back-up procedure documentation to state the number of keys required for continuity of the service. Consequently resulted in the missing details recorded in the relevant back-up practice documentation causing the relevant documentation to be non-conformant by auditors.
• Remediation Description: Back-up practice and routine documentation supplemented to consider explicitly how the back-up procedure ensure continuity from the requirement perspective.
• Commitment Summary: Immediate remediation done as described in this summary. Telia shall maintain continuously both the documentation and practice for the back-up routines to maintain requirement compliance.

Finding #3

• Incident Description: Auditor could not find evidence that the CA monitors requests concerning nonissued certificates on the responder as part of its security response procedures to check if this is an indication of an attack. Reference ETSI EN 319 411-1, 6.6.3 OCSP profile, [OVR-6.6.3-03]
• Incident Root Cause(s): OCSP monitoring solution was changed in late August 2024 and in that transition monitor to detect and alert OCSP requests made for certificates not found in Telia CA's database was not migrated to the new solution.
• Remediation Description: Required monitor and alarms deployed to detect and to notify about possible malicous / suspicious request made towards our OCSP responder end-points.
• Commitment Summary: OCSP responder monitoring shall be maintained to ensure future compliance to the requirement.

All Action Items disclosed in this Incident Report have been completed as described, and we request its closure.

Comment 6

[Ben Wilson]

I don't have any further questions and intend to close this on Friday, 20-Dec-2024.