1911335 - PKIoverheid: Delayed S/MIME audit report for MoD PKIoverheid G3 CA

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Jochem van den Berge]

Steps to reproduce:

Incident Report

Summary

On September 1, 2023 the requirement to conform to the Baseline Requirements for the Issuance and Management of Publicly-Trusted S/MIME Certificates (SBR) came into effect when Mozilla published Mozilla Root Store policy v2.9. Logius PKIoverheid operates as a SuperCA in which it only issues issuing CAs to governmental and commercial entities (Trust Service Providers, TSPs) and as such enforces any kind of policy (internal and external) via the PKIoverheid Programme of Requirements (PoR), available on https://cp.pkioverheid.nl.

When v2.9 of the MSRSP came into effect Logius already voiced their concerns that the text stated on https://wiki.mozilla.org/CA/Transition_SMIME_BRs would potentially lead to issues because of the short lead time to contract auditors to perform a Period-of-Time (PoT) audit with a start date of September 1, 2023 for some of her issuing CAs due to the fact that preparations for audits usually start several months before the audit period ends and some PKIoverheid issuing CAs had their audit period end around October 30.

Due to this, our reply to the survey regarding MRSP v2.9 compliance indicated that this would take more time (until October 2024). Issuing CAs would either include this required SBR audit in their annual audit, or where the audit period would exceed 365 days, would include this as an additional Period-of-Time (PoT) audit covering SBR controls specifically.

It only recently became clear that the Dutch Ministery of Defense (NL MoD, or MinDef) will be unable to have a PoT S/MIME audit (ETSI EN 319 411-1/2 with ETSI TS 119 411-6) in place before October 2024, and as such will exceed the deadline as communicated in our CCADB Survey response of August 2023.

Impact

Audit information on S/MIME compliance for the CA "Ministerie van Defensie PKIoverheid Organisatie Persoon CA - G3” (see https://crt.sh/?caid=132287) will not become available before January 2025.

Timeline

• 2023-01-01 Adoption of the S/MIME Baseline Requirements by the S/MIME Cert WG of the CA/Browser Forum. This document has a self-declared effective date of September 1, 2023.

• January/February/March 2023: Policy Authority (PA) PKIoverheid met several times with each TSP to assess the impact the SBRGs would have on their operations. Most TSPs indicated to issue no or few S/MIME certificates, however since their CAs have the emailProtection EKU they would need to be compliant to the S/MIME Baseline Requirements. Based on these sessions the PA developed an approach in which TSPs would:

a. migrate to a new non-S/MIME capable CA under the current G3 Root CA or new G4 Root CA, or
b. migrate to a new S/MIME capable CA under the current G3 Root CA which included strict S/MIME policies.

• April 2023: MoD communicated these alternatives were not an option due to operational concerns. As such, talks about MoD-specific alternatives continued over the summer and autumn of 2023.

• September 1, 2023: Mozilla adopted version 2.9 of the Mozilla Root Store Policy (MRSP), which mandated compliance with the S/MIME Baseline Requirements via either Webtrust (WTSM) or ETSI EN 319 411-1/2 with ETSI TS 119 411-6.

• September 2023: Logius voiced concerns (privately and publicly via the CCADB survey) that auditing within the requirement timeframe stated on the wiki page CA/Transition SMIME BRs - MozillaWiki would be infeasible and that the audit results of all TSPs within PKIoverheid would not be available before October 2024.

• October-December 2023: In informal and formal communications between Logius and the TSPs it was stated that S/MIME compliance should be part of the next auditing cycle.

• July 2024: After communications with the MoD, it became clear to the PA PKIoverheid that there would not be a valid audit statement for S/MIME compliance for the MoD within the time frame Logius specified in her response of the CCADB Survey of August 2023.

• August 2, 2024: Filing of this bug.

Root Cause Analysis

After careful analysis and reflection, we concluded the root cause of the audit delay was because of a misjudgement in available audit capacity and that our focus was on future solutions:

• The main focus of PKIoverheid has been on solutions migrating the user base of its TSPs to either single purpose S/MIME CAs with strict policies, or moving to non-S/MIME capable CAs. Arranging a timely Period-of-Time (PoT) S/MIME audit outside of the regular audit cycle was regarded as trivial.

• NL-MOD requires that all personnel, including Conformity Assessment Body auditors, must be vetted by the Dutch Military Intelligence and Security Service (MIVD) in the Netherlands. This is a precise and laborious process especially for foreign entities, including auditors.

• The Netherlands currently has only one accredited and vetted CAB (BSI) which did not have available capacity for an additional PoT S/MIME (ETSI TS 119 411-6) audit.

• Vetting an additional CAB which would have capacity for a timely audit was not feasible in the available timeframe.

• PKIoverheid historically had a labourious process in which changes in standards/requirements were analysed and sent to TSPs via email for verification. During 2023/2024 we have made the switch to using GitHub in which TSPs are party to discussions and changes. This increased agility greatly but the focus was too much on working with the new system and at the same time simplyfing our CP.

• Misjudgement in available audit capacity lead to a delay for the PoT S/MIME (ETSI TS 119 411-6) audit.

Lessons Learned

What went well

• Logius was (and is) in active discussions with TSPs to analyse the impact of the S/MIME Baseline Requirements on their operations and is actively moving to migrate TSPs to CAs either not publicy trusted or to CAs which are single purpose S/MIME CAs (e.g. only S/MIME strict).

What didn't go well

• Our focus was too much on the technical/future use cases and did not take into account the implications for current still valid CAs (“extant CAs”) and their auditing and compliance requirements in enough detail. Originally there was an expectation that all S/MIME-capable CAs would cease issuance and migrate to new CAs before September 1st 2024.

• Between September 1st, 2023 and July 2024 there was insufficient attention from the PA to check current S/MIME compliance preparations and consequently the commitment the PA PKIoverheid gave in the August 2023 CCADB survey. Although tickets/reminders are automatically created for the compliance officer to check TSP audit preparations and results in their yearly audit cycle, these checks were not implemented for the (additional) S/MIME PoT audits.

• Although PKIoverheid TSPs can use any ETSI auditor accredited by an EER NAB, in practice due to constraints like the one mentioned here, there is currently only one auditing firm available as a viable option for the NL MoD.

Where we got lucky

Not applicable.

Action Items

Action Item	Kind	Due Date
Exploration of Sourcing/mandating a backup MoD/TSP auditor	Mitigate	2025-01-30
Creation of a dedicated GitHub repo for compliance which uses GitHub actions to import commits/issues from both Mozilla and CABF repos, all of which will require action/signoff by the compliance officer	Prevent/Detect	2024-08-30
Investigating if it is possible to use these issues to (semi-)automatically create issues for affected TSPs and assign them to a TSP key-user for followup	Prevent/Detect	2024-10-01

Appendix

Not applicable.

Details of affected certificates

"Ministerie van Defensie PKIoverheid Organisatie Persoon CA - G3” (see https://crt.sh/?caid=132287

Comment 1

[Jochem van den Berge]

An status update from our side:

The second action item (a dedicated GitHub repo with auto-import of GitHub issues from both CABF and Mozilla) has been completed. Please find attached an example of a recent sign-off of ballot SMC-08

Comment 2

[Jochem van den Berge]

Attachment: example_compliance_check_SCWMG-SMC08.png

Comment 3

[Jochem van den Berge]

With regards to the third mentioned remediation action we can report the following. We have looked into turning GitHub compliance issues into individual tasks/issues per TSP. However, due to the setup of PKIoverheid (which involves several commercial TSPs which cannot or won't share sensitive information accessible to other TSPs) this would entail setting up a separate repository per TSP which would introduce a lot of overhead and which would potentially also lead tot confusion. As such, for now we have opted to manually produce a (PDF) compliance form based on the imported issues (if applicable to a TSP) and send these to TSPs for follow-up. In this form they need to indicate the expected impact of the change, the time needed for implementation and if they are capable of meeting the effective date of said change.

We acknowledge that this is a less than ideal solution with still a lot of overhead and we're striving to come up with a better solution which involves less manual effort on our part (and our TSPs). However, this will require a bit of a rethink of how we currently use GitHub Automation and GitHub in general for compliance matters. Until then, issues are monitored weekly and will be forwarded to TSPs when needed and compliance monitoring per TSP will be done in a separate (but linked) issue.

Kind regards,

Jochem van den Berge

Comment 4

[Jochem van den Berge]

Hello Ben,

It has been rather quiet with regards to this bug. The remaining action was still open and there wasn't anything to report on that front. We have now passed the deadline for that item and we do have something to report on that front. Also, there have been recent developments which impact this issue.

Remediation of previous issue

As stated in https://bugzilla.mozilla.org/show_bug.cgi?id=1911335#c0 one of the proposed remediation actions was looking into solutions for the limited amount of auditing capacity for ad-hoc audits when there is a scope extension (like the new ETSI TS 119 411-6). Further research has yielded that in the current contract between Logius and her TSPs there is already a provision which allows for Logius to demand an ad-hoc audit when needed. We will bring this to the attention of our TSPs and will update internal incident handling procedures accordingly. This, combined with the conclusions reached since (see below) and the further separation of S/MIME issuance in both our current G3 hierarchy and the upcoming G4 hierarchy we deemed sufficient to mitigate any future issue at this point. As such, we deem this remediation complete.

New issue
Although not explicitly stated in my original post, part of the remediation was always to perform a full period-of-time audit with with regards to S/MIME, it has since become clear that conducting an SBRG audit (via ETSI TS 119 411-6) and obtaining the associated report/certification is not a feasible option. Recently we have received news from the NL MoD that the desired result (a full period-of-time audit covering SBRG controls between 1-9-2023 and 15-11-2024) wasn't possible. This is due to two main reasons:

1. An (incorrect) interpretation of the definition of an extant CA in the SBRGs. Because of this, MoD thought that an issuing CA which was signed before 1-9-2023 could still issue S/MIME certificates up to 15-9-2024 without being audited against ETSI TS 119 411-6 (the MoD has since disabled S/MIME issuance in it's CA software and hasn't issued S/MIME certificates after September 15, 2024). During this period communication took place between Logius and MoD but this misunderstanding didn't come to light until recently. Both parties involved were under the impression that the other one understood the matter at hand and that the necessary actions were being undertaken.

2. An ETSI audit is a product certification audit. If new standards are added to the scope of certification, a scope extension audit has to be performed, which is a point in time audit (i.e. from a specific date onwards, the organisation is considered to be compliant with the new standard). This must be performed before a period-of-time audit can take place with those new criteria.

MoD NL was under the impression that no full audit was required for the reasons stated in bullet 1 and didn't engage an auditor in 2023 and 2024 to perform this scope extension audit. When Logius tried to follow-up with the MoD in late 2024 about this there was some confusion at the MoD side because they thought they were compliant with the Mozilla policy (based on the incorrect interpretation of a extant CA defined in Appendix A of the SBRGs). Now confronted with this fact and after discussions with both MoD and it's auditor, the conclusion was that due to the reasons stated in bullet 2 it is not feasible within the ETSI EN 319 403-1 auditing framework and the CAB’s procedures to retroactively perform an audit with regards to SBRG compliancy in that period.

An additional factor is the fact that since there isn't a revocation deadline/misissuance at play (not in the traditional sense per se) the deadlines (for follow-up) are further removed and as such it is easier to miss things and make assumptions that all will end well in the end. That is not necessarily a root cause but more a contributing factor.

Additional Remediation

The events listed above made Logius have a hard and long look at how follow-up for these kind of long term "incidents" could be handled better. Plus there is the immediate matter at hand. For this, we propose the following (additional) measures:

Reactive

Since the ETSI TS 119 411-6 standard was new at the time, a point-in-time audit should have been conducted then. As this was not done, it is now not possible to perform a retrospective audit according to ETSI. We offer an alternative approach.

The PKIoverheid CP (Programme of Requirements, or PoR) has historically (pre-SBRGs) incorporated relevant requirements from Root Programmes (such as MRSP) directly into the CP requirements. Furthermore, the original intent of the PKIoverheid PoR was always to go beyond the "basic" requirements by implementing a more stringent policy based on risk assessment. As a result, additional requirements were imposed on our TSPs beyond the baseline set by both ETSI and Root Store Programmes.

What we can offer is a comparison between the requirements and controls listed in the PKIoverheid PoR at that time and the SBRGs ans share this in this bug. We believe that the differences are minimal, indicating a reasonable level of assurance that the necessary controls were in place to achieve a (near) equivalent level of compliance with the SBR requirements. We're currently looking into options if this comparison can be checked or signed off by a CAB.

Preventative

Change the internal incident handling process for these kind of compliance/audit related issues to include more check-in/follow-up moments. With the issue at hand no check-in moment was planned until the audit period had ended and the audit itself had taken place. A fixed interval for check-in/ups is needed and will be included for all future incidents, unrelated to which kind of deadlines (short-term or long-term) are set. A weekly check-in via email and at least one (short) call every other week will keep things on the radar and will also detect potential issues sooner
For future issuance of S/MIME (currently no PKIoverheid TSP is issuing S/MIME certificates pending new issuing CAs which will have to conform to latest version of the SBRGs) the TSP will have to fill in a full self-assessment for approval by the PA PKIoverheid. This self-assessment will take the same form as the TLS Baseline Requirement self-assessment. The PA PKIo will check and consider this and all other evidence with regards to MRSP or SBRG compliance. Without a satisfactory result no new issuing CA will be signed.

Mitigating
For PKIoverheid govermental CAs, name constraints will be the default going forwards with regards to S/MIME. In general we've changed the setup for issuance of future PKIoverheid CA certificates already in the sense that S/MIME and qualified will be separate certificates issued by separate intermediate CAs (for the current G3 hierarchy included in the Mozilla Root Store) of fully independent hierarchies per Root CA for the upcoming G4 hierarchies (see also Logius | Logius introduces a new generation of PKIoverheid certificates).

Regards,

Jochem

Comment 5

[Ben Wilson]

Thanks, Jochem. This is a lot to digest. To start out:

• Would you happen to have any compliance mapping or checklist that provides some form of "cross-walk" between the Programme of Requirements and the S/MIME Baseline Requirements?
• It would also be helpful to understand how TSP compliance with the Programme of Requirements equates to compliance with the S/MIME Baseline Requirements.
• What process are used to determine that your TSPs have complied with the applicable requirements concerning S/MIME certificate issuance and maintenance (i.e. review of compliance assessments performed/provided by independent third parties)?

Comment 6

[Jochem van den Berge]

Hi Ben,

Thanks for your message. We're acknowledging receipt of this message and wish to state that we're currently working on answering your questions, but that we haven't fully finished discussions with the MoD. We're intending to give a full reply by Tuesday February 19 at the latest (but we're aiming to have this finished before that). For now I'll therefore leave the needinfo flag on.

Regards,

Jochem

Comment 7

[Jochem van den Berge]

Hello Ben,

After deliberations internally and with the NL MoD I'll answer the questions to the best of our abilities.

Would you happen to have any compliance mapping or checklist that provides some form of "cross-walk" between the Programme of Requirements and the S/MIME Baseline Requirements?

The checklist/mapping that we will be using would contain the following information fields:

• S/MIME requirement
• Applicable PoR requirement
• Applicable ETSI EN 319 411-1 requirement
• Controls implemented by the NL MoD to satisfy SBR and/or PoR during audit period
• Gap/difference

The information requested would partially come from the detailed Overview of Applicability that the TSPs prepare for every ETSI audit, and would partially have to be manually filled in.

It would also be helpful to understand how TSP compliance with the Programme of Requirements equates to compliance with the S/MIME Baseline Requirements.

What we're trying to achieve is a reasonable level of assurance that most of the SBRG requirements have been met, if not formally audited by a CAB due to the reasons stated earlier by making a comparison with the different requirements matched by the controls implemented by the MoD. Where a gap exists this would be identified and we could check if this isn't mitigaged by other controls or measures.

What process are used to determine that your TSPs have complied with the applicable requirements concerning S/MIME certificate issuance and maintenance (i.e. review of compliance assessments performed/provided by independent third parties)?

Logius currently has several measures to check if TSPs are selecting the right audit criteria. We have identified a gap between these measures which led (partially) to this issue. The first is a high-level check in which the TSP indicates before an audit starts which audit frameworks are in scope of the audit. This takes the form of a checklist in which the TSP ticks off the applicable audit frameworks and other applicable policies which the PA then approves (or rejects), which we also call the overview of applicability (OoA). In this case the TSP indicated that ETSI TS 119 411-6 was in scope but interpreted this not as a requirement to perform a full SBRG audit because they ceased issuance of S/MIME on 15-9-2024. As indicated earlier, the thinking was that no audit was needed during the extant period and that the CAB would only need to check that S/MIME issuance and profiles had been disabled before or on 15-9-2024.

The other control measure is that Logius monitors the ballots of the relevant WGs of the CABF that are applicable to PKIoverheid and when needed sends out forms to the TSPs when we expect this will impact TSP issuance or compliance. This is more aimed at current changes. and as such this only applies to changes after the 1.0.0 version of the SBRGs was approved. All changes since then didn’t result in a ballot form being sent out because they a) were not applicable to the TSP in question or b) would be effective after the extant period had expired. Since no TSP was issuing certificates after the extant period (currently we're working on new issuing TSP CAs but none have been signed yet) those changes were deemed to be part of a integrated package which would need to be implemented by the TSP while configuring the new CAs in the future.

Kind regards,
Jochem

Comment 8

[Martijn Katerbarg]

Jochem,

An additional factor is the fact that since there isn't a revocation deadline/misissuance at play

Can you expand on this? I agree there’s no a case of misissuance at play here, but misissuance is not the only reason which would trigger a revocation requirement.

Off course, I understand that this is not a desired outcome, but it is one of the options to resolve such an issue.

What we can offer is a comparison between the requirements and controls listed in the PKIoverheid PoR at that time and the SBRGs ans share this in this bug.

Has PKIoverheid discussed any alternative options?

For instance, ETSI is not the only audit scheme allowed. WebTrust has a specific S/MIME audit available. Was performing a WebTrust audit instead discussed as an alternative approach, and if so, why was this not taken?

PKIoverheid’s CPS currently seems to state they use WebTrust for CAs for their audit scheme, not ETSI.

Comment 9

[Jochem van den Berge]

Hello Martijn,

Thanks for your reply. These are good points, I'll go over them in my reply.

My first statement was more of an indication that the audit visit/report that was needed to remediate this action was always planned in the future to coincide with the annual audit for MoD NL, and as such it was another action item that had a due date further in the future than say a few days or weeks. As such, and because of the perceived clarity of the issue (an period-of-time audit was needed including SBR controls via ETSI TS 119 411-6) no additional contact moments were planned during this period. The other remediation measures proposed for this issue were mainly actions to be taken by Logius hence why contact moments between Logius and the MoD with regards to this issue were not frequent which made detection of this issue much more difficult. 

As for your point about Webtrust, I must clarify the position of the CP and CPS for PKIoverheid. The CPS you refer to is only applicable to the Root and intermediate CAs (level 1+2) and indeed only refers to Webtrust. The TSP CAs and the end-user certificates are covered by the PKIoverheid Programme of Requirement (CP), which can be found on https://por.pkioverheid.nl/. The PKIoverheid PoR currently doesn't allow for WT audits for TSPs since certificate issuance has mainly been qualified certificates (eSignatures) which until recently were dual-use (eIDAS/ETSI qualified + S/MIME).

A WT S/MIME audit  (or WTCA+WTSM) makes it more difficult to satisfy the requirements of eIDAS than an ETSI audit would (even though it is probably possible). To keep the certificate practices and policies at a level playing field between the different TSPs the PA PKIoverheid currently only allows ETSI audits (for Private TLS, qualified certificates and S/MIME). All future S/MIME certificate issuance (past the extant period) will be split over different issuing CAs (or even Roots) so in the future a pure WTSM audit for S/MIME is possible, but for the current TSP CAs this wasn't considered or deemed possible. 

Of course, since we're currently talking about a gap between regular ETSI EN 319 411-1 audits and the SBRGs, the difference could potentially be bridged by performing a retroactive WTSM audit for the MoD NL. However, since WT isn't normally used by our QTSPs (with exception of some commercial TSPs which also operate other non-PKIo CAs) this will require significant more time. Aside from the audit itself, procurement procedures will have to be followed and an auditor available on short notice will have to be found.Without consulting the MoD I can't give a clear indication yet if this is possible or how much time this would entail, but as a ballpark figure I would expect 5-6 months to be a realistic figure, assuming that we can find a WT auditor and no additional snags come up.   

Regards,

Jochem

Comment 10

[Malcolm D]

There's an expectation of weekly status updates to all non-closed incidents.
There have not been recent updates to Bug #1911335
In similar cases, the lack of updates is itself cause to file a separate incident report.

Comment 11

[Ben Wilson]

I am aware that PKIoverheid is working on posting their update here.

Comment 12

[Ben Wilson]

I am continuing to work with PKIoverheid and providing them with guidance related to their preparation and submission of an Incident Report.

Comment 13

[David Weissenberg]

Hello all,

As Ben indicated Logius PKIoverheid had been working on an update over the past period. While checking in with Ben via e-mail because of a logistical issue, this turned into an in-depth discussion (instead of logistics) which should have been done here in this bug instead of via e-mail. Our apologies for that.
To make things easier (since this bug has been open for a while), we've opted to write a full incident report using the new CCADB incident template.

Full Incident Report

---

Summary

• CA Owner CCADB unique ID: A000068

• Incident Description: In July 2024 it became clear to Logius that one of its TSPs (the Dutch Ministry of Defence, NL-MoD) wouldn't be able to obtain a valid S/MIME audit statement for the period beginning September 1, 2023, till September 15, 2024. Due to the fact that PKIoverheid is a "Super-CA" it is also important that issuing CAs (which are operated by external parties) have a valid audit statement on file which covers the relevant required standards. While the Logius Part (Root CA and level 2 CAs, audited bases on WTCA, WTNS and WTSM) and the majority of the PKIoverheid TSPs managed to obtain a valid S/MIME audit statement in time, for MoD-NL this wasn't possible. For this audit delay a bug was filed here. During the remediation period (which covered both remediation through the aforementioned audit plus additional remediation) it became clear that due to the way that ETSI product audits work there should have been a scope extension audit (point-in-time) in September 2023 before a full annual audit could be performed. As such, there is limited auditing gap related to the S/MIME Baseline Requirements from their effective date until September 15, 2024.

• Timeline summary:

• Non-compliance start date: 2024-12-14 > (i.e. when the SBRG audit became overdue)

• Non-compliance identified date: 2024-07-25

• Non-compliance end date: 2024-09-15

• Relevant policies: Mozilla Root Store policy v2.9 states (emphasis mine):

[...]For the email trust bit, a CA and all intermediate CAs technically capable of issuing email certificates MUST have the following audits, with at least one of the noted policies:

• ETSI EN 319 411-1 (LCP, NCP, or NCP+); or
• ETSI EN 319 411-2 (QCP-l, QCP-l-qscd, QCP-n, or QCP-n-qscd); and,
• for audit periods ending after October 30, 2023, a period-of-time audit performed in accordance with ETSI TS 119 411-6.

• Source of incident disclosure: Self Reported

Impact

• Total number of certificates: 239.232
• Total number of "remaining valid" certificates (2025-04-15): 145.110
• Affected certificate types: S/MIME certificates only
• Incident heuristic: All certificates issued by the CA "Ministerie van Defensie PKIoverheid Organisatie Persoon CA - G3” (see https://crt.sh/?caid=132287" between September 1, 2023 and September 15, 2024
• Was issuance stopped in response to this incident, and why or why not?: N/A. This concerns an auditing issue and not certificate issuance
• Analysis: N/A
• Additional considerations: N/A

Timeline

• 2023-01-01:

  • Adoption of the S/MIME Baseline Requirements by the S/MIME Cert WG of the CA/Browser Forum. This document has a self-declared effective date of September 1, 2023.

• 2023 January through 2023 March:

  • Policy Authority (PA) PKIoverheid met several times with each TSP to assess the impact the SBRGs would have on their operations. Most TSPs indicated to issue no or few S/MIME certificates, however since their CAs have the emailProtection EKU they would need to be compliant to the S/MIME Baseline Requirements. Based on these sessions the PA developed an approach in which TSPs would:
    1. migrate to a new non-S/MIME capable CA under the current G3 Root CA or new G4 Root CA, or
    2. migrate to a new S/MIME capable CA under the current G3 Root CA which included strict S/MIME policies.

• 2023-April:

  • MoD communicated these alternatives were not an option due to operational concerns. As such, talks about MoD-specific alternatives continued over the summer and autumn of 2023.

• 2023-09-01:

  • Mozilla adopted version 2.9 of the Mozilla Root Store Policy (MRSP), which mandated compliance with the S/MIME Baseline Requirements via either Webtrust (WTSM) or ETSI EN 319 411-1/2 with ETSI TS 119 411-6.

• 2023-09:

  • Logius voiced concerns (privately and publicly via the CCADB survey) that auditing within the requirement timeframe stated on the wiki page CA/Transition SMIME BRs - MozillaWiki would be infeasible and that the audit results of all TSPs within PKIoverheid would not be available before October 2024.
  • Mozilla's wiki guidance noted that "a reasonable list of non-compliances that the CA operator (or subordinate CA operator) is not yet in compliance with" and that "Major non-compliances should be reported in Bugzilla and corrected as soon as possible" and that "Submission of a CA's S/MIME BR audit report during the second year is expected to confirm that the issues that were listed in the first S/MIME BR audit report have been resolved".

• 2023-October through 2023-December:

  • In informal and formal communications between Logius and the TSPs it was stated that S/MIME compliance should be part of the next auditing cycle. The intent for MoD-NL was specifically to include an audit statement for the period September 1, 2023 till September 2024 and an additional statement for the period September 1, 2024 till November 15, 2024 (end of the normal ETSI EN 319 411 audit period or NL-MoD)

• 2024-July:

  • After communications with the MoD, it became clear to the PA PKIoverheid that there would not be a valid audit statement for S/MIME compliance for the MoD within the time frame Logius specified in its response of the CCADB Survey of August 2023.

• 2024-08-02:

  • Original filing of this bug

• 2024-12-14:

  • SBRG audit becomes overdue.

• 2025-February:

  • During discussions with both MoD NL and the CAB it became clear that the requested remediation (e.g. execution and report on audit on the basis of ETSI TS 119 411-6) wasn't possible due to the fact that this kind of scope extension can't be included in a period-of-time audit and required a scope extension audit. Such an audit can only report on a point-in-time and wouldn't give any statement on compliance with ETSI TS 119 411-6 over the original period (2023-2024)

• 2025-02-11:

  • Update in this bug informing Mozilla about the issue discovered above and offering alternative remediation measures with regard to S/MIME compliance

• 2025-02- 11 through 2025-02-28:

  • Back and forth between Logius, Mozilla and community members seeking clarification.

• 2025-03-04:

  • Check (by e-mail) from Logius to Ben Wilson if he had time to consider the approach outlined in https://bugzilla.mozilla.org/show_bug.cgi?id=1911335#c4 and https://bugzilla.mozilla.org/show_bug.cgi?id=1911335#c7

• 2025-03-04 through 2025-03-18:

  • The initial logistical question turned into a more in-depth discussion to clarify the proposed remediation.

• 2025-03-28:

  • https://bugzilla.mozilla.org/show_bug.cgi?id=1911335#c10 was posted asking for a public update

• 2025-04-23:

  • This update is posted on Bugzilla.

Related Incidents

None.

Root Cause Analysis

After re-visiting our original Root Cause analysis and after careful analysis, we conclude that the root cause of this compliance issue is a breakdown of communication between the parties involved (Logius, MoD-NL and CAB). Over the period from adoption of the SBRGs and the recent events it became clear that due to insufficient understanding and information certain choices or assumptions were made which weren't sufficiently challenged by the other parties. Contributing factors were of incorrect interpretation of (ETSI) auditing practices, too much of a focus on the internal CP change process and too much of a focus on future (technical) implementations instead of the current valid CAs.

Contributing Factor #1: Breakdown in communications between Logius, MoD-NL and CAB

• description: Although there are several contributing factors (see below), the main cause was a lack of communication and understanding between Logius and MoD-NL. When the SBRGs were adopted, Logius did act on this, but there was a failure to communicate between the parties and as such important matters were missed which caused this issue in the end. Had there been more communication and check-in moments, there would have been sufficient time to perform the required audit. Also, since the CAB of a TSP doesn't have direct contact with Logius, any kind of checks or question go via a TSP, lengthening the process and increasing the chance of errors.

• Timeline:

• 2023-01-01:

  • Adoption of the S/MIME Baseline Requirements by the S/MIME Cert WG of the CA/Browser Forum. This document has a self-declared effective date of September 1, 2023.

• 2023 January through 2023 March:

  • Policy Authority (PA) PKIoverheid met several times with each TSP to assess the impact the SBRGs would have on their operations. Most TSPs indicated to issue no or few S/MIME certificates, however since their CAs have the emailProtection EKU they would need to be compliant to the S/MIME Baseline Requirements. Based on these sessions the PA developed an approach in which TSPs would:
    1. migrate to a new non-S/MIME capable CA under the current G3 Root CA or new G4 Root CA, or
    2. migrate to a new S/MIME capable CA under the current G3 Root CA which included strict S/MIME policies.

• 2023-April:

  • MoD communicated these alternatives were not an option due to operational concerns. As such, talks about MoD-specific alternatives continued over the summer and autumn of 2023.

• 2023-09-01:

  • Mozilla adopted version 2.9 of the Mozilla Root Store Policy (MRSP), which mandated compliance with the S/MIME Baseline Requirements via either Webtrust (WTSM) or ETSI EN 319 411-1/2 with ETSI TS 119 411-6.

• 2023-09:

  • Logius voiced concerns (privately and publicly via the CCADB survey) that auditing within the requirement timeframe stated on the wiki page CA/Transition SMIME BRs - MozillaWiki would be infeasible and that the audit results of all TSPs within PKIoverheid would not be available before October 2024.

• 2023-October through 2023-December:

  • In informal and formal communications between Logius and the TSPs it was stated that S/MIME compliance should be part of the next auditing cycle.

• 2023-12-31:

  • End of 2023 audit cycle

• 2024-04-01:

  • Last day to provide SBRG audit report

• 2024-July:

  • After communications with the MoD, it became clear to the PA PKIoverheid that there would not be a valid audit statement for S/MIME compliance for the MoD within the time frame Logius specified in its response of the CCADB Survey of August 2023.

• Detection: This was realised during the writing of this update, while earlier posts were more focused on other contributing factors

• Interaction with other factors: Mainly with #3 and #4

Contributing Factor #2: Too much focus on S/MIME issuance from new CAs)

• Description:

The main focus of PKIoverheid has been on solutions migrating the user base of its TSPs to either single purpose S/MIME CAs with strict policies, or moving to non-S/MIME capable CAs. Arranging a timely Period-of-Time (PoT) S/MIME audit outside the regular audit cycle was regarded as trivial and something still in the future.

• Timeline:

• 2023-01-01:

  • Adoption of the S/MIME Baseline Requirements by the S/MIME Cert WG of the CA/Browser Forum. This document has a self-declared effective date of September 1, 2023.
  • Logius' annual audit period begins, with the regularly-scheduled end of the audit period on December 31, 2023

• 2023 January through 2023 March:

  • Policy Authority (PA) PKIoverheid met several times with each TSP to assess the impact the SBRGs would have on their operations. Most TSPs indicated to issue no or few S/MIME certificates, however since their CAs have the emailProtection EKU they would need to be compliant to the S/MIME Baseline Requirements. Based on these sessions the PA developed an approach in which TSPs would:
    1. migrate to a new non-S/MIME capable CA under the current G3 Root CA or new G4 Root CA, or
    2. migrate to a new S/MIME capable CA under the current G3 Root CA which included strict S/MIME policies.

• 2023-09-01:

  • Effective date of MRSP v2.9 mandating SBRG compliance
  • Mozilla guidance states that period-of-time audits must be performed for audit periods ending after October 30, 2023

• October 2023 till now:

  • PKIoverheid CAs have signed new (non-S/MIME) CAs but legacy population and non S/MIME issuance is still here for now (new CAs need to be trusted by other government agencies due to a switch to private CAs)

• 2023-12-31:

  • End of 2023 audit cycle

• 2024-04-01:

  • Last day to provide SBRG audit report

• Detection: This was only realised as a contributing factor around the time this bug was created (so 2024-08-02)

• Interaction with other factors: None

Contributing Factor #3: ETSI auditing practices

• Description: Logius operates as a "Super-CA" in which the Root CA and level 2 "domain" CAs are operated by Logius and level 3 (issuing) CAs are operated by external parties (Trust Service Providers). Logius has opted to use Webtrust as an auditing framework for <u>its</u> CAs while the TSPs use ETSI. This is due to the fact that TSPs mainly issue EU Qualified certificates (per eIDAS) for which ETSI offers a better integration than Webtrust does. However, this does result in a limited amount of experience and knowledge of auditing practices and as such it was assumed that a Period-of-Time audit was possible when new auditing standards were introduced (in this case ETSI TS 119 411-6) like Webtrust. The fact that a Scope Extension Audit was required was not clear until a meeting with MoD-NL and their CAB.

• Timeline:

• 2023-01-01:

  • Adoption of the S/MIME Baseline Requirements by the S/MIME Certificate WG of the CA/Browser Forum. This document has a self-declared effective date of September 1, 2023.
  • Logius' annual audit period begins, with the regularly-scheduled end of the audit period on December 31, 2023.

• 2023-09-01:

  • Effective date of MRSP v2.9 mandating SBRG compliance.

• 2023-12-31:

  • End of 2023 audit cycle.

• 2024-04-01:

  • Last day to provide SBRG audit report.

• 2025-February:

  • During discussions with both MoD NL and the CAB it became clear that the requested remediation (e.g. execution and report on audit on the basis of ETSI TS 119 411-6) was not possible due to the fact that this kind of scope extension cannot be included in a Period-of-Time audit and required a Scope Extension audit. Such an audit can only report on a Point-in-Time and would not give any statement on compliance with ETSI TS 119 411-6 over the original period (2023-2024).

• Detection: This only came up in a meeting between MoD-NL, Logius and the CAB in February 2025 when it was explained that a scope extension audit should have been performed in September 2023.

• Interaction with other factors: This issue had the same origin as contributing factor #2.

Contributing Factor #4: Auditor Unavailability

• Description: NL-MOD requires that all personnel, including Conformity Assessment Body auditors, must be vetted by the Dutch Military Intelligence and Security Service (MIVD) in the Netherlands. This is a laborious process especially for foreign entities, including auditors. The Netherlands currently has only one accredited and vetted CAB (BSI). As indicated in https://bugzilla.mozilla.org/show_bug.cgi?id=1911335#c0 this led to the issue that it was not possible to follow the guidance set out by Mozilla with regard to initial S/MIME audits. Had this been possible, this whole issue might have been detected in time.

• Timeline:
  2023-01-01:

• • Adoption of the S/MIME Baseline Requirements by the S/MIME Certificate WG of the CA/Browser Forum. This document has a self-declared effective date of September 1, 2023.
  • Logius' annual audit period begins, with the regularly-scheduled end of the audit period on December 31, 2023.

• 2023 January through 2023 March:

• • Policy Authority (PA) PKIoverheid met several times with each TSP to assess the impact the SBRGs would have on their operations. Most TSPs indicated to issue no or few S/MIME certificates, however since their CAs have the emailProtection EKU they would need to be compliant to the S/MIME Baseline Requirements. Based on these sessions the PA developed an approach in which TSPs would:
    1. migrate to a new non-S/MIME capable CA under the current G3 Root CA or new G4 Root CA, or
    2. migrate to a new S/MIME capable CA under the current G3 Root CA which included strict S/MIME policies.

• 2023-April:

• • MoD communicated these alternatives were not an option due to operational concerns. As such, talks about MoD-specific alternatives continued over the summer and autumn of 2023.

• 2023-09-01:

  • Mozilla adopted version 2.9 of the Mozilla Root Store Policy (MRSP), which mandated compliance with the S/MIME Baseline Requirements via either Webtrust (WTSM) or ETSI EN 319 411-1/2 with ETSI TS 119 411-6.

• 2023-09:

  • Logius voiced concerns (privately and publicly via the CCADB survey) that auditing within the requirement timeframe stated on the wiki page CA/Transition SMIME BRs - MozillaWiki would be infeasible and that the audit results of all TSPs within PKIoverheid would not be available before October 2024.

• 2023-12-31:

  • End of 2023 audit cycle.

• 2024-04-01:

  • Last day to provide SBRG audit report.

• Detection: This only came up in a meeting between MoD-NL, Logius and the CAB in February 2025 when it was explained that a scope extension audit should have been performed in September 2023.

• Interaction with other factors: This Contributing Factor was detected at the same time as #2, and if "fixed" earlier would have brought to light #2 at a much earlier (and fixable) time.

Contributing Factor #5: too much of a focus on the internal CP change process

• Description: PKIoverheid historically had a laborious process in which changes for the PKIoverheid Certificate Policy were analysed and sent to TSPs via email and then discussed in person. During 2023/2024 we have made the switch to using GitHub in which TSPs are party to discussions and changes. This increased agility greatly but the focus was too much on working with the new system and at the same time simplifying our CP (de-duplicating requirements from several ETSI standards for instance). While that has greatly improved our agility with regular (internal) changes there was insufficient attention for external (Mozilla/CABF) changes and the implications on TSP practices.

• Timeline: No specific events but this contributing factor kicked off during the summer of 2023.

• Detection: This was only realised when this bug originally came up (so 2024-08-02).

• Interaction with other factors: Together with #1 this diverted attention away from interpreting the implications of the SBRGs in a timely fashion.

Lessons Learned

What went well

• Logius was (and is) in active discussions with TSPs to analyse the impact of the S/MIME Baseline Requirements on their operations and is actively moving to migrate TSPs to CAs either not publicly trusted or to CAs which are single purpose S/MIME CAs (e.g. only S/MIME strict). This will limit the scope of future webPKI (S/MIME) certificates to a select few issuing CAs (removing dual-use certificates)

What didn't go well

• Our focus was too much on the technical/future use cases and did not take into account the implications for current still valid CAs (“extant CAs”) and their auditing and compliance requirements in enough detail. Originally there was an expectation that all S/MIME-capable CAs would cease issuance and migrate to new CAs before September 1st 2024.
• Between September 1st, 2023 and July 2024 there was insufficient attention from the PA to check current S/MIME compliance preparations and consequently the commitment the PA PKIoverheid gave in the August 2023 CCADB survey. Although tickets/reminders are automatically created for the compliance officer to check TSP audit preparations and results in their yearly audit cycle, these checks were not implemented for the (additional) S/MIME PoT audits.
• When proposing the original remediation (see https://bugzilla.mozilla.org/show_bug.cgi?id=1911335#c0) the actual audit itself wasn't listed as a deliverable/remediation, and it was assumed that this would be a non-issue until after the actual regular audit had already been performed.
• Although PKIoverheid TSPs can use any ETSI auditor accredited by an EER NAB, in practice due to constraints like the one mentioned here, there is currently only one auditing firm available as a viable option for the NL MoD.

Where we got lucky

Not applicable.

Action Items

The items below have been updated based on developments and responses in this bug. Since a lot of these include procedural and/or operational changes it is difficult to attach external Evaluation Criteria to them.

Action Item	Kind	Corresponding Root Cause(s)	Evaluation Criteria	Due Date	Status
Exploration of Sourcing/mandating a backup MoD/TSP auditor	Mitigate	#4	Requirement for ad-hoc audit written into contract with TSPs	2025-01-30	Done
Dedicated GitHub repo for compliance with auto-import from MRSP/CABF	Prevent/Detect	#5	Repo created and ballots/RFCs are processed (see example as attachment)	2024-08-30	Done
Investigating if it is to auto-assign GitHub issues to TSP key-user	Prevent/Detect	#5	Currently output from the measure above is manually send to a TSP for verification.	2024-10-01	Done
Execution and reporting result of S/MIME (ETSI TS 119 411-6) audit	Reactive	N/A	Not possible (see details above)	January 2025	Won't do
Separation of issuing hierarchies in S/MIME/non-S/MIME CAs	Mitigate	N/A	This will increase focus and remove the mixed-use of certificates which makes audit scope and requirements more clear. New Root CAs have been created and TSPs are currently migrating to these. Full remediation is not foreseen until 2028.	2028	Done
Analysis of gaps between different applicable audit criteria	Reactive	N/A	Attachment of the result of said analysis to this bug.	2025-04-25	In progress
Independent review of the audit gap analysis by a third party	Reactive	N/A	Independent review of the deliverable of the action item above, to be posted as a short analysis in this bug	TBD	To do
Implementation of SBRG/MRSP self-assessment (one-off and annual recurrence)	Prevent/detect	#2	Template will be shared here.	2025-05-01	To do
Strenghten TSP supervision by performing periodic site-visits (combined with audit kick-off meeting)	Prevent	#1, #3 and #4	N/A	2025-06-01	to do
Introduce monthly one-on-one sessions with TSPs to discuss operational issues	Detect	#1	N/A	2025-05-01	to do

Appendix

Not applicable.

Details of affected certificates

"Ministerie van Defensie PKIoverheid Organisatie Persoon CA - G3” (see https://crt.sh/?caid=132287)

Comment 14

[David Weissenberg]

Hello all,

An update from our side with regards to several action items:

• The gap-analysis has almost been concluded. As expected, this analysis turned out to be a lot of work. In the first version of the analysis the substantiation of the identified gaps is not always obvious for public not knowledgeable in the specifics of PKIoverheid and/or the NL-MoD PKI. As such, we are currently hard at work to present an updated version of the analysis which we hope to share by May 16 at the latest. In the meantime we would like to share an initial summary of the findings.

• Before sharing the preliminary results we would like to give a bit more of context with regards to this CA. NL-MoD, unlike other PKIoverheid S/MIME TSPs, issues S/MIME certificates purely for NL-MoD employees. As such, all S/MIME issuance is bound to a single domain (mindef.nl) and issuance is strictly controlled and tied into IAM procedures at NL-MoD. As indicated earlier, NL-MoD has ceased issuance of S/MIME certificates at the end of the extant CA period and hasn't resumed S/MIME certificate issuance under a new TSP CA since.  As for the preliminary analysis results themselves, in total roughly thirteen (13) areas gaps have been found between the SBR and existing controls or audit criteria. Some of these are actual cases of non-compliance, in other cases the implementation was in conformance with the SBRGs but there just was no other audit standard or the CP/CPS of NL-MoD which required this. Since a gap in requirement/audit coverage still applied we have marked these issues as such, even if no actual non-compliance was found. 

• The areas where gaps were found are:

  • Chapter 2. PUBLICATION AND REPOSITORY RESPONSIBILITIES: Several requirements with regards to either update frequency, SBRG policy, or technical re quirements were not described in the CPS of NL-MoD even though some were indeed implemented (total of 3 gaps, in section 2.2, 2.3 and 2.4).

  • Chapter 3. IDENTIFICATION AND AUTHENTICATION:  Requirements from section 3.2.2 (and subsections) were not described in the CPS (in sufficient detail) and/or not implemented as such in a literal sense. Since NL-MoD operates as an Enterprise RA and as indicated earlier only issues S/MIME certificates tied to employee e-mail addresses (and as such is tightly connected to the internal IAM process) we are currently refining our analysis with more data to give a clearer picture. The outcome of this could be a few additional minor findings at most, with regards to sections 3.2.2.1, 3.2.2.2 and/or 3.2.2.3.

  • Chapter 4. CERTIFICATE LIFE‑CYCLE OPERATIONAL REQUIREMENTS (specifically 4.2.2.1): The requirement with regards to re-use of validation data (e.g. not using data collected with regards to section 3.2.2.1 or 3.2.2.3) is not described in any other standard. Just like in the bullet above, we are now refining our analysis with more data to give a clearer picture of the actual process currently being used. This might also result in one identified case of non-compliance with the SBRGs.

  • Chapter 7. CERTIFICATE, CRL, AND OCSP PROFILES: Gaps have been found with regards to correct use of policy identifiers and keyUsage.

  • Chapter 8. COMPLIANCE AUDIT AND OTHER ASSESSMENTS (specifically 8.7 self-audits): No  3% self-audit of issued certificates had taken place during the period in question.

  • Chapter 9. OTHER BUSINESS AND LEGAL MATTERS (specifically 9.6.1): As with earlier identified gaps, the CPS of NL-MoD did not reflect compliance with the SBRGs and as such Section 9.6.1 did not describe the warranties as required by the SBRGs fully, even though (as indicated earlier) the majority of the procedures and controls were in place. 

• With regards to the external and independent analysis of this gap-document: If there are community members who are interested in executing a review of this analysis, please reach out to us either via this bug or to me or Jochem directly.

• The self-assessment template has been completed and I've attached it to the bug. For people who know the TBR self-assessment template this might look familiar (we based it partially on that). The approach for this will be that any TSP issuing S/MIME certificates or applying for an issuing CA which is capable of issuing S/MIME certificates will have to fill out this self-assessment before a decision can be made by Logius to actually proceed with the application. This self-assessment will then have  to be repeated at least annually.

• Currently we are looking at implementing periodic operational meetings with our TSPs. We are prioritising TSPs operating in a WebPKI (e.g. S/MIME) over TSPs operating purely in a private PKI context. Currently  we are looking at introducing this measure with a starting date of June 1, 2025 .

• The remediation measure with regards to more supervision (site visits) is still in progress. We might combine this with either the audit kick-off or with the periodic meetings already in place. 

Action Item	Kind	Corresponding Root Cause(s)	Evaluation Criteria	Due Date	Status
Exploration of Sourcing/mandating a backup MoD/TSP auditor	Mitigate	#4	Requirement for ad-hoc audit written into contract with TSPs.	2025-01-30	Done
Dedicated GitHub repo for compliance with auto-import from MRSP/CABF	Prevent/Detect	#5	Repo created and ballots/RFCs are processed (see example as attachment).	2024-08-30	Done
Investigating if it is to auto-assign GitHub issues to TSP key-user	Prevent/Detect	#5	Currently output from the measure above is manually sent to a TSP for verification.	2024-10-01	Done
Execution and reporting result of S/MIME (ETSI TS 119 411-6) audit	Reactive	N/A	Not possible (see details above).	January 2025	Won't do
Separation of issuing hierarchies in S/MIME/non-S/MIME CAs	Mitigate	N/A	This will increase focus and remove the mixed-use of certificates which makes audit scope and requirements more clear. New Root CAs have been created and TSPs are currently migrating to these. Full remediation is not foreseen until 2028.	2028	Done
Analysis of gaps between different applicable audit criteria	Reactive	N/A	Attachment of the result of said analysis to this bug.	2025-05-16	In progress
Independent review of the audit gap analysis by a third party	Reactive	N/A	Independent review of the deliverable of the action item above, to be posted as a short analysis in this bug.	TBD	To do
Implementation of SBRG/MRSP self-assessment (one-off and annual recurrence)	Prevent/detect	#2	Template will be shared here.	2025-05-01	Done
Strengthen TSP supervision by performing periodic site-visits (combined with audit kick-off meeting)	Prevent	#1, #3 and #4	N/A	2025-06-01	to do
Introduce monthly one-on-one sessions with TSPs to discuss operational issues	Detect	#1	N/A	2025-06-01	to do

Comment 15

[Jochem van den Berge]

Attachment: PKIoverheid_SBRG_and_MRSP_self-assessment.xlsx

Comment 16

[Jochem van den Berge]

Logius and NL-MoD have concluded our detailed analysis which can be found as a spreadsheet attached to this bug. To give a bit of clarity we will outline some highlights below and we will reiterate some information that was given earlier to provide context:

• As indicated in our previous post, NL-MoD operates as a kind of Enterprise CA and only issued S/MIME certificates to her own employees on a smartcard (QSCD). As such, certain processes or requirements don't really fit well and this resulted in several gaps between the requirements and actual practice or other audit standards

• NL-MoD has ceased issuance of S/MIME certificates per 1-Sep-2024 and currently doesn't issue S/MIME certificates (but it still issues non-S/MIME certificates from the same CA) .

• A number of gaps have been identified between the SBR and other audit standards and/or CPS/practices. Generally speaking, these gaps can be found in the area of:

  • In section 2 the CPS of NL-MoD didn't fully reflect the SBR requirements mainly with regard to publicly stating SBRG adherence

  • Gaps in validation and identification (section 3).  As indicated above, this has mainly to do with the fact that the SBRs are written to reflect an ecosystem in which the CA and the subscriber are not affiliates or the same organization, but are both separate legal entities which engage into a business contract. As such, control of the mailbox, validation of personal data etc. is based on certain standardized methods in which a CA can, directly or indirectly, (remotely) validate the data and identity that the applicant wants to include in a certificate. Since in this case we're talking about a CA which takes most of this data straight from her own personnel systems for the issuance of qualified certificates (with S/MIME capability) only for hew own personnel these methods are not necessarily suited.  Identification of the subject is done by physical representation when the applicant appears before a NL-MoD RA employee to collect his or her access card (Defensiepas) which contains the S/MIME certificate on a QSCD.

  • Gaps in the area of certificate problem reports and processing time of revocation requests (section 4). This has to do with the lack of clear instructions for submitting certificate problem reports  (procedures for this exist internally but are not reflected in the public CPS) and a lack of information in the CPS about processing CPRs

  • A gap was identified in section 7.1.6.4. Due to a misinterpretation error, the CABF policy OIDs for the Legacy profile (which the S/MIME certificates of NL-MoD most closely resemble) was not included. The Extant CA period mentioned in Annex B was incorrectly interpreted to also encompass the non-inclusion of policy OIDs in end-user certificates (since this also could lead to potential policy chaining issues)

  • In chapter 8,1 a gap was identified with respect to self-audit/sampling (8.7 self-audits). Although a sample/audit was taken of the population by the external auditor, no self-assessment was carried out by NL-MoD itself with regards to SBRG compliance. 

  • Finally, in chapter 9, a gap was identified in the CA representations and warranties sections with regards to representations and warranties for Application Software Providers (browsers) with regards to the CPS

In general, our conclusion is that the CPS of NL-MoD is missing some key info with regards to the SBRGs and could be improved to better reflect this. The requirements/practices listed with regards to mailbox authorization, while strictly speaking are not aligned with the SBRGs, are in our opinion still sufficient to satisfy the original intent of these requirement (to have a clear permission of the subscriber, and correctly validated subject information in the certificate). Since NL-MoD has ceased S/MIME issuance with the current CA (and is, per annex of the SBRGs not allowed to issue new S/MIME certificates after 15-9-2024), and is, per our remediation measures as stated earlier mandated to go through a S/MIME self-assessment (plus a scope extension ETSI audit) before signing a new S/MIME capable CA, we believe that these kind of gaps identified will not be an issue for NL-MoD in the future.  

As an additional measure we're willing to work with the SMCWG to suggest alterations to the SBRGs to better suit these kind of use cases in which the relationship between the CA and subject is very close (affiliate or same organization), for which the current requirements, while workable, might not be a good fit.

Other remediation measures listed in our earlier posts are still being worked on, for now we have no updates with regards to those.

Comment 17

[Jochem van den Berge]

Attachment: 20250516_gap_analysis_NL-MOD-SBRG-compliancy.xlsx

Comment 18

[Jochem van den Berge]

Hello all,

A small update from our side with regards to the open remediation actions. Besides that, we're open to provide further clarification if Mozilla or the community have additional questions based on our initial gap analysis. With regards to remediation actions:

• For the upcoming ETSI audit from TSP CIBG we'll be present during the kick-off of their audit as an observer to further our understanding of ETSI auditing practices and TSP implementations of PoR and ETSI/SBR/Mozilla requirements
• For all TSPs we've initiated meeting requests for a periodic meeting discussing compliance, operations and CP matters.

With that, the status of the remediation actions will be as follows:

Action Item	Kind	Corresponding Root Cause(s)	Evaluation Criteria	Due Date	Status
Exploration of Sourcing/mandating a backup MoD/TSP auditor	Mitigate	#4	Requirement for ad-hoc audit written into contract with TSPs.	2025-01-30	Done
Dedicated GitHub repo for compliance with auto-import from MRSP/CABF	Prevent/Detect	#5	Repo created and ballots/RFCs are processed (see example as attachment).	2024-08-30	Done
Investigating if it is to auto-assign GitHub issues to TSP key-user	Prevent/Detect	#5	Currently output from the measure above is manually sent to a TSP for verification.	2024-10-01	Done
Execution and reporting result of S/MIME (ETSI TS 119 411-6) audit	Reactive	N/A	Not possible (see details above).	January 2025	Won't do
Separation of issuing hierarchies in S/MIME/non-S/MIME CAs	Mitigate	N/A	This will increase focus and remove the mixed-use of certificates which makes audit scope and requirements more clear. New Root CAs have been created and TSPs are currently migrating to these. Full remediation is not foreseen until 2028.	2028	Done
Analysis of gaps between different applicable audit criteria	Reactive	N/A	Attachment of the result of said analysis to this bug.	2025-05-16	Done
Independent review of the audit gap analysis by a third party	Reactive	N/A	Independent review of the deliverable of the action item above, to be posted as a short analysis in this bug.	TBD	To do
Implementation of SBRG/MRSP self-assessment (one-off and annual recurrence)	Prevent/detect	#2	Template will be shared here.	2025-05-01	Done
Strengthen TSP supervision by performing periodic site-visits (combined with audit kick-off meeting)	Prevent	#1, #3 and #4	N/A	2025-06-01	Done

Comment 19

[incident-reporting]

Should a "Next update" be set on this bug?

Comment 20

[Jochem van den Berge]

Currently we’re waiting on the community/Mozilla to independently verify/review our analysis (remediation measure #7). As such, we can’t really put a date to that since we’re not performing the review. The only other update we can report is that TSP operational meetings/site visits (remediation measure #9) are currently happening and we still intend to attend a TSP ETSI audit kick-off but the first opportunity for that is not until August.

Comment 21

[Ben Wilson]

I have reviewed Comment #16 and Comment #18, and the detailed gap analysis of the audit review criteria posted in Comment #17, as well as the identified deficiencies in the CPS documentation with respect to the S/MIME Baseline Requirements. Based on my review, I believe PKIoverheid’s analysis of gaps between the S/MIME BRs and the applicable CPS documentation was sufficiently thorough and appropriately scoped, which facilitated the review.

As expected, the analysis clearly identified the anticipated deficiencies, particularly in areas such as mailbox validation and certificate issuance practices under the S/MIME BRs, which were not adequately addressed in the CPS documentation. I also acknowledge that issuance of S/MIME certificates under the extant CA ceased in September 2024.

Given the measures already taken, the additional steps implemented to address these issues, and PKIoverheid’s future plans regarding the implementation of the S/MIME BRs, this bug now appears to be approaching the point where it may be considered for closure.

Comment 22

[Jochem van den Berge]

Thank you, Ben, for your review. With that in mind we would like to submit our Closure Statement below.

Report Closure Summary

Incident description:
Between September 1, 2023, and September 15, 2024, the issuing CA "Ministerie van Defensie PKIoverheid Organisatie Persoon CA - G3” operated by the Dutch Ministry of Defence (NL-MoD) lacked a valid audit report attesting to compliance with the S/MIME Baseline Requirements (SBRGs), as required under Mozilla Root Store Policy v2.9. The audit gap arose from the absence of a scope extension audit under ETSI TS 119 411-6 following the effective date of the SBRGs, which resulted in in a temporary compliance deficiency for S/MIME issuance by NL-MoD due to the fact an annual ETSI 319 411-1 audit can’t be performed over a modified scope (e.g. new audit standards)

Incident Root Cause(s):
The primary root cause was a breakdown in communication and misalignment of expectations between Logius (as Super-CA), the NL-MoD, and their Conformity Assessment Body (CAB) with regards to auditing. Contributing factors included incorrect interpretation of “extant CA” definitions, a lack of familiarity with ETSI audit processes by the Super-CA (which required a point-in-time scope extension audit), limited auditor availability due to vetting requirements, overemphasis on future CA configurations rather than existing ones, and delayed detection of audit readiness gaps.

Remediation description:
Logius, in coordination with NL-MoD, ceased S/MIME issuance from the affected CA before the end of the extant CA period and performed a detailed gap analysis comparing the SBRGs against existing controls and CP/CPS documentation. Additional operational and procedural improvements were implemented, including the use of a compliance GitHub repo with automated issue tracking, a new self-assessment process for TSPs applying for or operating S/MIME-capable CAs, and tighter audit planning and monitoring by Logius. Where retrospective ETSI audit remediation was infeasible, Logius provided detailed evidence demonstrating near-equivalence of existing practices to SBRG controls.

Commitment summary:
Ongoing commitments beyond the completed action items include:

• Execution of the SBRG/MRSP self-assessment for all S/MIME-capable TSPs prior to (re)starting S/MIME issuance, to be repeated annually thereafter
• Continued refinement of audit scope management processes and advance audit planning.
• Participation in SMCWG discussions to improve the alignment of SBRGs with enterprise/internal CA models.
• Ongoing TSP supervision through site visits, audit kick-off participation, and recurring one-on-one compliance meetings.
• Simplified set-up going forward for new S/MIME capable CAs focused on single use hierarchies based on the S/MIME strict profile, thus reducing complexity and also allowing Webtrust S/SMIME audits for those CAs, offering more flexibility.

All Action Items disclosed in this report have been completed as described, and we request its closure.

Comment 23

[incident-reporting]

This is a final call for comments or questions on this Incident Report.

Otherwise, this bug will be closed on approximately 2025-08-08.