1652603 - Camerfirma: Failure to revoke within 7 days: OCSP EKU issue

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Ben Wilson]

Opening this to track incident response report for delayed revocation of OMC CA.

Comment 1

[Juan Angel Martin]

We would appreciate being able to discuss this matter in the bug https://bugzilla.mozilla.org/show_bug.cgi?id=1649944

We will incorporate the information of that bug into this one if considered necessary.

Comment 2

[Ben Wilson]

An update is needed in this bug or in Bug 1649944.

Comment 3

[Ryan Sleevi]

While this comment provided some statement, I have trouble reconciling how these controls are implemented or are at all reasonable. I'm hoping this is simply a communication issue, but I'm concerned that Camerfirma does not fully understand or appreciate the severity here.

Camerfirma folks: Please provide a holistic picture here about the timelines for all delays (including anything you're proposing to add to OneCRL, which MUST still be revoked independent of OneCRL), as well as an explanation about your system, design, and controls. As noted, an ETSI report is insufficient to provide sufficient assurance here, so you bear the burden of proof in proposing a comprehensive plan to address any concerns that relying parties have. The reasonableness of any delay is directly impacted by how well you can describe why it's impossible for there to have been misissuance in the past, how you're preventing that currently, and how relying parties can be certain going forward that the risk, to them, is justified.

I understand that you're viewing risk as "risk to your customers and/or business", but that's not the right measure of risk. The risk is to Subscribers/Relying Parties accepting invalid certificates, and you need to demonstrate comprehensively how that's being mitigated.

Importantly, and as part of this, please discuss your comprehensive plan to prevent any future delays in revocation. It's naturally deeply concerning that there's a proposed 6 month delay in replacing intermediates. That cannot be seen as consistent with the Baseline Requirements, and so the plan needs to address how it will be resolved.

Comment 4

[Juan Angel Martin]

While this comment provided some statement, I have trouble reconciling how these controls are implemented or are at all reasonable. I am hoping this is simply a communication issue, but I'm concerned that Camerfirma does not fully understand or appreciate the severity here.

Ryan, thank you for your insights. From Camerfirma we clearly understand the risk and that it is only avoidable with the destruction of the private key of the non-compliant CA guarded by Camerfirma's HSM's.

Camerfirma folks: Please provide a holistic picture here about the timelines for all delays (including anything you're proposing to add to OneCRL, which MUST still be revoked independent of OneCRL), as well as an explanation about your system, design, and controls. As noted, an ETSI report is insufficient to provide sufficient assurance here, so you bear the burden of proof in proposing a comprehensive plan to address any concerns that relying parties have. The reasonableness of any delay is directly impacted by how well you can describe why it's impossible for there to have been misissuance in the past, how you're preventing that currently, and how relying parties can be certain going forward that the risk, to them, is justified.

We have worked with our client and the Spanish national supervisor to minimize as much as possible the transition time to the new intermediate CA, focusing on the maintenance of certificates for critical health procedures that affect the health of patients.

Our plan of action is as follows.

2020-03-02 Issuance of the new intermediate CA compliant that replaces the current one. This intermediate CA does not belong to the Camerfirma hierarchy.
2020-07-17 Accreditation process initiated with the national supervisor
2020-08-03 certificates associated with non-critical services are no longer issued with the non-compliant intermediate CA.
2020-09-04 Incorporation of the new intermediate CA in the EU-TSL. Forecast based on meetings held with our client and the NSB Supervisor.
2020-09-04 Beginning of the replacement process of the health cards that store the certificates issued by this non-compliant intermediate CA.
2020-09-11 distribution of 20% of the critical health cards and issuance of new certificates by the new non-compliant intermediate CA
2020-09-18 distribution of 80% of critical health cards and issuance of new certificates by the new compliant intermediate CA
2020-09-25 distribution of 100% of critical health cards and issuance of new certificates by the new compliant intermediate CA
2020-10-02 revocation of the intermediate CA non-compliant and destruction of its private key.

During this period controls will be increased to avoid the commitment of the intermediate CA to allow taking advantage of the vulnerability and the issuing of incorrect OCSP responses. (https://bugzilla.mozilla.org/show_bug.cgi?id=1649944#c4)

We must take into account that we currently have the following controls:

• Only reliable Camerfirma personnel is authorized to operate with this intermediate CA's private key that is stored in HSMs with a FIPS 140-2 level 3 configuration.
• To perform any manual operation with this CA's private key, the participation of persons belonging to Camerfirma's reliable personnel is required.
• These operations can only be carried out within a high security zone with access restricted to reliable and authorized personnel belonging to Camerfirma.
• 2 people are required to access this zone and 4 people are required to operate with this password. All of them belonging to reliable and authorized personnel belonging to Camerfirma.
• The access to the HSM is protected by firewalls and the only software that can access that key is Camerfirma's own CA software. This software is not designed to perform OCSP response signing and is protected in such a way that any alteration in it would generate alarms that would cause the detection of that modification.

I understand that you're viewing risk as "risk to your customers and/or business", but that's not the right measure of risk. The risk is to Subscribers/Relying Parties accepting invalid certificates, and you need to demonstrate comprehensively how that's being mitigated. Importantly, and as part of this, please discuss your comprehensive plan to prevent any future delays in revocation. It's naturally deeply concerning that there's a proposed 6 month delay in replacing intermediates. That cannot be seen as consistent with the Baseline Requirements, and so the plan needs to address how it will be resolved.

I fully agree that we must always evaluate the risk from the perspective of the relying party. From this point, we offer in our proposal the minimization of the time of risk exposure and the incorporation of additional controls that avoid taking advantage of the detected vulnerability. With regard to avoiding possible future delays, we have been working for some time now, as we have described in previous bugs, in the line of making our customers know in advance and establishing with them the necessary procedures to ensure compliance with the deadlines set by the BRs. In all our commercial agreements we reinforce the fact of unilateral revocation by Camerfirma and the establishment of contingency plans to minimize the impact that this fact may cause in the services of our clients. In each case, however, the reasons and the impact must be assessed, so that the clause can be enforced contractually and damage to third parties can be minimised. In each case, however, the reasons and the impact must be assessed, so that the clause can be enforced contractually and damage to third parties can be minimised. In this specific case, the extreme sensitivity of the services affected and the need for the new intermediate CA replacing the non-compliant intermediate CA to require the eIDAS qualification, forces us to evaluate a realistic period for the complete solution. Taking into account these factors (internal and external) and after achieving with great effort to adjust as much as possible the deadlines required for its completion, and thanks to the favourable intermediation of the Supervisory Body, we are in a disposition to commit ourselves on the basis of the proposed Plan that will culminate according to our planning on October 2 of this year.

Comment 5

[Ana Lopes]

We want to inform that the National Supervisor has already finished the accreditation process for the new CA and we continue working on the next steps of the action plan to revoke the non-compliant CA as soon as possible.

Comment 6

[Ryan Sleevi]

To perform any manual operation with this CA's private key, the participation of persons belonging to Camerfirma's reliable personnel is required.

Is this a policy control or a technical control?

Comment 7

[Ana Lopes]

We have policy and a technical controls.

On one hand, we have 4 operators and 3 system administrators who have the needed pins to perform critical operations using private keys. Each operator has different pins and the system administrators have other different ones.

Every time we need to perform any operations at least two of those operators and one of the system administrators are required and none of them can be substituted because there is no one who knows the pins apart from them.

On the other hand, we have two procedures established, one for new CAs generation CA and another one for the CA revocation. Both of them include the steps that Camerfirma needs to follow and the involved roles and element custodian.

Comment 8

[Ana Lopes]

We want to inform that the step “2020-09-04 Incorporation of the new intermediate CA in the EU-TSL. Forecast based on meetings held with our client and the NSB Supervisor.” has been completed (inclusion approved by the Supervisor on September 7th)
We will continue working on the rest of the steps.

Comment 9

[Eusebio Herrera]

We want to inform that our customer hasn't replaced the 100% of the certificates yet, but we keep with the deathline for the next step:
2020-10-02 revocation of the intermediate CA non-compliant and destruction of its private key.

Comment 10

[Eusebio Herrera]

Today we have finished the last task of the plan of action:
2020-10-02 revocation of the intermediate CA non-compliant and destruction of its private key.

Comment 11

[Ben Wilson]

Do you have an auditor's attestation, report or letter regarding CA key destruction that can be uploaded as an attachment here?

Comment 12

[Ana Lopes]

Hi Ben,
No, we don't have any reports issued by a qualified external auditor that we can share. We have the two minutes for the two ceremonies: revocation and private key destruction elaborated by the internal auditor of Camerfirma. We only have those internal and confidential documents because we assumed that the participation of an external auditor for the elaboration of a report was not a requirement for any of them.

Comment 13

[Ben Wilson]

For the minutes of the proceedings, are those signed by internal witnesses? Can those be provided here?

Comment 14

[Ana Lopes]

Those minutes are cosidered confidential documents according to our document management.
Our proposal is asking the external auditor to include the revision of all specific documents and evidences generated related to the private key destruction that was performed and incorporate the conclusion in the report for the next audit (March 2021).
Do you think this would meet your needs?

Comment 15

[Ben Wilson]

Yes. That would be sufficient.

Comment 16

[Ben Wilson]

I intend to close this bug next week (Dec 7-11) with the understanding that Camerfirma will have its auditor address this key destruction event in the March 2021 audit report.