1696872 - FNMT: Missisuance of web site certificates without CA/Browser Forum’s reserved policy OID

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[alain]

Today March 8th (9.50am), we have realized that the required CA/Browser Forum’s reserved policy OIDs were not present in the following types of certificates:
FNMT’s OID: 1.3.6.1.4.1.5734.3.3.12.1 - EVCP (AC Administración Pública)
FNMT’s OID: 1.3.6.1.4.1.5734.3.9.16 - OVCP (AC Componentes Informáticos)
FNMT’s OID: 1.3.6.1.4.1.5734.3.9.17 - OVCP (AC Componentes Informáticos)
FNMT’s OID: 1.3.6.1.4.1.5734.3.9.18 - OVCP (AC Componentes Informáticos)
We have already suspended the issuance service for these types of certificates until we resolve the issue.
We will provide via this bug a complete incident report asap.

Comment 1

[alain]

We keep informing:
1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.
March 8th (9.50am)
Catching up on forum discussions and CA Compliance bugs, with which we had some delay, we realized that the required CA/Browser Forum’s reserved policy OID were not present in the following types of certificates:
FNMT’s OID: 1.3.6.1.4.1.5734.3.3.12.1 - EVCP (AC Administración Pública)
FNMT’s OID: 1.3.6.1.4.1.5734.3.9.16 - OVCP (AC Componentes Informáticos)
FNMT’s OID: 1.3.6.1.4.1.5734.3.9.17 - OVCP (AC Componentes Informáticos)
FNMT’s OID: 1.3.6.1.4.1.5734.3.9.18 - OVCP (AC Componentes Informáticos)

2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.
• September 30th, 2020: "Subscriber Certificates MUST include a CA/Browser Form Reserved Policy Identifier in the Certificate Policies extension".
• March 8th, 2021 – 9.50am: Identification of noncompliance issue
• March 8th, 2021 – 10.00am: Suspension of the issuance service for the related types of certificates issued by “AC Componentes Informáticos” and “AC Administración Pública”.
• March 8th, 2021 – 10.20am: Begin the identification of the affected certificates and with the notification process to subscribers in order to inform about the incidence, the need to revoke their certificates and the procedure to obtain a new one. Due to the great impact that these revocations will have on subscribers (mainly public administration) and citizens, FNMT will revoke all affected certificates within 5 days.
• March 8th, 2021 – 10.56am: Certificate profiles are updated within the required CAs and published in our website (“AC Componentes Informáticos Profiles v.1.16” and “AC Administración Pública Profiles v.2.6”
• March 8th, 2021 – 11.10am: Reactivation of issuance services.

3. Whether your CA has stopped, or has not yet stopped, certificate issuance or the process giving rise to the problem or incident. A statement that you have stopped will be considered a pledge to the community; a statement that you have not stopped requires an explanation.
Yes, on March 8th, 2021- 10:00am the issuance service was disabled until the certificate profiles had been successfully updated.

4. In a case involving certificates, a summary of the problematic certificates. For each problem: the number of certificates, and the date the first and last certificates with that problem were issued. In other incidents that do not involve enumerating the affected certificates (e.g. OCSP failures, audit findings, delayed responses, etc.), please provide other similar statistics, aggregates, and a summary for each type of problem identified. This will help us measure the severity of each problem.
• Affected certificates from "AC Administración Pública" (missing reserved CabForum OID: 2.23.140.1.1)
Total number of affected certificates: 240
Date of first affected certificate issued: 01/10/2020 17:42:38
Date of last affected certificate issued: 03/03/2021 14:58:07
• Affected certificates from "AC Componentes Informáticos" (missing reserved CabForum OID: 2.23.140.1.2.2)
Total number of affected certificates: 248
Date of first affected certificate issued: 30/09/2020 8:20:42
Date of last affected certificate issued: 01/03/2021 10:33:11

5. In a case involving certificates, the complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem. In other cases not involving a review of affected certificates, please provide other similar, relevant specifics, if any.
Please find attached 2 lists, one for "AC Componentes Informáticos" and other for "AC Administración Pública", including link to crt.sh. IDs for the affected certificates.

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.
7. List of steps your CA is taking to resolve the situation and ensure that such situation or incident will not be repeated in the future, accompanied with a binding timeline of when your CA expects to accomplish each of these remediation steps.
We are investigating the causes and evaluating possible corrective actions to be implemented so it does not reoccur. We will update the information of this incident report as soon as we obtain conclusions and adopt the appropriate corrective measures.

Comment 2

[alain]

Attachment: List of Certificates affected AC Administración Pública.xlsx

Comment 3

[alain]

Attachment: List of Certificates affected AC Componentes Informáticos.xlsx

Comment 4

[alain]

Completing incident report:
6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.
After carefully analysing the causes that led us to this situation, we may determine the following:
Since August, the compliance team’s normal activity was diminished due to personal circumstances. For this reason, the monitoring process for the CA / Browser Forum Baseline Requirements version 1.7.1 and the corresponding actuation on ballot SC31 (Browser Alignment) was not carried out.
Both our pre-issuance linting and our post-issuance test which is run hourly on crt.sh (cablint and x509lint), did not identify this incident. When reviewing forum discussions and CA Compliance bugs we realized that the certificates were missing the required CabForum OIDs.

7. List of steps your CA is taking to resolve the situation and ensure that such situation or incident will not be repeated in the future, accompanied with a binding timeline of when your CA expects to accomplish each of these remediation steps.
Date: Status - Action

1. March 9th, 2021: Completed - New protocol to strengthen the compliance surveillance of all the applicable requirements (BRs, EV Guidelines, eIDAS, ETSIs, RFC, Root Programs and related national regulation) with the participation of another department and consisting of:
   a. Peer review: Two members of the compliance team from CERES Department, shall review all applicable requirements and shall generate a monthly report with all changes that have occurred, indicating where appropriate the relevant dates to be taken into consideration. This report shall be electronically signed by both reviewers.
   In the event any action shall be implemented in order to comply with any new requirement, the compliance team will call an extraordinary meeting of the TSP Management Committee, following the already established procedure.
   b. This monthly report shall be then reviewed by the FNMT’s Security Information Department, who will be to in charge to confirm that latest versions for BRs, EV Guidelines, eIDAS, ETSIs, RFC, Root Programs and related national regulation have been duly checked.
   c. Quarterly in ordinary meeting, these reports will be presented to the TSP Management Committee.
2. March 9th, 2021: In progress (estimated June 2021) – Technical controls:
   In addition to this new protocol, and since both the pre-issuance and post-issuance linters may not be up to date enough with the latest requirements, the TSP Management Committee has approved the establishment of a proprietary control which will be developed to complement the test that is already executed hourly on the crt.sh linters.
   This new control will be fed with the monthly reports generated, so that any new requirement identified and involving any change in the certificate profiles will be integrated into this control.

This bug will be updated to confirm revocation of all affected certificates

Comment 5

[alain]

Today all affected certificates have been revoked (last certifiacate revoked at: 2021-03-13 08:45:11 UTC)

Comment 6

[Ben Wilson]

I will schedule this to be closed on or about next Friday, 26-Mar-2021, unless other issues or questions need to be resolved.