1718554 - Actalis: Delayed revocation of non-BR-compliant CA Certificate within 7 days

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Adriano Santoni]

1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in the MDSP mailing list, a Bugzilla bug, or internal self-audit), and the time and date.

This bug is related to Actalis not revoking within 7 days, as per the BR, the Subordinate CA certificate affected by bug 1717357.

2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.

See bug 1717357.

3. Whether your CA has stopped, or has not yet stopped, certificate issuance or the process giving rise to the problem or incident. A statement that you have stopped will be considered a pledge to the community; a statement that you have not stopped requires an explanation.

See bug 1717357.

4. In a case involving certificates, a summary of the problematic certificates. For each problem: the number of certificates, and the date the first and last certificates with that problem were issued. In other incidents that do not involve enumerating the affected certificates (e.g. OCSP failures, audit findings, delayed responses, etc.), please provide other similar statistics, aggregates, and a summary for each type of problem identified. This will help us measure the severity of each problem.

See bug 1717357.

5. In a case involving certificates, the complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem. In other cases not involving a review of affected certificates, please provide other similar, relevant specifics, if any.

See bug 1717357.

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

As to how and why the mistakes were made, see bug 1717357.

7. List of steps your CA is taking to resolve the situation and ensure that such situation or incident will not be repeated in the future, accompanied with a binding timeline of when your CA expects to accomplish each of these remediation steps.

The offending certificate was issued for a technically constrained Subordinate CA - under the Actalis Root - that Actalis has been operating for AgID since 2017, where AgID (Agenzia per l'Italia Digitale) is the Italian governmental authority overseeing the Italian digital agenda, under the Presidency of the Council of Ministers [1]. AgID is also the Italian national supervising body according to article 17 of the eIDAS Regulation [2].

That Subordinate CA (commonName: "AgID CA1") is a technically constrained CA dedicated to AgID, mainly used for issuing S/MIME certificates. It has also been used for issuing some TLS (SSL Server) certificates, limited to AgID domains (as imposed by name constraints), and a few client certificates.

The certificate issued by Actalis for "AgID CA1" in October 2020 has been found not to comply with the BR, in that since v1.7.1 (August 2020) the BR forbids Subordinate CA certificates to include both serverAuth and emailProtection in the EKU extension, even for technically constrained CAs. Actalis should therefore revoke the offending "AgID CA1" certificate within 7 days of being made aware of the problem (that is, by today 28 June).

In an effort to act responsibly towards all stakeholders, Actalis decided to delay the revocation of the offending Subordinate CA certificate after having considered the following.

While the immediate cessation of TLS certificates issuance under such CA is not a problem (in fact it has been already implemented, and all residual TLS certificates were revoked today), the situation is completely different as regards S/MIME certificates. The S/MIME certificates issued by such Subordinated CA are in fact used to sign all PEC messages, where PEC [2] is the Italian national certified e-mail system, used by millions of users and regulated by various laws and rules. PEC is in fact regarded as a critical national infrastructure, as its use is very wide and it is mandatory in various circumstances for communications with the Public Administration.

Now, since in S/MIME messages the Subordinate CA certificate is normally incorporated into the message itself, it is not difficult but materially impossible to replace the offending Subordinate CA: it is in fact "stored" within the S/MIME messages that have been already exchanged and that are stored in the users' mailboxes. Its revocation would therefore result in the invalidation of a good part of the previous PEC messages, at least where an e-mail client is used that checks the status of the chain. By just considering the sheer volume of messages exchanged in 2020, that is more than 2 Billions [3], it is clear that the impact would be huge. The PEC system is so critical for Italy that the main PEC providers have been included among the subjects referred to in Art. 1, Par. 2-bis, of the Decree-law DL 21 September 2019, n. 105 for the definition of the National Cyber Security Perimeter [4].

We evaluated the feasibility of analyzing the functionality of the PEC clients used by users, but the task immediately proved impossible. In fact, countless PEC applications are in use: in addition to standard e-mail clients (e.g. Mozilla Thunderbird, Microsoft Outlook, etc.), there are many PEC apps for both Android and iOS and many PEC "webmail" portals, whose functionalities - especially as regards certificate revocation checking - are near to impossible to figure out in a limited time frame. We know for sure, at any rate, the some widespread clients (e.g. Microsoft Outlook) do check certificate status upon opening an S/MIME message, and that is enough to conclude that a revocation would - for certain - adversely impact a huge number of users.

Not only interactive PEC applications are impacted, but also many unattended processes that leverage PEC communications, among which e-invoicing (mandatory in Italy), long term document archival services, registering of correspondence, etc.

Having considered the above, we concluded that a revocation delay was necessary to allow users sufficient time for past PEC messages to be read and used (or processed). After an analysis carried out together with AgID, we came to the conclusion that a 3-month delay constitutes a reasonable trade-off between the need to revoke the offending CA and the need to avoid a devastating impact on the national PEC system, also considering that "blocking" the offending CA for TLS certificates can be achieved by Mozilla thanks to the OpenCRL mechanism without affecting S/MIME certificates. This delay was quantified also based on the processing times (normally less than 90 days) of the main processing flows based by law on PEC communications, such as court notifications or the award of tenders, therefore a revocation delay of 3 months would allow those processes to complete while keeping the signature of the underlying PEC communication valid.

[1] https://www.agid.gov.it/en/agency/about-us
https://www.agid.gov.it/en/agency/responsibilities-and-functions

[2] https://www.agid.gov.it/en/platforms/qualified-electronic-signature/qualified-certification-service-providers

[3] https://www.agid.gov.it/it/piattaforme/posta-elettronica-certificata/statistiche-utilizzo-pec

[4] https://www.gazzettaufficiale.it/eli/id/2019/09/21/19G00111/sg

In this case, the impact of the revocation - hence the decision to delay it - does not depend on the difficulty of the users to replace the offending CA, but rather on the impossibility of doing so, a consequence of the fact that the type of application involved (S/MIME) is completely different from that (TLS) which is the subject of the BR. The fact that the certificate chain travels together with the messages (indeed within them), in a store-and-forward way, makes it completely infeasible to replace "in the field" a particular subordinate CA (which is part of a certificate chain), and therefore its revocation inevitably impacts all users; no preventive remedy is feasible in a limited time frame.

Comment 1

[Ryan Sleevi]

Kathleen: I don't believe I've seen a bug filed by Actalis asking for OneCRL, but if I'm reading Comment #0 correctly, I believe that's the request. In the event Actalis hasn't e-mailed you as you previously requested, setting a N-I so you are aware of the request.

Comment 2

[Ryan Sleevi]

Ah, sorry, I missed Bug 1717357, Comment #8 from Kathleen

Comment 3

[Ryan Sleevi]

Adriano: I appreciate the lengthy explanation in Comment #7. While I find this situation quite unfortunate, I do believe the response here is much more in the spirit and expectations of https://wiki.mozilla.org/CA/Responding_To_An_Incident#Revocation than many other past such incidents, and that's very encouraging and reassuring.

What I do believe is missing, however, is an analysis of some of the scope and impact of how many subscriber/PEC certificates we're discussing. Bug 1717357 only discusses the intermediate, but given that the Question 7 justification largely relies on the scope of impact, I would have hoped that Comment #4 would share some insight into the number of PEC certificates and why this is an issue.

Similarly, I appreciate that Actalis plans to take corrective action within 3 months, but can you share more of the plan to prevent these delays in the future? Am I understanding Bug 1717357, Comment #2 correctly that the primary corrective action is to replace this CA with one constrained to id-kp-emailProtection? I ask, because Mozilla Root Store Policy v2.7.1, Section 6.2 sets expectations for S/MIME revocations. It would seem that if another issue with PEC certificates is discovered, we might similarly face a 90 day delay in revocation, outside of Mozilla's expectations. Is anything being done long-term to mitigate that risk?

Comment 4

[Federico Ciofi]

Hello, I’m a colleague of Adriano Santoni.

The Italian national certified e-mail system allows users to send e-mails with legal value equivalent to a registered letter with return receipt. To date, the PEC system includes 18 authorized service providers [5] and for each at least one S/MIME certificate is issued (they are about 38 active S/MIME certificates).

To certify transmission and receipt of a certified email message, the email manager (provider) sends a receipt that constitutes legal proof of the transmission of the message and of any attached documentation. These S/MIME certificates allow the authentication of these receipts.
Confirming what was anticipated, the primary corrective action was to replace this CA with one new "AgID CA" constrained to id-kp-emailProtection and id-kp-clientAuth.

In S/MIME (PEC) messages the Subordinate CA certificate is normally incorporated into the message itself, its revocation would therefore result in the invalidation of the messages, at least where an e-mail client or an automatic message processing system is used that checks the status of the chain. It should be also considered that the PEC messages sent between PEC accounts are certified with a date and time stamp to show when you sent them and when they were received, with a record of receipt automatically emailed to you as an attachment. Within in Italy – though not outside it – the date and time stamp obtained through the use of the PEC system are enforceable against third parties. Unfortunately, the email clients or other PEC systems do not usually verify the e-signature on the date the message is sent/received.

Since the Italian certified e-mail system is governed by a complexity of national laws and regulations, a discussion forum with AgID and the other service providers will be necessary to identify possible alternatives actions to be taken.

[5] https://www.agid.gov.it/it/piattaforme/posta-elettronica-certificata/elenco-gestori-pec

Comment 5

[Ben Wilson]

I am closing this along with Bug #1717357 this Friday - 1-Oct-2021.