1559765 - Izenpe: Multiple invalid EV certificates issued

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Ryan Sleevi]

Multiple invalid EV certificates have been detected as issued by Izenpe. These certificates violate the EV Guidelines by virtue of using the ETSI ESI TS 119 412-1 profile specifying the PSD2 certificate, using a qcStatement-1 or qcStatement-2 to express a SemanticsIdentifier referring to either id-etsi-qcs-SemanticsId-Legal or id-etsi-qcs-SemanticsID-Natural, asserting the id-kp-serverAuth EKU within the extendedKeyUsage extension, and asserting an EV policy.

These certificates violate the EV Guidelines, as discussed in the CA/Browser Forum and within the mozilla.dev.security.policy community. CA/Browser Forum Ballot SC17 attempts to resolve this, but has not yet completed IP review, and further details additional requirements to impose beyond those specified in TS 119 412-1.

Please provide an incident report for these affected certificates, as described at https://wiki.mozilla.org/CA/Responding_To_An_Incident . Please include within your analysis of this incident your analysis of the past discussions, why these were not followed or engaged with prior to issuing these certificates, and include within the scope of remediation steps to address such oversight going forward.

Comment 1

[Ryan Sleevi]

Example certificate: https://crt.sh/?id=1544633200

Comment 2

[Oscar Garcia]

Hi,
First of all, thanks for your advice, once we have been notified this problem via bugzilla and while we analyze the problem, the first step given has been to stop the issuance of any EV certificate.
We will proceed identifying all the certifcates affected and notifying it to our customers to revoke them all.

We will provide a full detail of the problem and measures taken in the following hours.

Regards,

Comment 3

[Oscar Garcia]

Hi,
after some internal discussions, we are not completely sure about the problem and we would like to ask for a couple of clarifications.
The first thing is about the PSD2 certificates as long as we do not issue this type of certificates, why have they been marked as "PSD2" certificates?.
The second thing is the line we have followed. Including in the certificate the qc-statement2 refering to qcs-SemanticsId-Legal,
we must have included the "OrganizationIdentifier" in the subject, but in this profile we don't have that field.
If the EV violation comes by this last one, we have gathered all the information as long as we have fixed it and we are ready to publish.

Regards

Comment 4

[Ryan Sleevi]

Thanks for commenting publicly! [As an aside: For those following this bug, please note I simultaneously sent an e-mail to their problem reporting email, for which under the BRs they could have used to privately provide their analysis of the incident report]

Apologies for the confusion, I used PSD2 as a short-hand, as I was investigating PSD2 certificates, but as you point out, that's not a correct terminology for this profile.

My understanding was that the violations fall into either one of two buckets:

1. CAs asserting qcs-SemanticsId-Legal and including organizationIdentifier.
2. CAs asserting qcs-SemanticsId-Legal and omitting organizationIdentifier. The only definition for the semantics of asserting qcs-SemanticsId-Legal that I'm aware of are ETSI EN 319 412-1 V1.1.1 (2016-02) and ETSI TS 119 412-1 V1.2.1 (2018-05)

For the first case, I'm not aware of any such certificates being issued by Izenpe. The only such certificate I'm aware of is https://crt.sh/?id=1580893677 , but that is from a CA not trusted by Mozilla, hence no bug filed.

For the second case, this is the substantive issue that I focused on.

My filing of the issue is based on Izenpe's SSL CPS referring to the applicable ETSI standards, as well as its SSL CP. My understanding is that the relevant profile is from Page 40 of that CP, the sede_nivel_medio_ev_eidas profile.

This profile states that it does include organizationIdentifier in a manner compliant with that of EN 319 412-3 v1.1.1 (2016-02).

I see several issues, as a result:

1. The certificates do not seem to comply with the listed Certificate Profile, in that they omit the organizationIdentifier that is described in the profile as being 3 caracteres tipo-identidad + Country + - + identificador. Por ejemplo, si usaramos el CIF sería algo como: VATES-A01337260
2. As far as I can tell, the profile listed in ETSI EN 319 412-3 requires the presence of the organizationIdentifier in the Subject (c.f. Section 4.2.1) when asserting the qcs-SemanticsId-Legal OID in qualified certificates.

This latter part is more complex, so let me explain my reasoning: While EN 319 412-1 5.4.1 states as follows:

When the legal person semantics identifier is included, any present organizationIdentifier attribute in the subject field shall contain information using the following structure in the presented order

It can be clearly read that the organizationIdentifier is optional, because the constraint only applies to 'any present' organizationIdentifier.

However, because this is qualified, the requirements from ETSI EN 319 411-2 are incorporated for QCP-w, which incorporate the profile of ETSI EN 319 412-3. EN 319 412-3 modifies/extends/"monkeypatches" 319 412-2 in Sections 4.1 and 4.2.1 to extend a requirement for the assertion of organizationIdentifier.

The consequence of this is that, in addition to being incongruent with the CP/CPS, Izenpe is asserting within the certificate extension either (or both) of:

• extensions that "do not apply in the context of the public Internet" (as interpreted as meaning IETF standards, rather than ETSI), and which it can't "otherwise demonstrate the right to assert the data in a public context" (by virtue of being in conflict with the ETSI requirements that define this extensions usage)
• "semantics that, if included, will mislead a Relying Party about the certificate information verified by the CA", by virtue of asserting QCP-w with qcs-SemanticsId-Legal, but without including the requisite organizationIdentifier required by ETSI EN 319 412-3.

I realize this is a very long description of the problem, and I'd be happy to evaluate other responses. I apologize that I didn't clarify more of the baseline analysis in the report, to allow Izenpe to evaluate these arguments. I look forward to understanding if I've overlooked important and salient details, either in the CP/CPS or within the relevant ETSI EN/TS documents.

Comment 5

[Oscar Garcia]

Hi,
Once we have cleared out where the problem was, we have started reviewing our EV certificate profiles and our certificates, to be sure they comply with CABForum, ETSI and the Spanish Ministry of Industry.
We will try to summarize what we currently have.
Izenpe’s OID: “1.3.6.1.4.1.14777.6.1.1” . This is our profile for “usual” EV certificates. No ETSI extensions on it.
Izenpe’s OID: “1.3.6.1.4.1.14777.6.1.2” . This is our EV profile according to the Spanish Ministry. No ETSI extensions on it.
Izenpe’s OID: “1.3.6.1.4.1.14777.6.1.3” . This is our new EV profile with ETSI extensions on it.
Izenpe’s OID: “1.3.6.1.4.1.14777.6.1.4” . This is our new EV profile according to the Spanish Ministry with ETSI extensions on it.
Bugzilla’s incident refered to Page 40 of our CP, this is, our “sede_nivel_medio_ev_eidas” with OID “1.3.6.1.4.1.14777.6.1.4”. In this case, there are no certificates issued under this profile (yet) because as long as the Ballot 16 is in force, we can’t include the OrganizationIdentifier. The example supplied, refered to the “ssl_qualified” profile with OID “1.3.6.1.4.1.14777.6.1.3”. In this case, we have had an error in the definition of the profile and the “qcStatement-2” shouldn’t had been included . We will mark this error as #1.
In this reviewing process, we have detected another issue related with the “OrganizationIdentifier”. Certificates issued under profile “1.3.6.1.4.1.14777.6.1.2” incorporate this attribute. According to Ballot SC16, effective April 16, this attribute shall not be included. We will marks this as error #2.

1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.

   In both cases, we were notified via Bugzilla bug on June 17, 2019

2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.

   2019/06/17 11:52 CET. Notification received from Bugzilla.
   2019/06/17 12:00 CET. Start analyzing the problem.
   We internally notify not to issue any EV certificate (they are only issued in our office) while we analyze the problem.
   2019/06/17 15:00 CET. It seems to us that the problem lays on an incorrect profile definition of the certificate. However, some of the information provided in the Bug, does not fit with the problems we detected.
   2019/06/17 17:00 CET. We changed the profile definition for “1.3.6.1.4.1.14777.6.1.3” certificates and changed our CP (to delete the qcStatement-2 ).
   We get a preliminary list of affected certificates regarding the problem with etsi extension (#1). Some of them were revoked that same day.
   2019/06/18 17:26 CET. We request some further information and clarification via bugzilla.
   2019/06/19 13:00 CET. We detected and identified that EV certificates (1.3.6.1.4.1.14777.6.1.2) were issued with “OrganizationIdentifier” after April 16 (Problem #2).
   2019/06/19 18:00 CET. We changed our CP regarding “1.3.6.1.4.1.14777.6.1.2” certificates because it didn’t reflect that the “OrganizationIdentifier” attribute was included.

3. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.

   While we have been analyzing the problem, no EV certificates have been issued. Besides, no more certificates under “1.3.6.1.4.1.14777.6.1.2” profile will be issued. The “1.3.6.1.4.1.14777.6.1.4” profile will be used instead when “OrganizationIdentifier” is allowed again.

4. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.

   Problem #1: Certificates with qcStatement-2 and no id-etsi-qcs-SemanticsId-Legal attribute (OID 1.3.6.1.4.1.14777.6.1.3)
   Total affected: 14 certificates
   First issued: 2017/12/05
   Last issued:2019/06/14

   Problem #2: Certificates with invalid attribute in subject (2.5.4.97) according to Ballot SC16 (OID 1.3.6.1.4.1.14777.6.1.2)
   Total affected: 6 certificates
   First issued: 2019/04/17
   Last issued:2019/05/10

5. The complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem.

   Problem #1:

   OID CN Date issued Status
   1.3.6.1.4.1.14777.6.1.3 test-valid-qcpw.izenpe.eus 2017/12/05 https://crt.sh/?id=271159652 revoked
   1.3.6.1.4.1.14777.6.1.3 test-ev-qualified.izenpe.eus 2018/05/16 https://crt.sh/?id=465744896 revoked
   1.3.6.1.4.1.14777.6.1.3 www.nisae.izenpe.eus 2019/05/24 https://crt.sh/?id=1505310977
   1.3.6.1.4.1.14777.6.1.3 xpamoff10.bizkaia.eu 2019/06/12 https://crt.sh/?id=1569924113
   1.3.6.1.4.1.14777.6.1.3 mta10.bizkaia.eu 2019/06/14 https://crt.sh/?id=1575590286
   1.3.6.1.4.1.14777.6.1.3 test-ev-qualified.izenpe.eus 2019/02/08 https://crt.sh/?id=1184160027 revoked
   1.3.6.1.4.1.14777.6.1.3 www.adinberri.eus 2019/06/05 https://crt.sh/?id=1544633200
   1.3.6.1.4.1.14777.6.1.3 webmail10.bizkaia.eu 2019/06/12 https://crt.sh/?id=1569917193
   1.3.6.1.4.1.14777.6.1.3 www.intaremit.es 2019/06/12 https://crt.sh/?id=1570101238
   1.3.6.1.4.1.14777.6.1.3 wordpress10.bizkaia.eu 2019/06/12 https://crt.sh/?id=1569919990
   1.3.6.1.4.1.14777.6.1.3 xpamoff11.bizkaia.eu 2019/06/12 https://crt.sh/?id=1569928771
   1.3.6.1.4.1.14777.6.1.3 imap10.bizkaia.eu 2019/06/12 https://crt.sh/?id=1569934032
   1.3.6.1.4.1.14777.6.1.3 webdav10.bizkaia.eu 2019/06/12 https://crt.sh/?id=1569936385
   1.3.6.1.4.1.14777.6.1.3 sso10.bizkaia.eu 2019/06/14 https://crt.sh/?id=1575594152

Problem #2:

OID CN Date issued Status
1.3.6.1.4.1.14777.6.1.2 www.zeanuri.eus 2019/05/10 https://crt.sh/?id=1454725140 Revoked
1.3.6.1.4.1.14777.6.1.2 www.xn--maaria-xwa.eus 2019/05/06 https://crt.sh/?id=1442678451 revoked
1.3.6.1.4.1.14777.6.1.2 www.uribekosta.eus 2019/04/24 https://crt.sh/?id=1414804123 Revoked
1.3.6.1.4.1.14777.6.1.2 sedeico.gob.es 2019/04/23 https://crt.sh/?id=1412745877 Revoked
1.3.6.1.4.1.14777.6.1.2 www.lezama.eus 2019/04/19 https://crt.sh/?id=1402148575 Revoked
1.3.6.1.4.1.14777.6.1.2 www.lea-artibai.org 2019/04/17 https://crt.sh/?id=1402276670 Revoked

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

Problem #1: Certificates with qcStatement-2 and no id-etsi-qcs-SemanticsId-Legal attribute (OID 1.3.6.1.4.1.14777.6.1.3).
The profile “1.3.6.1.4.1.14777.6.1.4” must incorporate the “qcStatement-2” according to the Spanish Ministry. When we defined the “1.3.6.1.4.1.14777.6.1.3”, our starting point was the other one and we didn’t realize that if we got rid of the “OrganizationIdentifier” attribute, there was going to be an incoherence with another extension. We started issuing these certificates some weeks ago (apart from 2 certificates issued for testing some time ago)

Problem #2: Certificates with invalid attribute in subject (2.5.4.97) according to Ballot SC16 (OID 1.3.6.1.4.1.14777.6.1.2).
We haven’t realized on time that some of our profiles were affected by ballot SC16.

7. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.

In both cases, certificates are being revoked and we expect to finish this task for tomorrow.
Problem #1. The certificate profile has already been changed and updated the information on the CP.
Problem #2. To avoid future problems regarding Ballot changes, the technical staff will meet every time a new ballot is approved.

Comment 6

[Ryan Sleevi]

Thanks! I'm going to assign this to Wayne to see if he has further questions.

In investigating this, I greatly appreciated the profile documentation provided by Izenpe. I realize that the level of documentation was itself a cause for some of the issues - misalignment in the profile definition vs practiced - but I think, as a whole, the ecosystem would benefit if more CAs followed Izenpe's lead in the level of documentation provided in these profiles.

With respect to Issue #2, you mentioned that this was related to the Spanish Ministry. Is there a profile required under local law, regulation, or government order? If so, can you provide more details? It may be that Issue #2 is a failure to abide by the steps in 9.16.3 of the BRs, but that may also offer a path forward for alternative remediation. However, I think the adoption of SC17 may provide an equitable solution.

Do you plan to update your CP/CPS if/when SC17 passes with no IPR issues? If you do, knowing a timeline for that may allow a timeline for remediation being the publication of updated profiles, as SC17 also requires the use of an additional extension if the organizationIdentifier is present.

Comment 7

[Ryan Sleevi]

Actually, after correcting some of my queries, I believe Izenpe may have missed some certificates. If I've made a mistake in my analysis, please feel free to point it out.

An example is https://crt.sh/?id=339470606 for www.sestao.eus , which asserts OID 1.3.6.1.4.1.14777.6.1.2 , and has an organizationIdentifier.

If I'm wrong, please feel free to point it out.
If I'm correct, however, please explain how this certificate was missed? Looking through CT, I see a number of other certificates not mentioned here in this incident report, so it'd be useful to understand where the difference in results is.

Comment 8

[Oscar Garcia]

Hi,
just beginning with the last point, the last certificate you mention, including the "OrganizationIdentifier", was issued before the Ballot SC16 was approved, and therefore, it could be possible to issue certificates with that attribute at that time.
All the certificates listed previously with the "OrganizationIdentifier" attribute, were issued after April 16, 2019 when the ballot SC16 came into force and any additional field to those listed in 9.2 of EV guidelines attribute was banned.

Regarding the Spanish Ministry profile definition, in this link you can find the information:
https://administracionelectronica.gob.es/pae_Home/dam/jcr:474ae40a-fe06-45fa-aa8f-113049e9c889/2016_Perfiles_de_certificados_1-ed.pdf
From page 35 to 43 we have the definition of two types of certificates ("Nivel alto" and "Nivel Medio / Sustancial"). Though it states that OrganizationIdentifier is optional, this field is mandatory for ETSI if you want to make it interoperable.
The issue #2 could have been solved in two different ways. One would had been not to issue anything until SC17 is approved, and the second one, as you comment, using BR 9.16.3. But in both cases, we should had been able to react before any certificate was issued with the OrganizationIdentifier field after April 16. Once the last ballot is approved, we will have to update our profile definition (1.3.6.1.4.1.14777.6.1.4) and notify it to the Spanish Ministry because we are forced to it if there is any change (such as those included in ballot SC17). So for the following months, only EV certificates with OID "1.3.6.1.4.1.14777.6.1.3" will be issued.

And finally, I'd like to add that all misissued certificates listed before have already been revoked.

Comment 9

[Ryan Sleevi]

Note: Even under the prior versions of the EV Guidelines, the use of the organizationIdentifier was prohibited. That is, the old text included the following:

CAs SHALL NOT include Fully-Qualified Domain Names in Subject attributes except as specified in Sections 9.2.1 and SHALL NOT include any Subject Organization Information except as specified in Section 9.2.

SC16 merely clarified that requirement in a structure compatible with the BRGs. You can see past discussion in the CA/Browser Forum, for example, on 2018-10-28. This itself was a reference to the discussions with ETSI on 2018-05-17 regarding 9.2.8 of the EVGs, for which ETSI acknowledged then the incompatibility.

Thanks for providing the details for the relevant legislation. I don't think it would qualify under 9.16.3 - as you note, it's optional, and not required, and as best I can tell, is a voluntary scheme.

In any event, recognizing that the explanation is not fully in-line with what was discussed in the CA/Browser Forum for over a year, I still defer to Wayne, as captured in Comment #6.

Comment 10

[Wayne Thayer]

Oscar: I agree with Ryan that ballot SC16 was a clarification of the existing requirements. Was Izenpe aware of the discussions that Ryan highlighted? Is it still Izenpe's assertion that the certificates issued prior to ballot SC16 were not misissued?

Comment 11

[Oscar Garcia]

Hi,
First of all, sorry for the delay in answering, the main reason for this, it's that we have been messing around trying to find the reason why we included the OrganizationIdentfier.
And so far, we have no answer for that. The only explanation for this is a misunderstanding from our side of EVGL 9.2.9. which has led to some discussions as you pointed in your post and finally were redefined without any ambiguity after SC16.
Even when we issued certificates and the cablint/cerlint tools warned about this OID, we just thought this was a "warning" because it was not recommendable, but not forbidden.
Moreover, when we realized (late) about the consequences of ballot SC16, we disclosed the certificates issued since it was approved but still thinking that those issued previously were correct.
Now, we should start a remediation plan to fix this problem and start revoking all the certificates affected, but previous to this, in the current situation,
this is, with Ballot SC17 allowing this attribute, we would like to get some fedback whether other solutions may be acceptable.
For example, we would like to reissue those certificates with the new profile we defined (1.3.6.1.4.1.14777.6.1.4) but as there will be changes in the profile to comply with the last ballot, we have to notity the Ministry any changes made and wait for their answer.
So once we receive the answer, we would like to issue them again.

Comment 12

[Wayne Thayer]

Oscar: my understanding of your response is as follows:

• Izenpe acknowledges that the certificates containing organizationIdentifier that were issued prior to ballot SC16 were also misissued
• Izenpe plans to revoke these certificates, but would prefer to replace them with PSD2 compatible certificates that comply with the BRs as amended by ballot SC17
• Izenpe's new certificate profile that is compliant with ballot SC17 must be approved by a government ministry before those new certificates can be issued

If this is correct, I think that a few pieces of information would help the Mozilla community to make an informed recommendation on how to proceed:

• How many certificates need to be replaced? You should go ahead and disclose all of them.
• By when will they be revoked?

Comment 13

[Oscar Garcia]

Attachment: Misissued certificates wirh OrganizationIdentifier

Hi Wayne,
the amount of certificates that include the OID 2.5.4.97 and  would need to be replaced is 133. At the end of this post you can find a list of all them.
Regarding the revokation plan, as we commented previously, we would like to revoke them all as soon as the changes to Izenpe's "1.3.6.1.4.1.14777.6.1.4" profile changes have been approved by the Ministry. But we have no idea of how long would this take when we are so close to  Summer holidays. Even if we went ahead reissuing this certificates with the current profile (including the "OrganizationIdentifier", but no other information as added in ballot SC17), we would have problem with 80% of the certificates as they belong to municipalities whose politicians  have been recently elected due to the local elections held in Spain in May and those who would have to sign the requests still might have not take possesion of their position.

Comment 14

[Oscar Garcia]

Hi Wayne,
the amount of certificates that include the OID 2.5.4.97 and would need to be replaced is 133. At the end of this post you can find a list of all them.
Regarding the revokation plan, as we commented previously, we would like to revoke them all as soon as the changes to Izenpe's "1.3.6.1.4.1.14777.6.1.4" profile changes have been approved by the Ministry. But we have no idea of how long would this take when we are so close to Summer holidays. Even if we went ahead reissuing this certificates with the current profile (including the "OrganizationIdentifier", but no other information as added in ballot SC17), we would have problem with 80% of the certificates as they belong to municipalities whose politicians have been recently elected due to the local elections held in Spain in May and those who would have to sign the requests still might have not take possesion of their position.
Following with any of these two options, we think we could revoke and reissue all of them with the new profile during September/October.
It this wasn't acceptable, we would be forced to revoke all of them this month and reissue them as normal "DV" certificates , and once the profile is accepted by the Ministry, reissue all of them.

Regards,

Comment 15

[Ryan Sleevi]

(Reordered)

(In reply to o-garcia from comment #14)

Following with any of these two options, we think we could revoke and reissue all of them with the new profile during September/October.
It this wasn't acceptable, we would be forced to revoke all of them this month and reissue them as normal "DV" certificates , and once the profile is accepted by the Ministry, reissue all of them.

As captured in https://wiki.mozilla.org/CA/Responding_To_An_Incident , Mozilla is not in a position and does not determine whether or not a given revocation plan is "acceptable". The Baseline Requirements have specific requirements on the expectations of revocations. As with any of the BR requirements, a CA may at any point choose to ignore them in the issuance or operation of their CA. Mozilla cannot "force" a CA into compliance, but it can, will, and does keep record of areas of non-compliance.

As it pertains to revocation, this is spelled out in https://wiki.mozilla.org/CA/Responding_To_An_Incident#Revocation , and in particular, the expectations for CAs that do not revoke according to the BRs.

Regarding the revokation plan, as we commented previously, we would like to revoke them all as soon as the changes to Izenpe's "1.3.6.1.4.1.14777.6.1.4" profile changes have been approved by the Ministry. But we have no idea of how long would this take when we are so close to Summer holidays. Even if we went ahead reissuing this certificates with the current profile (including the "OrganizationIdentifier", but no other information as added in ballot SC17), we would have problem with 80% of the certificates as they belong to municipalities whose politicians have been recently elected due to the local elections held in Spain in May and those who would have to sign the requests still might have not take possesion of their position.

As captured in https://wiki.mozilla.org/CA/Responding_To_An_Incident#Revocation , the CA is responsible for ensuring timely revocation of their issued certificates. Furthermore, it is responsible for its policies and practices. A CA is not responsible, according to the BRs, for ensuring their Subscribers can get replacement certificates, but they are responsible for ensuring certificates are revoked in a timely fashion, and Subscribers are responsible to agreeing and acknowledging that.

Should Izenpe choose to delay revocation, due to the perceived disruption it may cause, Izenpe is responsible for developing and implementing a plan to ensure it never happens again. Note that this does not mean that such delays are acceptable - they may still be found as problematic - but it will certainly look much worse for a CA that both delays revocation and does not have a plan to ensure such delays never happen again.

Comment 16

[Oscar Garcia]

Hi Ryan,
thanks for your feedback. Finally we have decided to proceed with the inmediate revocation plan. All custumers have been notified and the first certificates have been already revoked. We plan to have finished all the process in the following days.
At the end of every day we will provide information regarding the revocation process progress.

Comment 17

[Oscar Garcia]

From the previously posted list:
2019/07/03
Revoked: 10
Expired: 2
Still Misissued: 121
2019/07/04
Revoked: 10
Expired: 2
Still Misissued: 121
Reissued: 22

Comment 18

[Oscar Garcia]

2019/07/05
Revoked: 34
Expired: 2
Still Misissued: 97
Reissued: 55

Comment 19

[Oscar Garcia]

2019/07/08
Revoked: 55
Expired: 2
Still Misissued: 66
Reissued: 87

Comment 20

[Ryan Sleevi]

Thank you for the updates. However, it’s quite concerning that Izenpe has not committed to a date yet. This is an absolute minimum requirement of any CA who is violating the BRs, as they have full power and ability to commit to a date. Please indicate when this issue will be resolved, and if it will be longer than this week. Furthermore, please demonstrate a comprehensive plan to ensure these reasons for delay never occur again.

Comment 21

[Oscar Garcia]

All the misissued certificates correspond to official sites (according to national regulation) of local or regional public entities. These sites are not only the website where they publish their official bulletins, but they are also the entry point of all invoices received from all their providers (it's a requirement of a national regulation). At the moment when we revoke that certificate, those public entities loose the ability to work with their providers. For that reason we have to issue a new certificate (and do all validations) before revoking the current one, that's the reason to delay it for some days. In any case, from the beginning our plan was to have all misissued certificates revoked by tomorrow July 10th, but we didn't put in this post Sorry for the absent-mindedness.

Regarding the actions to avoid future delays in revocation, we'll analyze the risk for each situation, taking into account the requirements of Mozilla and CABF to revoke within the timeline defined by them (24 hours or five days), depending on the impact.

This is the updated report for today:

2019/07/09
Revoked: 112
Expired: 2
Still Misissued: 29
Reissued: 111

Sorry for the inconveniences.
Best regards

Comment 22

[Ryan Sleevi]

Regarding the actions to avoid future delays in revocation, we'll analyze the risk for each situation, taking into account the requirements of Mozilla and CABF to revoke within the timeline defined by them (24 hours or five days), depending on the impact.

To be clear, the question is about trying to understand future situations. I want to emphasize: It is unacceptable to delay past the BR-mandated timeperiod. I can understand as a CA that you want to keep your relationship with your customer happy, but as a publicly trusted CA, adherance to the BRs is not only expected, it's required.

For example, one way to ensure there are not future delays is for such local and regional public entities to migrate to other CAs. I can understand why this may not be desirable, but it shows how you can reduce Izenpe's risk by ensuring you don't do business with entities that do not accept or acknowledge that revocation may be required with as little as 24 hours notice.

Of course, alternatives exist, such as migrating these customers to DV certificates so that they're easier to replace, or automating, or changing how you do OV/EV validation to facilitate quicker replacement. The important thing, however, is that such a delay in revocation must never happen again for Izenpe, and the community needs to know what steps Izenpe is taking to achieve that in the future. The alternative, of course, if that there is a CA that does not adhere to the BRs and does not set themselves up in a way to adhere to them, they ultimately end up distrusted. So such a plan to ensure future compliance is essential to future trust.

Comment 23

[Oscar Garcia]

First of all, this is the today report of the incidence. We revoked all certificates but one (done the following day) by the proposed date (July 10th).

We don't consider the option of revoking all affected certificates out of the period defined by the BRs (24h or 5 days). We’ve defined a specific plan to treat this kind incidence, the case of having to replace all certificates by DVs. Additionally, we’ve defined some additional measures to speed up the reissuance:

• Give specific information about this possible scenario to the applicant
• Store CSRs and register the information about when it can be reused
• Automate the communication with our customers, to send them the request form already filled. This includes a way to identify the applicant
• Automate the access to the .eus domains whois database
• Campaign to promote the inclusion of DNS TXT or DNS CAA entries by our customers

The deadline to apply all those actions is by next November.

Comment 24

[Wayne Thayer]

Oscar: Please confirm that the deadline for completing the actions described in comment 23 is November 2019. I will go ahead and set the "Next Update" on this bug to 1-December under this assumption. Also, please update this bug as each action is completed, and if anything causes the November date to slip.

Comment 25

[Oscar Garcia]

Yes, I confirm that the deadline is November 2019
Regards

Comment 26

[Oscar Garcia]

Hi, this is the state of actions up to the date:

• Give specific information about this possible scenario to the applicant => included specific information in the SSL Policy

• Store CSRs and register the information about when it can be reused => we expect to have the new version of web application to request all SSLs in January. It will control that a CSR is not reused

• Automate the communication with our customers, to send them the request form already filled. This includes a way to identify the applicant => that new web application will generate the request form, and will manage the relationship with customers. We have also modified the policy to require the application form to be signed with a Representative Qualified certificate.

• Automate the access to the .eus domains whois database => this access depends on the entity that gives the service. We can't assure at this point in time that they'll allow that access. Anyway we have added some other ways to validate the domain ownership, according to CABForum BRs.

• Campaign to promote the inclusion of DNS TXT or DNS CAA entries by our customers => we have started the campaign with our customers. We expect to finish it by the end of 2019.

Comment 27

[Ryan Sleevi]

Wayne: Are there any further questions here? Comment #26 is a follow-up to Comment #23, and while it sounds like there's still one piece for EOY, the campaign being started seems to meet the set of expectations set?

Comment 28

[Wayne Thayer]

(In reply to Ryan Sleevi from comment #27)

Wayne: Are there any further questions here? Comment #26 is a follow-up to Comment #23, and while it sounds like there's still one piece for EOY, the campaign being started seems to meet the set of expectations set?

In addition to completing the customer campaign, a new version of the CA's Web application is due in January according to comment #26.

Oscar: please continue to update this bug until those remediations have been completed.

Comment 29

[Oscar Garcia]

This is the updated state of actions:

• Give specific information about this possible scenario to the applicant
  Done. Included specific information in the SSL Policy

• Store CSRs and register the information about when it can be reused
  Done. Deployed new application for all web certificates

• Automate the communication with our customers, to send them the request form already filled. This includes a way to identify the applicant
  Done. Deployed new application for all web certificates

• Automate the access to the .eus domains whois database
  We're in the process of getting access to the .eus online database

• Campaign to promote the inclusion of DNS TXT or DNS CAA entries by our customers
  Done

Thanks

Comment 30

[Ryan Sleevi]

Oscar: Thanks for the update.

Note, per Comment #28, the expectation was an update would have been provided 1-January on things, rather than 6-February.

Regarding the remaining item:

• Automate the access to the .eus domains whois database

Do you have an estimated timeframe?

Comment 31

[Oscar Garcia]

We depend on the development that the .eus registrar must do, and their timeframe has been affected by the Covid. They estimate to have in production by October. We're going to take advantage of the delay to automate some other checks related to domain ownership. We've just finished the analysis, we'll update this bug once we have the estimated timeframe.
Regards

Comment 32

[Ben Wilson]

Can you provide a status update here? I'd like to re-set the date in the Whiteboard for the Next Update.

Comment 33

[Oscar Garcia]

According to our schedule we plan to have it in production by next October 9th

Comment 34

[Ben Wilson]

Hi Oscar
Could you provide me with a status update on this bug?
Thanks,
Ben

Comment 35

[Oscar Garcia]

Last October 28th we put into production environment all changes to integrate the web application with the PKI system. We have developed two applications; one for the customer, where all requests are registered, and the other one for Izenpe operators. The internal frontend is able to issue the certificate once all validations are correct. A qualified certificate with two authentication factors are required to access to both applications.
The system manages all evidences in the life-cycle of each certificate.
This change also includes that all validations are made automatically, so we've reduced the risk of human mistakes.
Thanks

Comment 36

[Ben Wilson]

I plan to close this incident on or about 9-November-2020 unless there are additional issues or questions to address.