Certinomis: certificate for test.com, O=Entreprise TEST

ASSIGNED
Assigned to

Status

ASSIGNED
5 months ago
12 days ago

People

(Reporter: agwa-bugs, Assigned: marc.maitre, NeedInfo)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [ca-compliance])

Attachments

(1 attachment)

190.82 KB, application/pdf
Details
(Reporter)

Description

5 months ago
Certinomis has issued the following pre-certificate:

https://crt.sh/?sha256=516e750a2fff294f5468e5b29a4fa0d9648fc9d7d8e7936d2f9847a30d3e0267

It contains a DNS SAN for test.com, and the subject contains O=Entreprise TEST.  I strongly suspect this is misissued.
As precertificates are considered a commitment to issue by the CA, this does appear to be a misissuance. Please provide an incident report for this problem, as described here:
https://wiki.mozilla.org/CA/Responding_To_A_Misissuance#Incident_Report
The incident report should be posted to the mozilla.dev.security.policy forum and added to this bug. The report should include an explanation of why this incident was not promptly reported to Mozilla.
Assignee: wthayer → franck.leroy
Flags: needinfo?(franck.leroy)
Whiteboard: [ca-compliance]

Updated

5 months ago
QA Contact: kwilson → wthayer

Comment 2

5 months ago
Hello

Here is an incident report aligned to the mozilla template :

1/ How your CA first became aware of the problem.
I've been aware by this bug openned by Andrew and affected to me by Wayne.

2/ A timeline of the actions your CA took in response.
2018-10-03  12:22:59 UTC    : human error in validation of a test certificate
2018-10-03  14:22    UTC    : bug openned by Andrew
2018-10-03  15:41    UTC    : buzilla notification done by Wayne, received my email
2018-10-03  16:03:55 UTC    : revocation of the invalid certificate

3/ Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem.
It is a human error, not a systemic one.

4/ A summary of the problematic certificates. 
One certificate : test.com

5/ The complete certificate data for the problematic certificates.

https://crt.sh/?sha256=516e750a2fff294f5468e5b29a4fa0d9648fc9d7d8e7936d2f9847a30d3e0267

6/ Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.
We had an issue with the SMTP server so the certificates were not sent to the customers.
When the SMTP issue has been resolved, the sales departement has tested the full application process by requesting a 'test certificate'.
The 'test certificate' process is to use "Entreprise TEST" account and apply for the domain "test.certinomis.com"

The operator made a mistake in the TLS certificate application, is has entered "test.com" instead of "test.certinomis.com"
The operator did not send the certificate to the customer, as there is no customer, it is for test purpose only.
When I've been notified of such error (and Andrew/Wayne were pretty fast!) I immediatly revoked the invalid certificate, and sent an incident email to Certinomis' CEO (I was out of office at an ETSI meeting).

7/ List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future.
We will change in a few days the testing process to avoid such human errors.
The "Entreprise TEST" account will be limited to one domain name : "test.certinomis.com".
So this error could not be done again in this process.

The incident was not reported "promptly" because the operator did not understand that he made a mistake.
The timeline of the event is very small, a window of less than 4 hours.
It is quite difficult to imagine a process that detects, analyses, and reports an incident in less than 4 hours.

All operators have been briefed of this incident, and a reminder of the validation/report rules has been done by Certinomis' CEO.

Best regards
Franck Leroy
Franck: please update this bug when the technical control limiting the "Enterprise TEST" account to a Certinomis domain has been implemented.

The incident report implies that Certinomis operators can override technical domain validation controls on any certificate request, so a similar error could be made on a fraudulent request from a customer. What prevents an operator from accidentally approving a request for a high value domain such as facebook.com or gmail.com?

Comment 4

4 months ago
(In reply to Franck Leroy from comment #2)
> Hello
> 
> Here is an incident report aligned to the mozilla template :
> 
> 1/ How your CA first became aware of the problem.
> I've been aware by this bug openned by Andrew and affected to me by Wayne.
> 
> 2/ A timeline of the actions your CA took in response.
> 2018-10-03  12:22:59 UTC    : human error in validation of a test certificate
> 2018-10-03  14:22    UTC    : bug openned by Andrew
> 2018-10-03  15:41    UTC    : buzilla notification done by Wayne, received
> my email
> 2018-10-03  16:03:55 UTC    : revocation of the invalid certificate
> 
> 3/ Whether your CA has stopped, or has not yet stopped, issuing certificates
> with the problem.
> It is a human error, not a systemic one.

At this point in the CA ecosystem, the potential to have human errors - particularly human errors that have affected other CAs in other incident reports - and that the CA has not taken steps to redress *does* highlight systemic flaws in the CA processes.

What is particularly telling about this is when it occurred - within days of Bug 1495524 - which seems to share a similar root cause. This suggests that the response to Bug 1495524 was problematic, in that one of the expectations of any CA, particularly one affected by a human error issue, would be to review its process and procedures with its staff to retrain them on the appropriate validation processes.

Would you like to provide any more detail beyond what is provided in 5.3.3, 5.3.4, and 5.7.1 to describe how Certinomis handles and responds to incidents, particularly those related to human errors, and what steps it takes to mitigate the ongoing risk of human factors?

> The incident was not reported "promptly" because the operator did not
> understand that he made a mistake.
> The timeline of the event is very small, a window of less than 4 hours.
> It is quite difficult to imagine a process that detects, analyses, and
> reports an incident in less than 4 hours.

It doesn't seem difficult, considering that this was noticed within 2 hours of the incident. Instead, it's quite difficult to imagine why a CA in 2018 would be able to have such an issue, days after having a similar incident, and has not implemented automated controls to the entry, validation, and processing of domains and certificates.

> 
> All operators have been briefed of this incident, and a reminder of the
> validation/report rules has been done by Certinomis' CEO.

This is troubling - again, in light of Bug 1495524 - and seriously calls into question the domain validation procedures practiced at Certinomis. One possible interpretation is that Certinomis' response to Bug 1495524 with its employees did not adequately stress the importance of proper domain validation when entering in information - which would be a lapse by Certinomis' PKI managers. Another possible interpretation is that the employees did not respond to the previous message, in which case, the remediation here is quite inadequate.

Comment 5

4 months ago
Created attachment 9015197 [details]
screenshot.pdf

Comment 6

4 months ago
Hello,

>The incident report implies that Certinomis operators can override technical domain validation controls 
Operators cannot override domain validation when the feature is activated.

This option for domain validation is neccessary as we can have customers that apply for publicly trusted or non publicly trusted certificates (named corporate) .

The issue was that the "Entreprise TEST" demo account was not correctly restricted for SSL/TLS certificates.

I uploaded some screenshots to explain the actions that has been done to prevent from similar errror.

The first screenshot shows that for the demo account, the area for CN (and SANs) is free text, so "test.com" is possible.

The second screenshot shows the "SSL restriction" activated with only one domain : certinomis.com

The third screenshot shows that with this option activated, only "certinomis.com" domain is possible for an application, and the free text area is only for the sub-domain.

The second action was to change the certificate type. Indeed to test if the system works correctly it is not necessary to issue a publicly trusted certificate.

The last screenshot shows that the certificate profile has been changed to issue a TLS certificate under "Corporate G4" CA, that is not publicly trusted.

Best regards
Franck Leroy

Comment 7

4 months ago
(In reply to Franck Leroy from comment #6)
> >The incident report implies that Certinomis operators can override technical domain validation controls 
> Operators cannot override domain validation when the feature is activated.
> 
> This option for domain validation is neccessary as we can have customers
> that apply for publicly trusted or non publicly trusted certificates (named
> corporate) .
> 
> The issue was that the "Entreprise TEST" demo account was not correctly
> restricted for SSL/TLS certificates.

These three statements do not provide sufficient assurance that the root issue has been mitigated. It sounds like the answer to Wayne's question is, in fact, that operators can override domain validation if the customers are not correctly restricted for SSL/TLS certificates.

This issue demonstrates at least one customer (Certinomis itself, apparently) was not properly restricted. The root cause analysis to date has not highlighted this as a systemic remediation - rather, a single account was restricted. Why is it not appropriate to re-examine every one of your customer accounts to ensure appropriate restrictions are configured? Or, perhaps more explicitly, to ensure that any unrestricted accounts cannot obtain certificates from publicly trusted CAs? Why does the account restriction even exist, since it relates to which CA is being used to issue (and thus should not matter how the 'account' is configured)

> I uploaded some screenshots to explain the actions that has been done to
> prevent from similar errror.
> 
> The first screenshot shows that for the demo account, the area for CN (and
> SANs) is free text, so "test.com" is possible.
> 
> The second screenshot shows the "SSL restriction" activated with only one
> domain : certinomis.com

This screenshot does not support the statement in comment #2 that:
> > The "Entreprise TEST" account will be limited to one domain name : "test.certinomis.com".
> > So this error could not be done again in this process.

It seems like the operator could request www.certinomis.com using this interface and potentially misissue certificates. Is that not correct?

Comment 8

4 months ago
Hello

>This issue demonstrates at least one customer (Certinomis itself, apparently) was not properly restricted. 
Yes, it is a demo account used internally to test or demonstrates the certificate application.

>Why is it not appropriate to re-examine every one of your customer accounts to ensure appropriate restrictions are configured?
You are right, I will do it as soon as possible.

>Why does the account restriction even exist, since it relates to which CA is being used to issue
Our system application exists since 2008 and managed many workflows on many different CAs.
Some CAs are not publicly trusted and mainly used internally in 'La Poste Group'.
So in that case, the GUI is not restricted in order to allow any subject in the certificate.

The rule is that no unrestricted account shall be allow to issue PTC. 
I miss this one, may be my fault, so I have to review them all.

>one domain name : "test.certinomis.com".
Yes the system is not designed to limit to some FQDNs, only domain names.
I can enter "test.certinomis.com" is the configuration, and in that case, any 'www' will produce: 'www.test.certinomis.com"

>It seems like the operator could request www.certinomis.com using this interface and potentially misissue certificates.
Yes, but I also changed the PKI certificate profile to use the corporate CA G4.
It could be usefull to make a demo that reflects a real certificate application, but no more under a publicly trusted CA.

Best regards
Franck Leroy
(Reporter)

Comment 9

4 months ago
Certinomis has also issued a pre-certificate for www.pourtest.com with O=Entreprise TEST in the subject:

https://crt.sh/?sha256=B77B246619EB66F97F72A753EC5A97F07D48DC44047EB717639019E0C0A8B628

I strongly suspect that the applicant's control over www.pourtest.com was not validated, as the domain is not registered to Certinomis, nor does it appear to be a customer of Certinomis, as all other certificates for this domain are issued by Let's Encrypt (https://crt.sh/?q=%25.pourtest.com).

This certificate was revoked on 2018-10-03 at 16:05:27 UTC - less than two minutes after the certificate for test.com was revoked (2018-10-03 16:03:55 UTC).  However, Certinomis did not mention this certificate in their incident report.

Here are five more unexpired (pre-)certificates with O=Entreprise TEST in the subject:

https://crt.sh/?sha256=08BCB09E6AE3B4E21F298F51C44F7ED8632FDC92E88E188D685B0900F87BF61D
https://crt.sh/?sha256=B1940AD03E642275D785C5C87290CD4B999B56C9F584423CB11F5A0565F8E7DD
https://crt.sh/?sha256=8F5E62DB7D52CA96FB6E60924F9D11AAD67958EF92A7BD46F0554B2EADEAD80E
https://crt.sh/?sha256=5365B1C22A20E3FB4A7F80D111D0EA9DACABC1663366B6AED5A0FB052C94C064
https://crt.sh/?sha256=4D9A21EFBF0D73E6C247BF5CE035B21AB5EB445E2D236E6751F078F7BEB14A18

Only the first two are revoked at this time, suggesting that Certinomis' investigation and remediation are still incomplete 25 days later.

I would also like to note that Certinomis' incident report was not posted to mozilla.dev.security.policy as required.
(Reporter)

Comment 10

3 months ago
It has been 15 days since I reported the incomplete remediation and the likely-misissued certificate for www.pourtest.com, and Certinomis has yet to post an update.  Per https://wiki.mozilla.org/CA/Responding_To_An_Incident, "Once the report is posted, you should provide updates at least every week giving your progress, and confirm when the remediation steps have been completed - unless Mozilla representatives agree a different update schedule with you."  I would therefore expect to have seen an update from Certinomis in the last 15 days addressing the new information.

Comment 11

3 months ago
Hello

I have reviewed all certificates issued under the "Entreprise TEST" demo account.

3 TLS certificates were issued under "Certinomis Web CA"
	names are : www.pourtest.com, www.seres-as2.fr, testpro.darva.com
4 TLS certificates were issued under "Certinomis Easy CA"
	names are : test.com, aboxnote.com, expe.visio.douane.gouv.fr, TRANSGOURMET - CHORUS TEST
	
The 7 certificates has been revoked.

The "Entreprise TEST" demo account is configured to "Certinomis Corporate CA" that is not publicly trusted. So demo usage will no more issue PTC certificates.

As said previously, one workflow in our RA software (used internally for test and demo) can bypass domain validation.
This workflow has been disabled.
A new development is running to implement domain validation in this workflow, then when available it will be possible to reactivate it.
The new version of the RA software is expected in production by the end of the year.

Bets regards
Franck Leroy

Comment 12

2 months ago
> The new version of the RA software is expected in production by the end of the year.

Franck, can you provide an update now that we're in the new year?
Flags: needinfo?(franck.leroy) → needinfo?(fr.leroy)
Franck is no longer a Certinomis representative. Reassigning to Marc.
Assignee: franck.leroy → marc.maitre
Flags: needinfo?(fr.leroy) → needinfo?(marc.maitre)

Comment 14

a month ago

Marc: It's been nearly two weeks. Do you have an update?

Comment 15

25 days ago

Marc: Checking to see for updates?

Updated

12 days ago
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true

Comment 16

12 days ago

Marc: It's been one month. Do you have updates?

You need to log in before you can comment on or make changes to this bug.