1542328 - Certinomis: Invalid TLD in SAN

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[François CHASSERY]

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36

Actual results:

Here is an incident report align the mozilla template :

1/ How your CA first became aware of the problem.
After a post issuance linting of our security team

2/ A timeline of the actions your CA took in response.

01/04/2019 14:27:07: issuance of a certifcate whithout the TLD
01/04/2019 14:35:04: revocation of the certificate
01/04/2019 14:38:13: issuance of one certificate without the TLD
01/04/2019 19:00:16: revocation of the certificate
02/04/2019 09:35:48 : issuance of one certificate without the TLD
03/04/2019 14:28:38 : revocation of the certificate
03/04/2019 09:41:34 : issuance of a certificate without TLD
03/04/2019 14:29:43 : revocation of the certificate

3/ Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem.
the cause has been identified and the problem is now corrected

4/ A summary of the problematic certificates.
2 certificates "www.internet.ccmu.net3-courrier.extra."
1 certificate "ws-h5b-mercure.transactis.extranet.sf.intra."
1 certificate "*.preproduction.ezyfront."

5/ The complete certificate data for the problematic certificates.

6/ Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.
A new domain validation function has been developped and seamed effective in pre-production.
But the tests has not been conducted long time enough, because we were in a hurry for it.
And after few days we detected issuance mistake because of an edge effect.

7/ List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future.
we went backwards and the code is under analysis

Comment 1

[Ryan Sleevi]

(In reply to François CHASSERY from comment #0)

01/04/2019 14:27:07: issuance of a certifcate whithout the TLD
01/04/2019 14:35:04: revocation of the certificate
01/04/2019 14:38:13: issuance of one certificate without the TLD
01/04/2019 19:00:16: revocation of the certificate
02/04/2019 09:35:48 : issuance of one certificate without the TLD
03/04/2019 14:28:38 : revocation of the certificate
03/04/2019 09:41:34 : issuance of a certificate without TLD
03/04/2019 14:29:43 : revocation of the certificate

Over multiple days, invalid certificates were issued. This raises serious concerns about Certinomis' handling.

1. Why was an incident report not filed immediately after detecting the bad certificate?
2. Why, after a second bad certificate that day (with revocation), was issuance not halted until the problem could be addressed?

3/ Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem.
the cause has been identified and the problem is now corrected

In a later response, you reply "the code is under analysis".

Please provide a more meaningful explanation about what the cause was, why it happened, and how you have fixed it.

5/ The complete certificate data for the problematic certificates.

Why was this not provided?

6/ Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.
A new domain validation function has been developped and seamed effective in pre-production.
But the tests has not been conducted long time enough, because we were in a hurry for it.
And after few days we detected issuance mistake because of an edge effect.

A more thorough explanation is needed here. Why would the duration impact the tests? Why did you rush it? Why did it take you several days to detect the misissuance? What was the edge effect? More details are needed.

7/ List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future.
we went backwards and the code is under analysis

I'm not sure this at all instills confidence. Please provide a more thorough response.

Comment 2

[Ryan Sleevi]

Relevant to this discussion: https://crt.sh/?id=1352972593&opt=cablint,zlint , issued the same day this problem was reported as fixed, and not in the impacted set of certificates. (From https://bugzilla.mozilla.org/show_bug.cgi?id=1539531#c5 )

Comment 3

[François CHASSERY]

Here are the events:

As we have made the commitment previously, Certinomis has developed a domain name testing feature to integrate into its RA software.
This function has been installed and tested in pre-production.
But we are already behind the original schedule and we wanted to correct the problems detected previously as soon as possible.
The tests therefore focused only on correcting domain name errors, and the non-regression tests were insufficient: the few cases I reported in bug 1542328 arrived with domain names longer than those used during the tests.
Indeed, the edge effect of the new function was to truncate the end of these long domain names, which is the TLD (i.e. "laposte.fr" is missing after the last right dot)
So we did the way back and removed the new feature from the RA software.
So it is not contradictory to say in the same time that we have stopped the cause of this problem and that we are in the process of analyzing what has not worked (analysis of the code).

here are the forgotten informations :

https://crt.sh/?id=1337502574
https://crt.sh/?id=1337480938
https://crt.sh/?id=1348826250
https://crt.sh/?id=1344291274
https://crt.sh/?id=1340567152

https://crt.sh/?id=1352972593&opt=cablint,zlint reported by ALex Gaynor is not the same (the TLD is not missing). I received the alert late on Friday our time and the new bug report will be opened today before I leave.

Comment 4

[Ryan Sleevi]

(In reply to François CHASSERY from comment #3)

As we have made the commitment previously, Certinomis has developed a domain name testing feature to integrate into its RA software.
This function has been installed and tested in pre-production.
But we are already behind the original schedule and we wanted to correct the problems detected previously as soon as possible.
The tests therefore focused only on correcting domain name errors, and the non-regression tests were insufficient: the few cases I reported in bug 1542328 arrived with domain names longer than those used during the tests.
Indeed, the edge effect of the new function was to truncate the end of these long domain names, which is the TLD (i.e. "laposte.fr" is missing after the last right dot)
So we did the way back and removed the new feature from the RA software.
So it is not contradictory to say in the same time that we have stopped the cause of this problem and that we are in the process of analyzing what has not worked (analysis of the code).

Thanks for the update.

These all seem very relevant to the timeline. Could you please provide the timeline for these changes, including the following events:

• Development of the proposed functionality begins
• Development of the proposed functionality ends
• Review with the CAB
• Implementation and testing in pre-production
• Deployment in produciton
• Disabling in production

Comment 5

[Firefox Bug Husbandry Bot]

Correcting bug type to task.

Comment 6

[Wayne Thayer]

The Certinomis Root CA is being removed from the Mozilla root store in bug 1552374, so I am resolving this bug. Additional comments that may be useful when considering any future application by Certinomis are welcome.