1559376 - Entrust: Certificate Issued with Incorrect Country Code

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Dathan Demone]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:67.0) Gecko/20100101 Firefox/67.0

Steps to reproduce:

Entrust Datacard issued a single certificate with an incorrect country code.

Comment 1

[Dathan Demone]

1. How your CA first became aware of the problem
   Entrust Datacard issued 1 certificate with an incorrect country code.
   This issue occurred due to a process that is used to issue certificates for embargoed and sanctioned countries. Our online store does not allow a customer to submit country codes for embargoed countries. In these rare cases, customers will sometimes submit the order using a different country and contact us to update the country. Once it is determined that the customer’s organization is based in a sanctioned or embargoed country, we have a process that allows the customer to fill out a form that was created by our legal team to provide more information about their organization and to agree to the specific Canadian Regulations for that country. Once the form is filled out and signed, we review it and decide if the certificate can be issued. Once it is decided that the certificate can be issued, the order is approved by 2 verification agents, queued for issuance, and the country code is updated using an SQL script, as our vetting tools do not allow the verification team to select an embargoed country.
   In this case, the certificate was issued before the SQL script was completed to update the country code. The invalid country code was noticed immediately when the script was about to be executed as part of our regular process.

2. A timeline of the actions your CA took in response

June 13, 2019, 2:30 AM UTC – The order was approved with the place holder country code and the workflow to update the country code via SQL script was initiated.

June 13, 2019, 10:21 AM UTC – The certificate was issued with the incorrect country code before the country code was updated via an SQL script.

June 13, 2019, 1:49 PM UTC – The person responsible for running the SQL script noticed that the certificate had already been issued with an incorrect country code.

June 13, 2019, 2:30 PM UTC – The incorrect certificate was revoked and replaced with a certificate that contains the correct country code before the customer downloaded the certificate. Revoking the bad certificate disables the customer’s ability to download the certificate.

June 13, 2019, 2:45 PM UTC – Investigation started to determine the root cause if other certificates were impacted, and steps to mitigate future mis-issuance for these orders.

3. Confirmation that your CA has stopped issuing TLS/SSL certificates with the problem
   At this time, we believe that this issue is isolated to this certificate due to the workflow we have in place to make sure that these orders are both updated and checked during the validation and certificate issuance process.
   We will be performing some additional scans of our certificate database to make sure that there are no other incorrect country codes for sanctioned or embargoed countries.

4. A summary of the problematic certificates

The country code “AD” was used instead of the country code “MM”. The rest of the certificate information was correct.

5. The complete certificate data for the problematic certificates

https://crt.sh/?id=1573082810

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

Due to the nature of sanctioned country orders, these orders are handled manually and with a great deal of care. The process described in (1) has been used for many years without any issues. In this case, there was a miscommunication between the second level order approver and the person issuing the certificate, which resulted in the certificate being issued before the country code update occurred using the SQL script. This situation is believed to be isolated to human error, but this has opened up a review of the process, systems, and tools we use to handle these types of orders. As a result of this specific miss-issuance, we have determined that the timing of the country code update is not occurring early enough in our validation and certificate issuance process and that we should be handling the sanction check as part of our validation process to avoid having to use the country code update workaround.

7. List of steps your CA is taking to resolve the situation

In the short term, we are modifying our process to require that sanctioned country orders country codes be updated before the order is approved at the first level in our two-step verification process. Once the country code is updated and approved by the first level verification specialist, the orders will be reviewed by a second verification audit specialist before it is queued for issuance.

In the long term, we are building out a sanctioned and embargoed country check in the vetting process. This will allow the first level verification specialist the ability to chose an embargoed country during the vetting process and not require that a separate person run an SQL script to make the required changes outside of the vetting process. This will ensure that embargoed country orders are approved and queued for issuance like the rest of our regular orders.

Comment 2

[Ryan Sleevi]

Can you provide a timeline for the 'short-term' and 'long-term' changes and mitigations? When will those steps be completed?

Similarly, can you provide an update and result of the scans regarding incorrect country codes?

I won't deny I'm concerned with how the process of "run a SQL script" complies with the auditing requirements of the BRs; that tends to be particularly difficult to maintain appropriate records at that level, especially if incidents occur such as the execution of an improper command.

Comment 3

[Dathan Demone]

The short term process change has been implemented and is in place today. Any change to verification data via SQL script will be reviewed by two verification agents before the certificate is queued for issuance.

The long term fix that I described in the original post is currently scheduled to go live in September 2019. This fix will eliminate the need to run SQL scripts and allow verification agents the ability to select embargoed countries from a drop-down menu in the vetting application.

We performed scans and found one other order where the country code did not appear to be correct. In this case, C=GH was used when C=LR should have been used instead. The certificate is showing as revoked but has not yet expired. After further investigation, this mis-issued certificate is a result of the same issue described in my original post.

https://crt.sh/?id=1195965145

Comment 4

[Ryan Sleevi]

Thanks. I'm going to pass this to Wayne to see if he has further questions, or if the September timeline is acceptable.

I'm encouraged to see steps to move away from the SQL script approach. As noted, I worry about how that could comply in practice with the auditing requirements. I suspect that the answer may rest in the 'short-term' process change, but as it sounds like that's a new process, I'm not sure it's something that was existing. I suspect this is an opportunity for the Baseline Requirements to more expressly forbid such practices. As CAs are expected to disclose these Bugzilla issues to their auditors, my hope is that as your auditors examine this issue, they pay close attention to how Entrust previously guaranteed data integrity and audit logging, in the event there were matters of non-compliance or concern that should have their attention drawn to, as well as to ensure that the September mitigations adequately close down any existing loopholes or opportunities to bypass those controls undetectably.

Comment 5

[Wayne Thayer]

No further questions. Setting next update to 1-October.

Comment 6

[Dathan Demone]

The long term fixed that I described in my earlier post to eliminate the need to run SQL scripts to update country codes for embargoed and/or sanctioned countries was implemented in a release that went live on September 3rd. We will no longer be using SQL scripts to update country codes for these orders.

Comment 7

[Wayne Thayer]

Thanks for the update. It appears that all questions have been answered and remediation is complete.