- How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.
On 5 January 2021 at approximately 17:00 UTC, the Entrust Verification team discovered a single EV SSL certificate with an invalid Business Category value. This was discovered during a regular EV business profile re-validation. It appears that during a contact update change on the business profile in 2019, the business category was inadvertently updated by an agent from its correct value (Government entity) to an incorrect value (Private Organization).
- A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.
28 January 2019: EV validation was performed for the organization and the correct Business Category field (Government Entity) was set at this time. Note that many other EV verifications had been performed for this profile, dating back as far as 2013.
18 April 2019: The subscriber contacted Entrust to request a contact change for one of their verified contacts. During this contact update, the first level Verification Specialist accidentally changed the Business Category field from “Government Entity” to “Private Organization”. Not expecting there to be any changes to the Business Category field during this contact change, the second level Verification Audit Specialist approved the changes to the EV business profile
13 January 2020: EV re-verification was performed by Entrust on this business profile, as it was approaching the end of the 13-month data re-use period. At this time, the incorrect Business Category from the previous verification was not detected and the same Business profile data was re-approved for the next 13 months along with the rest of the EV business information.
18 March 2020: The subscriber issued an EV SSL certificate based on the EV business profile with the incorrect Business Category value.
5 January 2021 18:00 UTC: The Entrust Verification team discovered the incorrect Business Category value for the business profile during a re-verification check.
5 January 2021 18:05 UTC: An internal investigation is started
5 January 2021 19:00 UTC: The issue is confirmed by the Entrust compliance team
6 January 2021 15:43 UTC: The subscriber is notified that the certificate must be revoked within 5 days. A 5-day revocation deadline is set for 10 January 2020 at 19:00 UTC.
8 January 20201 17:20:35 UTC: The certificate is revoked.
- Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.
The business profile for this specific customer has been corrected as 5 January 2021 and all future certificates for this subscriber will reflect the correct Business Category.
In addition, a scan was performed on all other EV certificates that contain the keywords “County”, “Government”, “Agency”, “Department”, “City”, and “Province” in the Organization Name field, and none of these profiles were set to Private Organization or Business Entity.
- A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.
A single EV certificate was impacted by this issue
- The complete certificate data for the problematic certificates.
- Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.
Entrust performs validation on existing EV business profiles every 13 months, as per the requirements in the EV guidelines. The original mistake that introduced the incorrect Business Category value was made in April 2019 and was not detected in the next re-verification in January 2020. To explain how this issue was introduced and not detected, we need to explore what happened on 18 April 2019 and also on 13 January 2020.
After conducting multiple interviews with senior agents along with agents who were involved with the original mistake on 18 April 2019, we found that the main reason this mistake was first introduced is due to a lack of highlighting changes with the previous verification data when updating a business profile in our Verification system. When this incident was being reviewed on 5 January 2021, it was immediately clear that the Business Category should have been set to Government entity based on the name of the organization (it was a County) and the supporting verification documents that were collected. This was not an issue with understanding the verification process for the Business Category. In addition, we received feedback that our current interface does not clearly list which fields in our EV verification tickets are being used as values in the certificate Subject DN.
With respect to the re-verification of this incorrect data on 13 January 2020 that eventually led to the certificate including the incorrect business category, it was determined that this was ultimately a case of human error that was driven by potentially confusing fields in our Verification system when dealing with Government entities. When reviewing the Jurisdiction information for this EV business profile, there are two fields labeled “Incorporating Agency” and “Registration Agent” that were both set to “Government Entity, in addition to a field labeled “Business Category” that was set to “Private organization”. The drop-down values for “Incorporating Agency” and “Registration Agent” fields should not use the term “Government entity”, as it can easily be confused with the Business Category value that will eventually appear in the certificate.
- List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.
In order to reduce the risk of future incidents like this one where information is inadvertently updated and not detected by a second-level verification agent, we are proposing major UI changes that will clearly show what information has changed from a previously verified business profile, especially information that will appear in the certificate Subject DN.
In addition, we will also introduce a UI that clearly displays what information is going to appear in the certificate Subject DN when a business profile is being verified so that these can be distinguished from other fields that we use to track evidence sources and contact information, for example.
We will also change the drop-down values for the two tracking fields “Incorporating Agency” and “Registration Agent” so that they do not include values such as “Government entity”, which is the exact same value that would appear in the Business Category drop-down and the certificate subject DN.
We have not yet identified a target release date for these changes. As a next step, we will work with our engineering and Product Management teams to schedule these changes in a future release.