Closed Bug 1500621 Opened 11 months ago Closed 10 months ago

Digicert: Internal Domain Name cert mis-issuance


(NSS :: CA Certificate Compliance, task)

Not set


(Not tracked)



(Reporter: brenda.bernal, Assigned: brenda.bernal)


(Whiteboard: [ca-compliance])

1.       How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in, a Bugzilla bug, or internal self-audit), and the time and date.

On 10/16/2018, we were notified by a third party that they had identified an internal name on a publicly issued certificate, unrelated to their business, within the past week.

2.       A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.
10/16/18 A certificate with an internal domain name was reported to us by a third party

10/16/18 We investigated the root cause including the source of the request and the system that issued the problem certificate. We identified a gap in our domain pre-validation process that allowed a certificate containing an internal name to be submitted for validation by a customer. This was intentional as certificates for use cases other than the WebPKI use the same validation engine. Once an internal name was queued for validation, a validation agent overrode the base domain’s classification as “private” and manually performed a WHOIS procedure which is only allowed when automatic WHOIS data retrieval used to determine the email address for a method 2 email-based domain validation fails. The WHOIS lookup procedure allowed a validation agent to send a domain confirmation email to the customer containing a base domain substring of the FQDN, and the customer proceeded to approve the internal name in response to the method 2 confirmation. Because the automated WHOIS data retrieval failed in this case, it exposed an unintended ability for the validation agent to perform an override, categorize the domain as authorized for the WebPKI, and incorrectly approve the certificate. 

10/17/18 We revoked the one problem certificate and moved the pre-issuance check behind the validation process, ensuring the system blocks internal names for public certs regardless of validation staff mistakes. We also implemented changes on our portals to prevent the gap that allowed internal names into our domain pre-validation process for WebPKI. We have also improved our linting process pre-issuance to catch similar situations.

10/17/18 We ran a script over our existing certificate database to identify certs affected by this issue. No additional certificates were identified.
3.	Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.
On 10/17/18, we implemented two fixes and a linting enhancement to prevent this problem from re-occurring.  The linting enhancement checks the certificate pre-issuance but post validation to ensure no additional issues exist.

4.	A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued. 
 Impact of one certificate, issued on 10/10/2018.
7.	The complete certificate data for the problematic certificates.
6.       Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.
Limited front-end PSL checking allowed an internal domain name request to be submitted to our validation queue for human review. Although this is intentional because we support non-WebPKI systems, there was an override in the validation process that allowed a validation agent to approve the certificate for WebPKI. The override was added to allow a validation agent to override a pre-issuance check when manually accessing WHOIS data and providing the method 2 Domain Contact email address to the system. The override feature was only available if the automated WHOIS Domain Contact email address lookup service failed for the submitted domain. The override also allowed the validation staff to incorrectly approve non-WebPKI certificates for WebPKI once validation completed. A combination of these two problems permitted a manual override of the pre-issuance check and incorrect approval of the certificate for issuance.

7.	List of steps CA is taking to resolve the situation and ensure it will not be repeated. 
We implemented additional software rules and checks to prevent agents from overriding the pre-issuance check when automated WHOIS Domain Contact email address lookups fail and to prevent agents from associating internal domain names with domains on public suffixes. We also enhanced the existing front-end screening in our domain pre-validation system.
Brenda: thank you for the incident report and prompt handling of this misissuance.

Will you please post your report to the forum? While I understand that it is difficult to respond in both places, posting to the list ensures that the community is aware of the problem while the bug provides a permanent, easily searchable record.

Also, please explain how pre-issuance linting missed this. Both CABLint and ZLint correctly identify the issue, so while I understand how this slipped through the request submission and vetting process, I don't understand why it wasn't caught pre-issuance by a linter.
Assignee: wthayer → brenda.bernal
Whiteboard: [ca-compliance]
Hey Wyane, 

Before issuing this cert, DigiCert used a modified cablint to check for compliance. The modified cablint warned about potential issues but did not block issuance once approved by a validation staff. We have replaced our modified cablint check with zlint. Although zlint is currently not hard-blocking errors (while we evaluate it), we plan to turn it into a hard-fail system in the next week. 

Here’s a detailed account of why this was flagged in error by our linter but was able to be bypassed:
1.	A CSR was submitted with a base domain calculated as at.project12c from the FQDN
2.	This was allowed to enter our system, which equally accepts private trust and public trust requests, and the pre-validation check marked it as authorized for private trust only, due to detection of an internal name in the order
3.	When an agent retrieved the order to work on it, they saw a warning "internal names are on this order" that is produced by the pre-validation check. They can only approve the domain for private trust unless they manually upload WHOIS information, a feature that exists when automated WHOIS retrieval does not succeed for a variety of data formatting reasons. An agent then incorrectly fetched and uploaded WHOIS information for to the system, not at.project12c (this is a screenshot, not parsed).
4.	Any time a WHOIS is completed successfully, the domain becomes accepted for public trust because we interpret WHOIS success, even in a manual fetch, as a public trust eligible domain. WHOIS should be impossible for a private domain but the override allowed the validation staff to incorrectly complete the validation despite the warnings.
5.	Once the WHOIS was loaded, the agent next indicated the email addresses they retrieved from the WHOIS record, which caused domain control verification emails to be sent that comply with BR method 2. These email addresses were for the domain but were marked as valid WHOIS information for the .project12c domain.
6.	The customer approved the DCV.
7.	Another agent retrieved the order and the pre-issuance linter still flagged an internal name warning that was presented to the agent front and center on the page
8.	The agent ignored the internal name warning from the linter and approved the order, allowing the certificate to issue.
Closed: 10 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.