Open Bug 1910258 Opened 2 months ago Updated 3 days ago

DigiCert: Typo in TLS Org Name

Categories

(CA Program :: CA Certificate Compliance, task)

Tracking

(Not tracked)

ASSIGNED

People

(Reporter: martin.sullivan, Assigned: martin.sullivan)

Details

(Whiteboard: [ca-compliance] [ov-misissuance])

This is a preliminary report.

On Thursday 25th July 2024 DigiCert was contacted by an external researcher who let us know of a typo in the Org value for a single TLS certificate

https://crt.sh/?sha256=DBFCCB6A4F0E0FAE56043217C70148B8C2C1E6B4541B86DC182A0A6852720FE9

The issue being the value of Gmb which should be GmbH so missing the H.
This certificate will be revoked within timeline requirements of CABF and our CP/CPS.

DigiCert is currently investigating how this occurred and is scanning to see if there is any other instances of this issue.

We expect to have a full report no later than 2nd August.

Assignee: nobody → martin.sullivan
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Whiteboard: [ca-compliance] [ov-misissuance]

SUMMARY
DigiCert Issued a single Certificate that was missing the H in GmbH in the org name.

IMPACT
A single certificate
https://crt.sh/?sha256=DBFCCB6A4F0E0FAE56043217C70148B8C2C1E6B4541B86DC182A0A6852720FE9

TIMELINE -
07:09 June 24 2024 – certificate was issued

19:47 July 25 2024 – Certificate problem report received by DigiCert investigation started

21:00 July 25 2024 – Investigation confirms issue Certificate is scheduled for revocation confirmed this is a single certificate.

20:18 July 27 2024 - this Bug was raised

18:50 July 30 2024 – Certificate revoked

ROOT CAUSE ANALYSIS

DigiCert's systems have a variety of automated mechanisms for pulling organization names directly from the primary registration sources.
However occasionally this information cannot be read successfully by the tooling and it is referred to a validations specialist to confirm.
In this instance the Specialist did not notice that the pulled value was missing the H in GmbH and approved the certificate. The second approver also missed it.

LESSONS LEARNED

WHAT WENT WELL

Incident management went to plan, and all tasks completed within expected timelines.

WHAT DIDN'T GO WELL

There are still some cases where some human interaction is needed. We continue to make improvements to the automation to reduce the need for human interaction.

WHERE WE GOT LUCKY

This impacted only 1 certificate

ACTION ITEMS

Block for value of “Gmb” in Org names – Prevent – 31-Aug-2024

A reminder for the 2nd approver that will show a manual edit has been made, including the before and after values so the second approver can more easily see what the first specialist did. – Prevent – 31-Aug-2024

Improving the accuracy of automatic data pulls from sources via AI technology – Prevent – TBD as still scoping/prototyping

APPENDIX
DETAILS OF AFFECTED CERTIFICATES

https://crt.sh/?sha256=DBFCCB6A4F0E0FAE56043217C70148B8C2C1E6B4541B86DC182A0A6852720FE9

Hi Tim, thanks for the succinct report. I'd like to ask a question regarding one of the action items:

Improving the accuracy of automatic data pulls from sources via AI technology – Prevent – TBD as still scoping/prototyping

Can you provide additional details on what you mean by "AI technology"? If you are talking about current-generation Large Language Models, can you explain how you expect models which are prone to frequent hallucinations to "improve the accuracy of automatic data pulls"?

Thanks for the question Aaron,

It's not really an LLM thing. I share your concerns about frequent hallucinations and mostly avoid using LLMs myself. The only thing more error prone is the humans they've been trained to emulate ... which is why we're trying to avoid using humans where we can, too.

LLMs are not the only AI technology that has been rapidly improving lately. Image analysis and other associated technology for reliably pulling information out of what I will call "somewhat structured data" is rapidly improving, too.

This is useful because one of the challenges in this area is that there are LOTS and LOTS of registration authorities on the planet, and they all have their own bespoke data formats, documents, and interfaces.

I'm not sure where this effort will go, but wanted to disclose it since it is a major ongoing effort of ours internally, and it's very relevant to completely eliminating typos like this once and for all.

-Tim

The two August 31 items are on track for completion by the promised deadline.

Both August 31 items are code-complete and in final review and testing.

Both items are scheduled for deployment today. We will post an update when they are deployed.

Block for value of “Gmb” in Org names – Prevent – 31-Aug-2024

A reminder for the 2nd approver that will show a manual edit has been made, including the before and after values so the second approver can more easily see what the first specialist did. – Prevent – 31-Aug-2024

these have both been deployed.

The remaining action item here is the AI project. I'm working with the relevant product manager to see if we can get it into shape to present and discuss at the upcoming Seattle CA/Browser Forum F2F.

No news this week.

Looks like the AI part probably won't be ready to present by the Seattle F2F; I'm still trying to see a demo myself.

You need to log in before you can comment on or make changes to this bug.