1798626 - SHECA: UniTrust: EV certificate with wrong Registry Country Name

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[chenxiaotong]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6.1 Safari/605.1.15

Comment 1

[chenxiaotong]

Accident report

1.How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.
In August 31st SHECA was made aware of this problem via this Bugzilla.

2.A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.
20220831 10:00 CST SHECA was aware of this issue by Bugzilla notification.
20220831 10:30 CST Investigation is started by Information Security & Compliance.
20220831 11:00 CST Confirmed the certificate was mis-issued, and suspended any EV SSL certificates to be issued immediately.
20220831 14:30 CST Started review of historically issued certificates and outstanding requests. Initiated root cause analysis.
20220901 15:30 CST Certificate revoked.
20220902 16:30 CST Root cause Confirmed.
20220905 16:30 CST Updated linter in production system.

3.Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.
Yes, we have already stopped issuing any EV certificates once we are aware of the issue.

4.A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.
See #5.

5.The complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem.
https://crt.sh/?id=7398207090
https://crt.sh/?id=7398187286
https://crt.sh/?id=7398138453
https://crt.sh/?id=6313246866

6.Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.
The jurisdictionCountryName doesn't match the jurisdictionStateOrProvinceName was caused by default value of jurisdictionCountryName is set to CN. Vetting member haven’t choose the correct value accordingly due to lack of overseas vetting experience.

7.List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.
The impacted certificate has been revoked and we have updated the linter which validates jurisdictionCountryName and countryName. We have strengthened vetting team training, will avoid any similar issue happens again.

Comment 2

[Mathew Hodson]

(In reply to chenxiaotong from comment #1)

2.A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.
20220831 10:00 CST SHECA was aware of this issue by Bugzilla notification.
20220831 10:30 CST Investigation is started by Information Security & Compliance.
20220831 11:00 CST Confirmed the certificate was mis-issued, and suspended any EV SSL certificates to be issued immediately.
20220831 14:30 CST Started review of historically issued certificates and outstanding requests. Initiated root cause analysis.
20220901 15:30 CST Certificate revoked.
20220902 16:30 CST Root cause Confirmed.
20220905 16:30 CST Updated linter in production system.

This timeline seems incomplete. https://crt.sh/?id=6313246866 shows the revoked date in the CRL as 2022-03-10. You acknowledged the issue with https://crt.sh/?id=7398187286 on 2022-09-07 in bug 1787537. Can you verify the date and times for when you became aware of the issue for each certificate and when it was revoked and make sure they are included in the timeline.

Has UniTrust scanned all issued certificates and confirmed there are no other certificates with similar issues? Can you provide details on when that was done and how the search was performed?

6.Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.
The jurisdictionCountryName doesn't match the jurisdictionStateOrProvinceName was caused by default value of jurisdictionCountryName is set to CN. Vetting member haven’t choose the correct value accordingly due to lack of overseas vetting experience.

7.List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.
The impacted certificate has been revoked and we have updated the linter which validates jurisdictionCountryName and countryName. We have strengthened vetting team training, will avoid any similar issue happens again.

Can you provide more details on what linting was implemented at the time the affected certificates were issued? What is the updated behaviour of the linter? Can you describe in more detail what fields are checked and in what way? How will you prevent incorrect values in other fields?

What was the training before and what specific steps did you tale to strengthen that training?

Comment 3

[Chris Clements]

SHECA, please provide a detailed response to the questions asked in Comment 2.

Announced in the Chrome Root Program policy v1.4, questions should have a response provided within seven calendar days.

Comment 4

[chenxiaotong]

(In reply to Mathew Hodson from comment #2)

(In reply to chenxiaotong from comment #1)

2.A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.
20220831 10:00 CST SHECA was aware of this issue by Bugzilla notification.
20220831 10:30 CST Investigation is started by Information Security & Compliance.
20220831 11:00 CST Confirmed the certificate was mis-issued, and suspended any EV SSL certificates to be issued immediately.
20220831 14:30 CST Started review of historically issued certificates and outstanding requests. Initiated root cause analysis.
20220901 15:30 CST Certificate revoked.
20220902 16:30 CST Root cause Confirmed.
20220905 16:30 CST Updated linter in production system.

This timeline seems incomplete. https://crt.sh/?id=6313246866 shows the revoked date in the CRL as 2022-03-10. You acknowledged the issue with https://crt.sh/?id=7398187286 on 2022-09-07 in bug 1787537. Can you verify the date and times for when you became aware of the issue for each certificate and when it was revoked and make sure they are included in the timeline.

Has UniTrust scanned all issued certificates and confirmed there are no other certificates with similar issues? Can you provide details on when that was done and how the search was performed?

6.Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.
The jurisdictionCountryName doesn't match the jurisdictionStateOrProvinceName was caused by default value of jurisdictionCountryName is set to CN. Vetting member haven’t choose the correct value accordingly due to lack of overseas vetting experience.

7.List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.
The impacted certificate has been revoked and we have updated the linter which validates jurisdictionCountryName and countryName. We have strengthened vetting team training, will avoid any similar issue happens again.

Can you provide more details on what linting was implemented at the time the affected certificates were issued? What is the updated behaviour of the linter? Can you describe in more detail what fields are checked and in what way? How will you prevent incorrect values in other fields?

What was the training before and what specific steps did you tale to strengthen that training?

Hello Mathew，for the https://crt.sh/?id=6313246866 was issued and revoked on Mar.10 2022 by customer. We were aware of the issue on Aug. 31 2022, then reviewed the historically issued certificates, of which included it.

We scanned all issued certificates and confirmed there are no other certificates with similar issues in our system，checked all EV-certificates which we issued, and checked other type certificates on a pro-rata. We can provide reasonable assurance that the similar problems are solved.

We have trained operators on the information they need to pay attention when issuing certificates, updated the operating procedures for issuing certificate, and conducted regular checks on issued certificates.

Comment 5

[Mathew Hodson]

Dates weren't provided for all the certificates even after they were requested, but we can see from bug 1787537 that SHECA became aware that https://crt.sh/?id=7398187286 was misissued on 2022-09-07 and the revocation date in the CRL is 2022-09-19. That doesn't meet the five day requirement from the BRs and would need a separate incident report.

Comment 6

[Ben Wilson]

Please provide an update on the status of your remediation steps related to this bug. As required by root store policies (https://www.ccadb.org/cas/incident-report), incident reports must be updated on a weekly basis "until you confirm that the resolution steps have been completed, unless a Root Store Operator has agreed to a different schedule by setting a 'Next Update' date in the 'Whiteboard' field of the bug or has announced they consider closing the bug and no further comments have been posted."

Comment 7

[jasmine.tang]

(In reply to Mathew Hodson from comment #5)

Dates weren't provided for all the certificates even after they were requested, but we can see from bug 1787537 that SHECA became aware that https://crt.sh/?id=7398187286 was misissued on 2022-09-07 and the revocation date in the CRL is 2022-09-19. That doesn't meet the five day requirement from the BRs and would need a separate incident report.

Please refer to the following link for a separate incident report：https://bugzilla.mozilla.org/show_bug.cgi?id=1856503
Please let us know if there are any further questions or concerns about this bug.Thank you.

Comment 8

[Ben Wilson]

I plan to close this item on Wed. 11-Oct-2023, unless there are additional questions or concerns expressed.