Li-Chun Chen submitted the following incident report to the mozilla.dev.security.policy mailing list:
- How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.
One of our staffs in PKI group was taking samples of the issued certificates from crt.sh database for some reasons and found a mis-issued certificate which has a test (unregistered) FQDN on February 15, 2019 at 1:55 (UTC). He then notified us (Public CA) of the incident. We decide to make an initial investigation and found another mis-issued certificate which also has a test FQDN on February 15, 2019 at 2:30 (UTC).
- A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.
Timeline (all times are UTC):
15 February 2019 1:55: Find a mis-issued certificate with a FQDN www.raotest.com.tw
15 February 2019 1:59: Revoke the first mis-issued certificate
15 February 2019 2:07: Public CA starts an action plan and initial investigation
15 February 2019 2:17: Further certificate issuing suspended
15 February 2019 2:30: Find another mis-issued certificate with a FQDN publicca.rao.com.tw
15 February 2019 4:10: Initial investigation completed
15 February 2019 4:25: Certificate issuing restarted
15 February 2019 4:40: Hold an investigation meeting
- Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.
Yes, we had stopped issuing certificates before we investigated and revoked any certificate with an unregistered FQDN. We have asked our CA system vendor to include an automatic FQDN-checking function in the hotfix which will be released not after 30 March 2019 to avoid making the same mistakes.
- A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.
Number of certs: 2
First cert: issued on Nov 12, 2018 at 11:53:02 (UTC)
Last cert: issued on Jan 29, 2019 at 06:43:59 (UTC)
- The complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem.
https://crt.sh/?id=1153958924 with FQDN www.raotest.com.tw
https://crt.sh/?id=940459864 with FQDN publicca.rao.com.tw
- Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.
For the certificate with FQDN www.raotest.com.tw:
One of our RAOs intended to take a screenshot of certificate application process for training material in test environment, but she was not aware that she is in the formal environment. After the certificate was issued, the RAO forgot to revoke it.
For the certificate with FQDN publicca.rao.com.tw:
The same as the former cause, but the mis-issued certificate had been revoked immediately without notifying Public CA.
The mistakes have not been detected (even by our internal self-audit) because the amount of mis-issued certificates is minor. The RAO involved in this incident has been verbally warned and needs to undergo a retraining process in accordance with our employment contract.
- List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.
To avoid making the same mistakes, the following steps will newly be introduced:
Step 1. Implementation of a two-stage manual verification by different RAOs. Effective 26/02/2019.
Step 2. Implementation of an automatic FQDN-checking function prior to issuing certificates. Effective 30/03/2019.
Step 3. Implementation of a scheduling program to periodically and automatically check the newly-issued certificates from our repository. Effective 30/05/2019.