Closed Bug 1904038 Opened 1 year ago Closed 4 months ago

Chunghwa Telecom: “Test Website - Valid" URL disclosed to CCADB is expired

Categories

(CA Program :: CA Certificate Compliance, task)

Product:
CA Program
Component:
CA Certificate Compliance
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: tmkuo, Assigned: tmkuo)

Details

(Whiteboard: [ca-compliance] [policy-failure])

Incident Report

Summary

During the email communication with the Browser contact, this problem was found and notify by the Browser contact.

Impact

A total of 1 certificate (good.epkiov.hinet.net) is affected.

Timeline

All times are UTC.

2024-06-12

  • 00:22 Received the response email including the notification of expired.
  • 17:43 Investigated and reviewed the cause.

2024-06-13

  • 10:51 Submit the certificate request.
  • 17:17 Complete the review and certificate issuance.
  • 18:30 Complete certificate replacement.

Root Cause Analysis

The CA system sent an expiration notification letter, but the website administrator of ePKI ignored it.

Lessons Learned

  • Even well-trained administrators can make low-level mistakes, we have warned the website administrator of ePKI should not ignore again, and calendar reminders is set to avoid the same mistake.

Where we got lucky

Action Items

Action Item Kind Due Date
Complete certificate replacement. Detect 2024-06-13
Conduct a discussion and review meeting, and calendar reminders is set to avoid the same mistake. Prevent 2024-06-20

Appendix

Details of affected certificate

https://good.epkiov.hinet.net

Based on Incident Reporting Template v. 2.0

Does Chunghwa Telecom believe this incident report is complete? It is remarkably thin on details and I think compares very unfavourably to reports being submitted by your peers in the web PKI community.

In particular, the timeline seems to be missing key events like the initial issuance of the certificate, and the date the certificate expired. I also believe your action items fail to provide meaningful assurance that this event won't happen again.

Why isn't the deployment of certificate renewal automation being considered as an action item?
What about improvements to your own monitoring practices as an action item? Why was this not discovered by your own team?

Assignee: nobody → tmkuo
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Whiteboard: [ca-compliance] [policy-failure]

(In reply to Daniel McCarney from comment #1)

Does Chunghwa Telecom believe this incident report is complete? It is remarkably thin on details and I think compares very unfavourably to reports being submitted by your peers in the web PKI community.

In particular, the timeline seems to be missing key events like the initial issuance of the certificate, and the date the certificate expired.

Let me restructure the timeline as follows:

Timeline

All times are UTC.

2022-09-08:

2023-09-08

  • 19:59 The expiration of the cert.

2024-06-12:

  • 00:22 Received the response email from the browser, including the notification of expired, where the cert expired 278 days ago.
  • 17:43 Investigated and reviewed the cause.

2024-06-13

  • 10:51 Submit the certificate request.
  • 17:17 Complete the review and certificate issuance.
  • 18:30 Complete certificate replacement.

I also believe your action items fail to provide meaningful assurance that this event won't happen again.
Why isn't the deployment of certificate renewal automation being considered as an action item?
What about improvements to your own monitoring practices as an action item? Why was this not discovered by your own team?

Thanks for your comment, we will discuss ways to automate certificate renewal and whether it can be achieved in a short time.

The feasibility of certificate renewal automation is confirmed which can be deployed in our new CA system. We are still evaluating the timeline of this feasibility.

We have deployed an automate certificate renewal mechanism in our test CA environment, and it is still under testing.

The testing of the automate certificate renewal mechanism in our test CA environment is finished, we will deploy it to our production environment next week.

Currently, ACME has been implemented in our CA production environment, and the certificate for test website-valid (with FQDN good.hipkig1ov.hinet.net) have been automatically applied and installed in a scheduled manner on 2024-09-19.

We are continuing to monitor this issue.

We are continuing to monitor this issue.

All action Items disclosed in this incident have been completed and and our CA system has implemented an automatic certificate renewal mechanism, so we request to close this incident.

We are continuing to monitor this issue.

We are continuing to monitor this issue.

We are continuing to monitor this issue.

We are continuing to monitor this issue.

We are continuing to monitor this issue.

Report Closure Summary

  • Incident description: “Test Website - Valid" URL (good.epkiov.hinet.net) disclosed to CCADB is expired for 278 days, resulting in non-compliance with the TLS BR Section 2.2 and Root Program policies. These policies require CAs, at a minimum, shall host separate Web pages using subscriber certificates that are valid, revoked, and expired.

  • Incident Root Cause(s): The CA system sent an expiration notification letter, but the website administrator of ePKI ignored it. The other root cause was that we did not an automate certificate renewal and scheduling mechanism at the time.

  • Remediation description: Our CA system has implemented an automatic certificate renewal mechanism. In addition, the expiration and automatic certificate renewal notice of test websites will be sent to multiple CA roles for multiple confirmations of the effective scheduling mechanism.

  • Commitment summary: Even well-trained administrators can make low-level mistakes, we have warned the website administrator of ePKI should not ignore again. As mentioned, the calendar reminders and the system notice of expiration and automatic certificate renewal are sent to multiple CA roles to avoid the same mistake. For these test websites, the automated solution is adopted in our CA system to minimize "hands-on" input required from humans during certificate issuance and renewal. With ACME, the only "hands-on" input from humans are initial software installation and configuration, applying software updates, and updating subscriber account information.

All Action Items disclosed in this report have been completed as described, and we request its closure.

Unless there are additional questions or comments to discuss, I will close this sometime later next week (Thurs - Fri, 17 or 18 of April).

Flags: needinfo?(bwilson)
Status: ASSIGNED → RESOLVED
Closed: 4 months ago
Flags: needinfo?(bwilson)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.