1890123 - Entrust: Failed to provide a preliminary incident report according to TLS BR 4.9.5

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Paul van Brouwershaven]

Preliminary Incident Report

We did not provide a preliminary incident report to the subscribers and the entity who filed the certificate problem report in bug 1883843 as required by the TLS Baseline Requirements section 4.9.5.

We will provide a full incident report including a list of all impacted certificates on or before 2024-04-19.

Comment 1

[Paul van Brouwershaven]

Entrust: Failed to provide a preliminary incident report according to TLS BR 4.9.5

Incident Report

Summary

We did not provide a preliminary incident report to the subscribers and the entity who filed the certificate problem report in bug 1883843 as required by the TLS Baseline Requirements, section 4.9.5.

This section requires that:

Within 24 hours after receiving a Certificate Problem Report, the CA SHALL investigate the facts and circumstances related to a Certificate Problem Report and provide a preliminary report on its findings to both the Subscriber and the entity who filed the Certificate Problem Report.

We confirmed the mis-issuance to the reporter in just under 26 hours, and the preliminary report was posted within 43 hours of when the problem was reported.

We contacted subscribers affected by this incident once we decided to revoke all impacted certificates. We did this to avoid confusing subscribers with information about an error in the EV Guidelines for which we were initially not planning to revoke their certificates.

Impact

This incident impacts the certificate problem reporter and 944 subscribers of certificates related to bug #1883843.

Timeline

2024-03-04:

• 13:00 Received report via email from Ryan Dickson regarding 10+ certificates that seemed to be missing the required certificatePolicies:policyQualifiers:qualifier:cPSuri.
• 14:55 Reporter informed about investigation.

2024-03-05:

• 14:40 Mis-issuance confirmed to reporter, indicated that we will post a Bugzilla report with more details.

2024-03-06:

• 08:00 Incident report posted on Bugzilla, see bug #1883843.

Root Cause Analysis

1. Why was there a problem?

We did not provide a preliminary report to the subscribers and the entity who filed the certificate problem report in bug #1883843 as required by the TLS Baseline Requirements, section 4.9.5.

2. Why did we not provide a preliminary report within 24 hours as required?

It took us longer than expected to verify this unusual incident and discuss its implications.

3. Why was it not detected that we were about to exceed the 24 hours?

The certificate problem report flow did not include specific steps that needed to be completed within a given time frame or an automatic reminder to the case owner.

4. Why was this not included in the certificate problem report flow?

It was not included in our incident handling procedures.

5. Why was it not included in the incident handling procedures?

This was missed when writing the procedure.

Lessons Learned

• Our incident handling procedures didn’t include reminders to prevent this type of incident.

What went well

• We confirmed the receipt of the certificate problem report and that we started the investigation to the reporter within two hours.

What didn't go well

• We should provide a preliminary report within 24 hours. It took us 1 day, 1 hour and 40 minutes to confirm the mis-issuance to the reporter and 1 day, 19 hours to make a report available.
• While we informed the certificate problem reporter about where the preliminary incident report would be posted we should have provided the report or a direct link to the report on Bugzilla within the required timeframe.
• We did not inform subscribers within 24 hours of the certificate problem report.

Where we got lucky

Action Items

Action Item	Kind	Due Date
Improve incident management procedures	Prevent	2024-06-30

Appendix

Details of affected certificates

See bug #1883843.

Comment 2

[Paul van Brouwershaven]

We have no updates for this week and will continue to monitor the bug.

Comment 3

[Paul van Brouwershaven]

We have no updates and will continue to monitor the bug.

Comment 4

[Paul van Brouwershaven]

We have no updates and will continue to monitor the bug.

Comment 5

[Ben Wilson]

Based on Comment #1, I am setting the next update to 2024-06-17.

Comment 6

[ngook.kong]

We have no updates for this week but will have an update on “Improve incident management procedures" by 6/30 per our posted due date in Comment 1.
https://bugzilla.mozilla.org/show_bug.cgi?id=1890123#c1

We will continue to monitor the bug.

Comment 7

[Bruce Morton]

We have no updates, but plan to update action status this week.

Comment 8

[Paul van Brouwershaven]

All action items in this bug have been completed, we request this bug be closed.

Action Item	Kind	Due Date
Improve incident management procedures	Prevent	Complete

Comment 9

[Paul van Brouwershaven]

We have no updates and all action items have been completed, we request this bug to be closed.

Comment 10

[Ben Wilson]

We need to review the improved incident management procedure. Additionally, before this bug is closed, we intend that action items related to this bug (e.g., improving incident management procedures) be added to Bug #1901270 for Entrust to address. Such additional action items will serve as remediation for compliance issues identified in the June 21 report and in any responses to the report from Mozilla and the Mozilla community.

Comment 11

[Bruce Morton]

We have no updates for the incident at this time. Are waiting for the improved incident management procedure issues to be addressed.

Comment 12

[Bruce Morton]

(In reply to Ben Wilson from comment #10)

We need to review the improved incident management procedure. Additionally, before this bug is closed, we intend that action items related to this bug (e.g., improving incident management procedures) be added to Bug #1901270 for Entrust to address. Such additional action items will serve as remediation for compliance issues identified in the June 21 report and in any responses to the report from Mozilla and the Mozilla community.

We wanted to acknowledge your comment and let you know that we are actively gathering additional input and feedback from Ryan Hurst, who we recently brought on as an external consultant, and Michael Klieman, our new VP of Global Product Management, and we are making additional revisions to our Incident Response Plan and Change Management Policy to reflect this input. We believe these changes will improve the quality of these processes and documents. In addition, while initially prepared for only internal use, we are going through the materials so we can share them with you and other external parties to help provide evidence of the remediation steps we’ve taken.

Our current plan is to have updates and a version available for you completed by 7/31/2024. Is this timing acceptable?

Comment 13

[Bruce Morton]

(In reply to Ben Wilson from comment #10)

We need to review the improved incident management procedure. Additionally, before this bug is closed, we intend that action items related to this bug (e.g., improving incident management procedures) be added to Bug #1901270 for Entrust to address. Such additional action items will serve as remediation for compliance issues identified in the June 21 report and in any responses to the report from Mozilla and the Mozilla community.

Please see https://bugzilla.mozilla.org/show_bug.cgi?id=1901270#c7.

Comment 14

[Paul van Brouwershaven]

We kindly request closing this incident if there are no further questions about the provided documentation.

Comment 15

[Ben Wilson]

I'll look to close this on or about Friday, 9-Aug-2024.