1943528 - Entrust: delayed revocation

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Hanno Boeck]

On 2024-01-22 (16:13 GMT), I reported a certificate with a compromised private key to Entrust's problem reporting address. (The key was compromised in the Fortigate leak.)

I have not received any reaction from Entrust yet, and the certificate is not yet revoked.

The affected cert:
https://crt.sh/?id=15076085055

Comment 1

[Paul van Brouwershaven]

Thank you for bringing this to our attention. Upon receiving a copy of the original email from the reporter, we verified the key compromise and promptly revoked the affected certificate and blocked the compromised key.

We are also investigating why the original CPR was not received by our support team.

A detailed incident report will be provided once our investigation is complete, but no later than Friday, February 7th.

Comment 2

[Bruce Morton]

Incident Report

Summary

On 2025-01-22 (16:13 UTC), a Certificate Problem Report (CPR) was submitted for a subscriber certificate key compromise through our official reporting email address by the reporter of this bug. Entrust was unaware of this CPR until the creation of this bug on Bugzilla on 2025-01-24 (08:44 UTC). Upon investigation, it was determined that the CPR had been quarantined by the Proofpoint Email Protection solution. The summary of spam messages caught and placed into quarantine was only sent out once every 24 hours and was not reviewed by the tracking system.

Impact

This report affected a single certificate ( https://crt.sh/?id=15076085055 ) which was revoked on 2025-01-24 at 14:38 UTC. The certificate was revoked 36 hours and 47 minutes after the original CPR was submitted, and 3 hours and 44 minutes from the time we were made aware of the issue.

Timeline

All times are UTC.

2025-01-22:

• 16:13 UTC: A Certificate Problem Report (CPR) was submitted for a subscriber certificate key compromise through our official reporting email address.

2025-01-24:

• 08:44: The CPR submitter reported this bug on Bugzilla.
• 09:20: An internal compliance issue was created to track this report.
• 09:25: Our support team was unable to locate the CPR in question.
• 09:44: Initial investigation showed that this CPR was not recorded by our tracking system, the email was not received by the applicable shared mailbox, and it was not in the junk folder. However, similar CPRs by the same reporter around the same time were addressed accordingly.
• 09:44: The reporter was requested to provide a copy of the report to help with the investigation and verify the key compromise.
• 13:38: The reporter provided our team with a copy of the original CPR and the mail server log files showing that the message was accepted.
• 14:38: The report was verified and the certificate in question was revoked.
• 14:55: The subscriber was informed about the revocation.
• 15:09: This bug was updated with an acknowledgment and a status update (see Comment 1).
• 16:10: Investigation showed that the CPR was quarantined as spam by Proofpoint. After releasing the CPR from quarantine, a corresponding ticket was created.
• 16:33: Further investigation showed that a spam digest email was received by the CPR email box but was not noticed or processed. Even if noticed, it would have been after the 24-hour window to act on this report.
• 16:48: The issue was escalated to the InfoSec team to identify potential resolutions.
• 17:00: Support was requested to update its procedures to monitor the CPR mailbox for unprocessed messages and to schedule spam digest emails multiple times a day.

2025-02-03:

• The Proofpoint configuration was updated to ensure that the spam digest email is sent every 4 hours whenever any messages are quarantined.

2025-02-04:

• A CPR mailbox rule was established to forward the spam digest email to support and compliance for better visibility.

Root Cause Analysis

1. Why was the certificate, whose key was reported compromised, not revoked within 24 hours?

Because the Certificate Problem Report (CPR) was not received by the appropriate team.

2. Why was the CPR not received?

Because the CPR email was quarantined by the Proofpoint Email Protection solution.

3. Why was it quarantined?

The email was classified as spam by the Proofpoint system, possibly due to the inclusion of a certificate and private key in the message body. However, the precise reason remains unclear. It is necessary to consider that CPR reports may be flagged by spam filters.

4. Why was the spam digest missed?

Because the spam digest email, which lists all quarantined messages, was sent out only once every 24 hours and was ignored by the tracking system. Additionally, there was no formal procedure in place to check the CPR mailbox for spam digest reports.

5. Why was there no formal procedure in place to check the CPR mailbox for spam digest reports?

Because monitoring the CPR mailbox for spam digest reports was not considered, leading to the absence of a formal procedure.

Conclusion

The root cause of the issue was the incorrect identification of the CPR email as spam by the Proofpoint system, combined with the infrequent and unmonitored spam digest emails. This resulted in us being unaware of the CPR and thus a delay in the revocation of the compromised certificate.

Lessons Learned

What went well

• The Bugzilla report was picked up quickly

What didn't go well

• A spam digest email was received by the CPR email box but was not noticed or processed.

Where we got lucky

• The CPR reporter reported this on Bugzilla and responded promptly to our information requests.

Action Items

Action Item	Kind	Due Date
Update the frequency SPAM digest emails to every 4 hours	Detect	Completed
Check the CPR mailbox for unprocessed message and SPAM digest emails	Prevent	Completed
Forward a copy of the SPAM digest emails for the CPR mailbox to support and compliance	Prevent	Completed

Appendix

Details of affected certificates

https://crt.sh/?id=15076085055

Comment 3

[Bruce Morton]

Incident Report Closure Summary

• Incident Description: A Certificate Problem Report (CPR) was submitted for a subscriber certificate key compromise through our official reporting email address by the reporter of this bug. Entrust was unaware of this CPR until the creation of this bug. Upon investigation, it was determined that the CPR had been quarantined by the Proofpoint Email Protection solution. The summary of spam messages caught and placed into quarantine was only sent out once every 24 hours and was not reviewed by the tracking system.
• Incident Root Cause(s): The root cause of the issue was the incorrect identification of the CPR email as spam by the Proofpoint system, combined with the infrequent and unmonitored spam digest emails. This resulted in us being unaware of the CPR and thus a delay in the revocation of the compromised certificate.
• Remediation Description: Update the frequency SPAM digest emails to every 4 hours. Check the CPR mailbox for unprocessed message and SPAM digest emails. Forward a copy of the SPAM digest emails for the CPR mailbox to support and compliance.
• Commitment Summary: Entrust committed to ensuring that CPRs, incidents, and bugs are addressed in accordance with the CCADB policy to help ensure the community is aware of the issues and how the issues are addressed, and to help ensure protection of relying parties.

All Action Items disclosed in this Incident Report have been completed as described, and we request its closure.

Comment 4

[Ben Wilson]

I'll close this on Wed. 19-Feb-2025.