1906470 - Entrust: S/MIME mailbox address case mismatch between subject and subjectAltName

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Bruce Morton]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36

Comment 1

[Bruce Morton]

Preliminary Incident Report

Summary

On 2024-07-02 19:50 UTC we received an internal certificate problem report based on a pkilint error about a potential certificate mis-issuance provide in this bug #1906467.

As part of that investigation, we ran pkilint on all S/MIME certificates which attest to have been issued in accordance with the S/MIME Baseline Requirements. We found two (2) S/MIME certificates which were issued with a mailbox address case mismatch between the subject name and the subjectAltName, and were therefore not repeated as required by section 7.1.4.2.1 of the S/MIME Baseline Requirements, which states:

All Mailbox Addresses in the subject field or entries of type dirName of this extension SHALL be repeated as rfc822Name or otherName values of type id-on-SmtpUTF8Mailbox in this extension.

Impact

Two (2) S/MIME Sponsor-Validated certificates issued at 2023-10-05 14:58 UTC and 2024-02-19 07:51 UTC. These certificates were not expired or revoked when the problem was identified.

Timeline

All times are UTC.

2024-07-03:

• 10:58 Investigation of another incident bug #1906467 determined that a certificate was mis-issued. Directions were provided to halt S/MIME certificate issuance and to search the S/MIME certificate database for any other impacted certificates.
• 13:13 S/MIME certificate issuance was halted.
• 19:07 pkilint back test data on previously issued certificates was provided and analysis begun.

2024-07-04:

• 05:04 Investigation determined that 2 S/MIME certificates were mis-issued per mis-issued S/MIME BR section 4.9.11 item 11.
• 17:03 Informed subscriber 1 of impacted certificate.
• 17:21 Informed subscriber 2 of impacted certificate.

Next steps

All impacted certificates will be revoked on or before 2024-07-09 05:04 UTC, which is per S/MIME BR section 4.9.1.1 item 11, 5-day timeline.

A full incident report including the root cause analysis, lessons learned, and action items will be posted within two weeks per the CCADB full incident report requirement, which is on or before 2024-07-18 05:04 UTC.

Comment 2

[Bruce Morton]

The impacted certificates were revoked 2024-07-08 11:44:50 UTC and 22:11:42 UTC, respectively.

Comment 3

[Bruce Morton]

We are working on the full incident report and it will be posted per schedule.

Comment 4

[Bruce Morton]

Incident Report

Summary

On 2024-07-02 19:50 UTC we received an internal notification of a pkilint error about a potential certificate mis-issuance, separately reported as bug #1906467.

As part of that investigation, we ran pkilint on all S/MIME certificates which attest to have been issued in accordance with the S/MIME Baseline Requirements. We found two (2) S/MIME certificates which were issued with a mailbox address case mismatch between the subject name and the rfc822Name subjectAltName, and were therefore not repeated as required by section 7.1.4.2.1 of the S/MIME Baseline Requirements, which states:

All Mailbox Addresses in the subject field or entries of type dirName of this extension SHALL be repeated as rfc822Name or otherName values of type id-on-SmtpUTF8Mailbox in this extension.

Accordingly, revocation was required within 5 days per S/MIME BR section 4.9.1.1 item 11.

Impact

Two (2) S/MIME Sponsor-Validated certificates issued at 2023-10-05 14:58 UTC and 2024-02-19 07:51 UTC. The mailbox address in each certificate subject email field were not a character-to-character match to the mailbox address in the subjectAltName; as such mailbox addresses were not repeated. These certificates were not expired or revoked when the problem was identified.

Timeline

All times are UTC.

2024-07-03:

• 10:58 Investigation of another incident, bug #1906467, determined that a certificate was mis-issued. Directions were provided to halt S/MIME certificate issuance and to search the S/MIME certificate database for any other impacted certificates.
• 13:13 As an immediate incident response, S/MIME certificate issuance was halted.
• 19:07 pkilint back test data on previously issued certificates was provided and analysis begun.

2024-07-04:

• 05:04 Investigation determined that 2 S/MIME certificates were mis-issued because they were not in accordance with section 7.1.4.2.1 of the S/MIME Baseline Requirements (S/MIME BRs) and revocation was required within 5 days per S/MIME BR section 4.9.1.1 item 11.
• 17:03 Informed subscriber 1 of impacted certificate.
• 17:21 Informed subscriber 2 of impacted certificate.

2024-07-05:

• 18:51 Software fix deployed to reject mailbox address for the subject email field, if different from the mailbox address in the subjectAltName. Certificate issuance was re-instated.

2024-07-08:

• 11:44:50 First impacted certificate was revoked.
• 22:11:42 Second impacted certificate was revoked.

Root Cause Analysis

Why was the issue not detected by pkilint, similar to when the mis-issued certificate described in #1906467 was detected, but instead only identified through a manual scan?

These certificates were issued prior to April 2024 when we implemented pkilint as a post issuance check.

Why does the mailbox address in the CN field of the impacted certificates have different casing than the mailbox address in the subjectAltName field?

Due to the design of our CA software that issues some of our Sponsor-Validated S/MIME certificates, if a customer requested a certificate with a DN matching existing certificates (but with a case change in the mailbox address in the E= field) where the latest previous certificate had expired, our CA software would retain the case of the E= field from the previous certificate and include the mailbox address from the new request in the RFC822Name in the subjectAltName field. Due to the design of the CA software that issues some of our Sponsor-Validated S/MIME certificates, if a customer requested a certificate with a DN matching existing certificates, and a case change in the mailbox address in the E= field, and where the latest previous certificate had expired, our CA software would retain the case of the E= field from the previous certificate and include the mailbox address from the new request in the RFC822Name in the subjectAltName field. We have made changes to block issuance of S/MIME certificates where the latest certificate issued with a case-insensitively matching DN has expired.We have made changes to address this case and block issuance of S/MIME certificates if the latest certificate request does not exactly match the case of the DN in the expired certificate.

Why did our CA software prevent the case change of the email attribute during a certificate rekey?

A bug in one version of our S/MIME CA software prevented DN case changes for users whose certificates had expired. The software is scheduled to be deprecated by September 2024.

Why was the incident not detected before a manual review was conducted?

When these certificates were issued we had zlint post-issuance but this test case was not covered at that time. Our current pkilint post-issuance check was implemented after these certificates were issued. Pre-issuance linting for S/MIME will be added to use zlint and pkilint.

Lessons Learned

What went well

• our investigation of [bug 1906467] was sufficiently broad that these previously undetected issues were identified
• only two mis-issued certificates
• certificate issuance had already been halted, so no other mis-issued certificates were issued
• revocation was completed within the required timeframe

What didn't go well

• Issue detected after certificate signing

Where we got lucky

• only two certificates/subscriber were impacted

Action Items

Action Item	Kind	Due Date
Deprecate existing CA software	Prevent	2024-08-31
Implement pre-sign linting for S/MIME	Detect	2024-11-30
Institute re-scanning of previously issued unexpired certificates (including revoked) as part of release process when introducing updated linters	Detect	2024-07-17

Appendix

Details of affected certificates

Issuer: https://crt.sh/?caid=12549

• S/N: 3C7AC3C16A2B8B02000000004C3D9978; SHA-256 Fingerprint: 3B83442FF64EDBC8D8014DA546254CC61655D6C212FE02C4AE6C0720914E2051
• S/N: 2CF9C22BF6397B0E000000004C3D603F; SHA-256 Fingerprint: C6CDAC4AA517097A8573E74919A0EF086F8C3A834F499AB1205674CD385EE894

Comment 5

[Bruce Morton]

Action Items

Action Item	Kind	Due Date
Deprecate existing CA software	Prevent	2024-08-31
Implement pre-sign linting for S/MIME	Detect	2024-11-30
Institute re-scanning of previously issued unexpired certificates (including revoked) as part of release process when introducing updated linters	Detect	Done

Comment 6

[Bruce Morton]

Based on https://bugzilla.mozilla.org/show_bug.cgi?id=1906467#c8, we have created a delayed revocation incident report for that incident which also extends to this incident, see delayed revocation bug https://bugzilla.mozilla.org/show_bug.cgi?id=1910237.

Comment 7

[Paul van Brouwershaven]

We have no updates for this week but will keep following this bug for any questions or feedback.

May we please set the next update at 2024-08-31.

Comment 8

[Bruce Morton]

Action Items

Action Item	Kind	Due Date
Deprecate existing CA software	Prevent	Done
Implement pre-sign linting for S/MIME	Detect	2024-11-30
Institute re-scanning of previously issued unexpired certificates (including revoked) as part of release process when introducing updated linters	Detect	Done

The existing CA software has been deprecated.

May we please set the next update to be 31 October 2024? Thanks.

Comment 9

[Bruce Morton]

We are on track to implement pre-sign linting for S/MIME before 2024-11-30.

May we please set the next update to be 2024-11-30? Thanks.

Comment 10

[Bruce Morton]

Action Items

Action Item	Kind	Due Date
Deprecate existing CA software	Prevent	Done
Implement pre-sign linting for S/MIME	Detect	Done
Institute re-scanning of previously issued unexpired certificates (including revoked) as part of release process when introducing updated linters	Detect	Done

Pre-sign linting for S/MIME deployed 2024-11-07.

All actions are completed. We request this bug be closed. Thanks.

Comment 11

[Bruce Morton]

All actions are completed. We request this bug be closed. Thanks.

Comment 12

[Ben Wilson]

It appears that there have been no additional issues raised by the community? I'll look at closing this on 4-Dec-2024. Meanwhile, even though it is not yet a "requirement", could Entrust submit a "Closing Summary"? Here is an example: https://bugzilla.mozilla.org/show_bug.cgi?id=1927675#c7.

A closing summary should briefly:

• describe the incident, its root cause(s), and remediation;
• summarize any ongoing commitments made in response to the incident; and
• attest that all Action Items have been completed.

Here is a markdown template:

Incident Report Closure Summary

• Incident Description: [Two or three sentences summarizing the incident.]
• Incident Root Cause(s): [Two or three sentences summarizing the root cause(s).]
• Remediation Description: [Two or three sentences summarizing the incident's remediation.]
• Commitment Summary: [A few sentences summarizing ongoing commitments made in response to this incident.]

All Action Items disclosed in this Incident Report have been completed as described, and we request its closure.

Comment 13

[Bruce Morton]

Incident Report Closure Summary

• Incident Description: Two (2) S/MIME certificates which were issued with a mailbox address case mismatch between the subject name and the rfc822Name subjectAltName.
• Incident Root Cause(s): A bug in one version of our S/MIME CA software prevented DN case changes for users whose certificates had expired; as such, the subject name was not updated.
• Remediation Description: Entrust Certificate Services (ECS) has deprecated older CA software from S/MIME CA and migrated all users to newer CAs. Implement pre-sign linting for S/MIME certificate issuance.
• Commitment Summary: ECS has also deprecated the older version of the CA software and have migrated to new CA software for all certificate types. ECS has deployed pre-sign linting to S/MIME, TLS, Code Signing, Client Authentication, and eIDAS qualified certificates. We are working to deploy pre-sign lining to all public trust certificate types.

All Action Items disclosed in this Incident Report have been completed as described, and we request its closure.

Comment 14

[Ben Wilson]

I will close this on Wed. 4-Dec-2024.