1886467 - Entrust: clientAuth TLS Certificates without serverAuth EKU

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Paul van Brouwershaven]

Attachment: 20240320-clientAuth-certificates-without-serverAuth.txt

Incident Report

Summary

As part of #1883843, we identified 15 EV certificates with the Extended Key Usage (EKU) attribute set to id-kp-clientAuth but lacking the id-kp-serverAuth attribute. Our system allows customers to request certificates with both id-kp-serverAuth and id-kp-clientAuth or just one of these.

Section 7.1.2.7.10 of the TLS Baseline Requirements specifies that the id-kp-serverAuth (OID 1.3.6.1.5.5.7.3.1) MUST be present in all subscriber certificates issued under this policy.

Impact

This incident affects 1176 TLS certificates which only have the id-kp-clientAuth EKU.
While these certificates are non-compliant, they do not pose a security risk to subscribers, users, or the ecosystem in general.

Timeline

All times are UTC.

2024-03-20:

• 08:00 Suspicion on potential miss-issuance as part of processing the list of certificates in relation to incident #1883843
• 10:00 Confirmed suspicion and triggered the incident response procedure (waking up due to time zone differences)
• 10:30 Asked to stop issuance of certificates without id-kp-serverAuth EKU for all profiles linked directly or indirectly to the TLS BRs.
• 13:56 Issuance stopped.
• 14:00 We started to approach impacted customers and asked them to replace and revoke their id-kp-clientAuth only TLS certificates.

Root Cause Analysis

The Baseline Requirements v1.8.7 states in 7.1.2.3:

Either the value id-kp-serverAuth [RFC5280] or id-kp-clientAuth [RFC5280] or both values MUST be present.

Ballot SC-62v2 changed this behavior (see this redline) and forbid the issuance of certificates with just id-kp-clientAuth by making id-kp-serverAuth required.

Our systems allowed customers to set the appropriate key-usage based on their use-case (in accordance with 7.1.2.3 of the BRs at that time), with a default off id-kp-serverAuth and id-kp-clientAuth and the option to select id-kp-serverAuth or id-kp-clientAuth.

In the review of SC-62 we have not considered that the EKU can be configured by the customer and only considered the default option.

Lessons Learned

• When reviewing changed requirements, we should have a closer look at user configurable options.

What went well

• During the review of #1883843 we self-identified this non-compliance.

What didn't go well

• In the review of SC-62 we have not considered that the EKU can be configured by the customer.
• Linters such as zlint, pkilint, certlint and cablint did not detect this issue.

Where we got lucky

• The number of impacted certificates is relatively low

Action Items

Action Item	Kind	Due Date
Fix TLS BR EKU checking zlint	Mitigate	2024-04-30
Review all certificate profiles for any other potential errors	Prevent	2024-04-30

Appendix

Details of affected certificates

See attached list of certificates (these are being submitted to CT at the moment).

Comment 1

[Ryan Dickson]

Hi Paul,

Can you share more about how Entrust was evaluating these certificates using pkilint? While testing this locally, it does look like pkilint returns an error.

The following results are returned when evaluating the first certificate disclosed in the list of affected certificates (https://crt.sh/?q=B382D017898F8ACEC7377F3511999928853DCA1BB7544319C412A7B2F13A1CBE):

$ curl -s https://crt.sh/?d=12449094639 | lint_cabf_serverauth_cert lint -d - -s ERROR
SubscriberEkuAllowanceValidator @ certificate.tbsCertificate.extensions.7.extnValue.extKeyUsageSyntax
    cabf.serverauth.subscriber.serverauth_eku_absent (ERROR)

Further, evaluating the same certificate with zlint also appears to return an error.

$ zlint -pretty 12449094639.crt 

"e_key_usage_and_extended_key_usage_inconsistent": {
  "result": "error",
  "details": "KeyUsage [KeyEncipherment DigitalSignature] (00000101) inconsistent with ExtKeyUsage clientAuth"
 },

This is also displayed via crt.sh.

While it seems there is still opportunity to improve Zlint with the specific issue you cited in the Actions List --- given that an error was returned by the tool, can you help us understand the circumstances that did not lead to this being detected?

The reason why these answers are important is because they help us understand what didn't go right, and they may help prevent other future incidents across the ecosystem.

Comment 2

[Paul van Brouwershaven]

Apologies for the confusion. Just to clarify, we have implemented the other linters, but we haven't yet implemented or tested these certificates with pkilint. We are integrating pkilint into our post-issuance flow. The inclusion of pkilint in this incident report was accidental; it was included to assess whether it would detect this issue or not, allowing us to potentially contribute a check for it.

Comment 3

[Paul van Brouwershaven]

The error from zlint is from a lint that we helped to contribute and included in zlint 3.6, which has not yet been deployed within our infrastructure.

ERROR: The certificate MUST only be used for a purpose consistent with both key usage extension and extended key usage extension.

This error is related to clientAuth which should not be combined with Key Encipherment according to RFC 5280. But the original lint checking the EKUs for the Baseline Requirements has not been updated yet.

Comment 4

[Mathew Hodson]

(In reply to Paul van Brouwershaven from comment #0)

Timeline

All times are UTC.

2024-03-20:

• 08:00 Suspicion on potential miss-issuance as part of processing the list of certificates in relation to incident #1883843
• 10:00 Confirmed suspicion and triggered the incident response procedure (waking up due to time zone differences)
• 10:30 Asked to stop issuance of certificates without id-kp-serverAuth EKU for all profiles linked directly or indirectly to the TLS BRs.
• 13:56 Issuance stopped.
• 14:00 We started to approach impacted customers and asked them to replace and revoke their id-kp-clientAuth only TLS certificates.

The timeline is missing important dates: when did reviews take place if any, when did requirements take effect, when was the first certificate issued and therefore was the start of the incident? This is called out specifically at https://www.ccadb.org/cas/incident-report

"The Timeline section must include a detailed timeline of all events and actions leading up to and taken during and after the incident. The timeline must include not just the actual discovery of the incident and subsequent events, but also relevant events occuring beforehand (e.g. something changed or was introduced)."

Comment 5

[Paul van Brouwershaven]

(In reply to Mathew Hodson from comment #4)

The timeline is missing important dates: when did reviews take place if any, when did requirements take effect, when was the first certificate issued and therefore was the start of the incident? This is called out specifically at https://www.ccadb.org/cas/incident-report

"The Timeline section must include a detailed timeline of all events and actions leading up to and taken during and after the incident. The timeline must include not just the actual discovery of the incident and subsequent events, but also relevant events occuring beforehand (e.g. something changed or was introduced)."

Sorry for omitting this information in the initial incident report, here is an updated timeline:

Timeline

All times are UTC.

2023-03-31:

• Drafted certificate profile updates for SC-62.

2023-04-03:

• Updates reviewed by a second person of the compliance team.

2023-09-11:

• Updated certificate profiles deployed to production.

2023-09-15:

• SC-62 became effective.

2023-09-15:

• clientAuth only EKU is now prohibited in due to profile changes introduced by SC-62.

2024-03-04:

• 13:00 Report received from Ryan Dickson (in a personal capacity) about #1883843.

2024-03-19 (incident #1883843):

• 11:27 Uploaded first report with NULL pre-certificate link for clientAuth only certificates (as these are not logged to CT).
• 16:07 Aaron Gable commented about NULL entries in the updated report.
• 17:11 Investigation showed that the NULL entries relate to clientAuth only certificates (no suspicion on mis-issuance yet).

2024-03-20:

• 08:00 Suspicion on potential miss-issuance.
• 10:00 Confirmed suspicion and triggered the incident response procedure (waking up due to time zone differences).
• 10:30 Asked to stop issuance of certificates without id-kp-serverAuth EKU for all profiles linked directly or indirectly to the TLS BRs.
• 13:56 Issuance stopped.
• 14:00 We started to approach impacted customers and asked them to replace and revoke their id-kp-clientAuth only TLS certificates.

Comment 6

[Paul van Brouwershaven]

We are working on the delayed revocation in bug #1887705 for this incident, we don't have any comments on this bug for now but we are monitoring it for any questions.

Comment 7

[Paul van Brouwershaven]

If there are no other comments, it is requested set the next update to April 30, 2024.

Comment 8

[Paul van Brouwershaven]

In accordance with the action item "Fix TLS BR EKU checking zlint" we have submitted a pull request Add lint to cover TLS BR v2 EKU checks which is currently awaiting review.

Comment 9

[Bruce Morton]

Action Items

We are extending the due date to review all certificate profiles for any other potential errors to 10 May 2024.

Action Item	Kind	Due Date
Fix TLS BR EKU checking zlint	Mitigate	2024-04-30
Review all certificate profiles for any other potential errors	Prevent	2024-05-10

Comment 10

[Paul van Brouwershaven]

Updating action item "Fix TLS BR EKU checking zlint" to done as this lint has been proposed and merged.

Action Item	Kind	Due Date
Fix TLS BR EKU checking zlint	Mitigate	Done
Review all certificate profiles for any other potential errors	Prevent	2024-05-10

Comment 11

[Dimitris Zacharopoulos]

This is a very interesting bug and I would like to ask if this would be considered a mis-issuance if the certificatePolicies extension of the leaf certificates did not include any of the TLS BR policy OIDs.

The TLS BRs describe a profile of how a Technically Constrained non-TLS Issuing CA looks like but IMO it cannot, and should not mandate the profile of non-TLS leaf certificates based on the CA/Browser Forum Server Certificate Working Group Charter.

The intent of the CA was to issue client authentication certificates and that's the reason it did not include the id-kp-serverAuth KeyPurposeId in the EKU extension.

One interpretation is that since such certificates ARE NOT "TLS Server Certificates", they are out-of-scope of the TLS BRs.

A second interpretation is that by asserting a TLS BR-associated policy OID in these leaf certificates, they become in-scope of the TLS BRs.

My inclination is that despite, those certificates having been issued by a TLS-capable Issuing CA, they are out-of-scope of the TLS BRs. Provided these certificates follow RFC 5280 and can be properly parsed, Browsers should never consider such certificates server TLS-capable. They are by design "technically constrained".

Similarly, leaf certificates issued from other type of Subordinate CAs described in the TLS BRs (Technically Constrained non-TLS CAs), are also considered out-of-scope, regardless if they include a TLS BR OID in their certificatePolicies extension.

As long as the issued certificates are meeting RFC 5280, it should be ok.

Thoughts?

Comment 12

[Bruce Morton]

Action Item	Kind	Due Date
Fix TLS BR EKU checking zlint	Mitigate	Done
Review all certificate profiles for any other potential errors	Prevent	Done

The final action is complete; we will continue to monitor the bug.

Comment 13

[Clint Wilson]

(In reply to Dimitris Zacharopoulos from comment #11)

A second interpretation is that by asserting a TLS BR-associated policy OID in these leaf certificates, they become in-scope of the TLS BRs.

I believe this interpretation to be accurate based on the wording of Section 7.1.2 of the TBRs:
If the CA asserts compliance with these Baseline Requirements, all certificates that it issues MUST comply with one of the following certificate profiles....

---

Similarly, leaf certificates issued from other type of Subordinate CAs described in the TLS BRs (Technically Constrained non-TLS CAs), are also considered out-of-scope, regardless if they include a TLS BR OID in their certificatePolicies extension.

If they include a TLS BR OID, that seems to me to intrinsically bring the certificate into scope of the TBRs. A CA cannot assert a policy OID without complying with the policy represented by that OID (at least that's my reading of RFC 5280 Section 4.2.1.4).

I do think you raise a valid point worth further discussion. As I understand it, the intended outcome from a Relying Party perspective is to already have subordinate CAs which only issue a single type of certificate, i.e. subCAs must issue certificates containing specific (not anyEKU) EKUs and if a subCA issues a TLS certificate, then all certs that subCA issues need to include the serverAuth EKU -- issuing a clientAuth-only certificate from that same subCA would (and should) be a misissuance.

This is already mostly clear for subCAs issued in compliance with the profiles defined by SC-62, since Non-TLS subCAs cannot contain TBR policyIdentifiers and thus any certificates issued by those subCAs cannot contain TBR policyIdentifiers while being compliant with RFC 5280, while all TLS subCAs contain a TBR policyIdentifier and so restrict certificates being issued by those subCAs which don't comply with the TBRs.

The part that only makes this "mostly clear" is that Affiliated CA Certificates (including Affiliated Technically Constrained Non-TLS Subordinate CA Certificates) contain the anyPolicy policyIdentifier, and so could assert compliance with the TBRs in certificates they issue.
Perhaps this (imo intentional) constraint isn't adequately spelled out anywhere and perhaps it's not something that can be added or enforced by the TBRs (or SBRs), so perhaps this is something for Root Programs to clarify further instead.

Somewhat separately, it also seems like it may be worthwhile to require TLS-capable Roots reissue and revoke any non-SC-62 subCAs so we can ensure ecosystem-wide compliance with the SC-62 CA Certificate profiles. I would hope this could be avoided, and I haven't investigated whether this would be a widespread enough issue to warrant that type of rotation, but it would minimally help to remove some scenarios which otherwise lack sufficient clarity (for example, a somewhat similar question was raised as part of https://github.com/cabforum/servercert/issues/506).

FWIW, the reason I said above that this is what I think we're already supposed to have is that we're actively working to shift that expectation up one level, i.e. to a state where all Roots must only issue subCAs which contain specific EKUs and if a subCA is issued with the serverAuth EKU, then all subCAs and all certificates issued by all subCAs need to also include the serverAuth EKU.

Comment 14

[Mathew Hodson]

(In reply to Dimitris Zacharopoulos from comment #11)

My inclination is that despite, those certificates having been issued by a TLS-capable Issuing CA, they are out-of-scope of the TLS BRs. Provided these certificates follow RFC 5280 and can be properly parsed, Browsers should never consider such certificates server TLS-capable. They are by design "technically constrained".

Similarly, leaf certificates issued from other type of Subordinate CAs described in the TLS BRs (Technically Constrained non-TLS CAs), are also considered out-of-scope, regardless if they include a TLS BR OID in their certificatePolicies extension.

As long as the issued certificates are meeting RFC 5280, it should be ok.

Thoughts?

These certificates don't comply with Entrust's stated practices in their CPS, so it is clearly an incident. Otherwise, the individual root programs determine which certificates are in scope, so it would depend on the specific program requirements.

https://www.entrust.com/sites/default/files/documentation/licensingandagreements/entrust-certificate-services-cps-3-20.pdf

7.1.6.1 Reserved Certificate Policy Identifiers

Subscriber Certificates must include one of the following reserved Certificate Policy Identifiers, if the CA is asserting the Certificate meets the associated certificate policy:

SSL Certificates 2.23.140.1.2.2
EV SSL Certificates 2.23.140.1.1
Code Signing Certificates 2.23.140.1.4.1
EV Code Signing Certificates 2.23.140.1.3
Time-Stamp Certificates 2.23.140.1.4.2
Verified Mark Certificates 1.3.6.1.4.1.53087.1.1

Effective 1 September 2023:
S/MIME Mailbox-validated Strict 2.23.140.1.5.1.3
S/MIME Organization-validated Strict 2.23.140.1.5.2.3
S/MIME Sponsor-validated Legacy 2.23.140.1.5.3.1
S/MIME Sponsor-validated Strict 2.23.140.1.5.3.3

Certificates containing a reserved certificate policy identifier indicates compliance with the associated requirements and are issued and managed in accordance with those requirements.

Comment 15

[Dimitris Zacharopoulos]

Clint, Mathew, you both raise good points.

Similar to https://bugzilla.mozilla.org/show_bug.cgi?id=1887705, I think this deserves more discussion (my understanding and expectations/rationale are a bit different from Clint's position), probably in the CA/B Forum Server Certificate Working Group (SCWG) that "controls" the TLS BRs, but I think Mathew is not a member of the CA/B Forum so he can't post to the SCWG public mailing list.

@Mathew would you be interested in joining the CA/B Forum SCWG as an Interested Party? If joining the CA/B Forum is not an option and you are still interested to participate in this discussion, I would be happy to move it to m.d.s.p. and later to the CA/B Forum SCWG

Comment 16

[Bruce Morton]

Thanks for your input, all actions have been completed. We will continue to monitor this bug.

Please move next update to June 17, 2024, which is after we provide the June 7 report to Mozilla.

Comment 17

[Bruce Morton]

All actions are completed. We are requesting closure of this incident.

Comment 18

[Zacharias]

I’d like to note that this bug was supposed to be updated 2024-06-17, not today which is the 21st

Comment 19

[Ben Wilson]

I'll close this on or about next Wed. 26-June-2024, unless there are questions or issues that would block closing it.