1627346 - Entrust: S/MIME Certificate Issued with Incorrect Policy OID

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Bruce Morton]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36

Steps to reproduce:

Entrust issued 6 S/MIME certificates in May 2019 with the incorrect certificate policy OID.

Comment 1

[Bruce Morton]

1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.

On 31 March 2020, Entrust Datacard discovered that six (6) S/MIME certificates which were issued in May 2019 with the incorrect certificate policy OID. The certificates were not revoked.

2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.

13 May 2019, UTC 20:19 to 21:28 - Six (6) S/MIME certificates were issued to the Entrust QA team with the incorrect certificate policy extension OID.
13 May 2019, 21:56 UTC - It was discovered using post issuance linting software that the certificate policy extension OID was incorrect.
14 May 2019, 11:52 UTC - It was confirmed that the certificate policy extension OID had been corrected.
31 March 2020, 18:00 UTC - Deloitte advised as part of their annual compliance audit that 6 S/MIME certificates were issued with the incorrect certificate policy extension OID.
2 April 2020 - All 6 certificates were revoked.

3. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.

The CA has stopped issuing certificates using the incorrect certificate policy OID.

4. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.

Six S/MIME certificates were issued on 13 May 2019 between 20:19 and 21:28 UTC with the certificate policy extension OID value of 2.16.840.1.114028.10.1.4.2, where the OID value should have been 2.16.840.1.114028.10.1.4.1.

5. The complete certificate data for the problematic certificates.

This Subordinate CA https://crt.sh/?id=1412330549 mis-issued S/MIME certificates with the following serial numbers:

00DFCF8BF507A47C76000000005563A164
1581651F87A4DED7000000005563A166
09D1729E35EC26CE000000005563A168
4F0A440A4B4B27FF000000005563A169
5480E73BDACD958D000000005563A16D
0091C6FD2DF87775ED000000005563A16F

Note that the EKUs for these certificates are clientAuth and emailProtection.

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

The incorrect certificate policy extension OID was implemented in the QA test system and was not discovered. The incorrect certificate policy extension OID was then put into production. The post issuance linting software was updated for the new CA and discovered the issue on both the QA and Production systems on the same day. The problem was resolved and fixed immediately. The problem was fixed before the updated certificate service was released to customers. The problem was not treated the same as the normal certificate problem report (CPR) process and as such we failed to revoke the mis-issued certificates.

7. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.

The certificates were issued for QA testing and were never used for signing email. All certificates have been revoked, see http://crl.entrust.net/class1-sha256.crl:

00DFCF8BF507A47C76000000005563A164, revoked 2 April 2020, 19:10:41 UTC
1581651F87A4DED7000000005563A166, revoked 2 April 2020, 19:09:26 UTC
09D1729E35EC26CE000000005563A168, revoked 2 April 2020, 19:05:36 UTC
4F0A440A4B4B27FF000000005563A169, revoked 2 April 2020, 19:08:05 UTC
5480E73BDACD958D000000005563A16D, revoked, 1 April 2020, 17:43:21 UTC
0091C6FD2DF87775ED000000005563A16F, revoked, 1 April 2020, 17:45:07 UTC

By 31 May 2020, we will update our internal CPR process similar to how it is described in our CPS section 4.9.3. The plan will be that the internal CPR process will mimic the external process and ensure that all mis-issued certificates are discovered, logged, investigated and revoked if required.

Comment 2

[Wayne Thayer]

Bruce: thank you for this incident report. Two questions:

1. Has Entrust confirmed that no other misissuances of this nature have gone unreported?
2. Your answer to question 7 describes a plan to ensure that revocation occurs, but it doesn't explain what Entrust will do to prevent future misissuances in the first place. Please confirm that Entrust is treating this as "mistakes happen", or describe your analysis of what happened to allow the incorrect OID to be put in place and how that type of issue will be prevented in the future.

Comment 3

[Bruce Morton]

(In reply to Wayne Thayer from comment #2)

Bruce: thank you for this incident report. Two questions:

1. Has Entrust confirmed that no other misissuances of this nature have gone unreported?

No other misissuances of this nature have gone unreported. Also note that we have custom post-issuance linting running on all certificates generated. This linting checks the certificate policy OID values are correct for all certificates.

2. Your answer to question 7 describes a plan to ensure that revocation occurs, but it doesn't explain what Entrust will do to prevent future misissuances in the first place. Please confirm that Entrust is treating this as "mistakes happen", or describe your analysis of what happened to allow the incorrect OID to be put in place and how that type of issue will be prevented in the future.

The mistake happened as the QA CA was templated from another CA with the "incorrect" OID value. This value did not cause any technical issues with QA testing and so was missed. The QA CA was used to template the production CA. In the future, this type of error will be detected in one of two ways: 1) a QA test certificate will be issued and will be inspected to ensure it meets the certificate profile, and 2) the post issuance linting software will be updated and run against the new QA certificates.

Comment 4

[Ryan Sleevi]

(In reply to Bruce Morton from comment #3)

(In reply to Wayne Thayer from comment #2)

1. Has Entrust confirmed that no other misissuances of this nature have gone unreported?

No other misissuances of this nature have gone unreported. Also note that we have custom post-issuance linting running on all certificates generated. This linting checks the certificate policy OID values are correct for all certificates.

I originally planned to echo Wayne's question when reviewing this, but I think this follow-up still gives me concern, because I think "this nature" might mean different things for us.

I'll try to phrase it differently, because I also think Wayne's follow-up was relevant here:

1. Has Entrust confirmed that there were no other system configuration errors that may have been placed into production, but detected promptly, that have not been reported.

In short, I see the "mismatched OID" as a symptom, and the concern here I'm trying to get at is that there were one or more misissued certificates that were not disclosed here as CA incidents. The proposed remediation in Comment #1 doesn't seem to capture that these should have been disclosed as a CA Incident when they were detected, separate and independent from their revocation and remediation. Your description in Comment #3 focuses on detection during QA, which is good, but I think it's overlooking the lack of report as another problem to be remediated. Comment #1 notably lacks "disclosed to Mozilla" as part of the external CPR steps, so that's somewhat telling in its omission.

Comment 5

[Bruce Morton]

(In reply to Ryan Sleevi from comment #4)

1. Has Entrust confirmed that there were no other system configuration errors that may have been placed into production, but detected promptly, that have not been reported.

There have been no other non-disclosed system configuration errors that have been placed into production. This has been confirmed through our linting process, plus all certificates in the audit period (1 March 2019 to 29 February 2020) were provided to Deloitte and were also linted.

In short, I see the "mismatched OID" as a symptom, and the concern here I'm trying to get at is that there were one or more misissued certificates that were not disclosed here as CA incidents. The proposed remediation in Comment #1 doesn't seem to capture that these should have been disclosed as a CA Incident when they were detected, separate and independent from their revocation and remediation. Your description in Comment #3 focuses on detection during QA, which is good, but I think it's overlooking the lack of report as another problem to be remediated. Comment #1 notably lacks "disclosed to Mozilla" as part of the external CPR steps, so that's somewhat telling in its omission.

Our CPR process, per CPS section 4.9.3, includes publicly posting an Incident Report. As such it will be disclosed to Mozilla as we will use the Bugzilla method for public disclosure.

Comment 6

[Wayne Thayer]

Bruce: please update this bug when the internal CPR process update is complete. Setting next update to 31-May per comment #1.

Comment 7

[Ben Wilson]

See comment #6

Comment 8

[Bruce Morton]

Internal CPR process has been updated. The process is the same as per CPS section 4.9.3, its just that the source of the information will come from an internal source.

Comment 9

[Ben Wilson]

(In reply to Bruce Morton from comment #8)

Internal CPR process has been updated. The process is the same as per CPS section 4.9.3, its just that the source of the information will come from an internal source.

So there is internal documentation that has been updated? Also, I looked at CPS section 4.9.3, and I'm not sure which information source you are referring to. (Also, the romanette numbering of the CPR process starts at (v) - so I think you have an issue with Continue Numbering or Restart Numbering, if you're using Word to edit your CPS.)

Comment 10

[Bruce Morton]

(In reply to Ben Wilson from comment #9)

So there is internal documentation that has been updated?
Yes

Also, I looked at CPS section 4.9.3, and I'm not sure which information source you are referring to.
I'm referring to the CPR information with text starting with "Subscribers, Relying Parties, ASVs, Anti-Malware Organizations and other third parties may submit a CPR."

(Also, the romanette numbering of the CPR process starts at (v) - so I think you have an issue with Continue Numbering or Restart Numbering, if you're using Word to edit your CPS.)
I typically do not restart the numbering in the same section, the result is that I won't have a section with a number such as (i) twice. This makes it easier to provide a reference.