1536287 - Entrust: AffirmTrust Issuing CA Impacted by EJBCA Serial Number Issue

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Dathan Demone]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0

Steps to reproduce:

Entrust Datacard's AffirmTrust brand has been impacted by the EJBCA Serial issue where expected 64 bit serial numbers are actually only 63 bits long.

Comment 1

[Dathan Demone]

1. How your CA first became aware of the problem

After the following serial number issue was reported against EJBCA
https://groups.google.com/forum/#!msg/mozilla.dev.security.policy/13lXh5gomB8/tQtknWTtBgAJ

Entrust Datacard performed an investigation into its use of EJBCA and found that only the offline AffirmTrust root CAs utilized EJBCA with the indicated 64bit configuration issue. This resulted in AffirmTrust issuing CAs’ serial numbers not meeting the BR 7.1, 64-bit requirements.

2. A timeline of the actions your CA took in response

March 13, 2019: News about the EJBCA serial number issue was reported and confirmed
March 14, 2019: Our internal investigation started
March 18, 2019: Our operations team confirmed that Seven issuing CA/intermediate certificates from our AffirmTrust brand were impacted by the reported problem and only included 63-bit serial numbers. They also found one end entity certificate with a 63-bit serial number that is still active.

3. Confirmation that your CA has stopped issuing TLS/SSL certificates with the problem

We will increase our serial number byte length to 128 bits by March 22nd using a re-sign operation to ensure that our intermediate certificates comply with the minimum 64-bit serial number length.

4. A summary of the problematic certificates

Seven issuing CA/Intermediate certificates were impacted, as EJBCA was used to cross-certify these certificates with an AffirmTrust root CA.

We also discovered that there is one active end entity certificate issued by our AffirmTrust CAs. Prior to our acquisition of AffirmTrust, EJBCA was used for both the roots and the issuing CAs. The AffirmTrust issuing CA software was changed from EJBCA to the Entrust Authority PKI on 14 January 2017 and no end-entity certificates issued after this date are impacted by the EJBCA serial number bug.

5. The complete certificate data for the problematic certificates

Production Intermediate Certificates:

https://crt.sh/?id=131879270
https://crt.sh/?id=60529989
https://crt.sh/?id=59260195
https://crt.sh/?id=170271912
https://crt.sh/?id=170271911
https://crt.sh/?id=170271910

Non-production Intermediate Certificates (no active end-entity certificates) :

https://crt.sh/?id=98714371

End Entity Certificate that was impacted:
https://crt.sh/?id=306928500

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

The EJBCA issue that impacted certificate serial numbers was first reported last week. Our internal certificate linting tools failed to identify the issue since only end entity certificates are checked and none of the end entity certificates have the issue.

7. List of steps your CA is taking to resolve the situation

Entrust Datacard will re-sign the issuing CA keys and increase the serial number size to 128 bits by March 22.

We have decided that we will not revoke the issuing CA/intermediate certificates impacted by the EJBCA 63 bit serial number issue at this time. The strain that this will put on our customers to replace their intermediate certificates drastically outweighs the potential risk we see in allowing these intermediate certificates to remain valid. After an in-depth assessment, our security team believes that there are no practical security issues or exploits in this scenario, keeping in mind that all of our issuing CAs are signed with SHA-2, that they are isolated, and that our roots are kept in an offline state.

The CA/Browser Forum Baseline Requirements Version 1.6.4 state:

Effective September 30, 2016, CAs SHALL generate non-sequential Certificate serial numbers greater than zero (0) containing at least 64 bits of output from a CSPRNG.

This language was introduced in Ballot 164, replacing the original requirement:

CAs SHOULD generate non‐sequential Certificate serial numbers that exhibit at least 20 bits of entropy.

The statement of intent in the ballot describes resistance to attacks on hash functions, especially to hash function collision attacks, as the rationale for this change:

As demonstrated in https://events.ccc.de/congress/2008/Fahrplan/attachments/1251_md5-collisions-1.0.pdf, hash collisions can allow an attacker to forge a signature on the certificate of their choosing. The birthday paradox means that, in the absence of random bits, the security level of a hash function is half what it should be. Adding random bits to issued certificates mitigates collision attacks and means that an attacker must be capable of a much harder preimage attack. For a long time the Baseline Requirements have encouraged adding random bits to the serial number of a certificate, and it is now common practice. This ballot makes that best practice required, which will make the Web PKI much more robust against all future weaknesses in hash functions. Additionally, it replaces “entropy” with “CSPRNG” to make the requirement clearer and easier to audit, and clarifies that the serial number must be positive.

As indicated above, the AffirmTrust offline root certification authorities have issued currently valid cross-certificates with serial numbers containing less than 64 bits coming from a CSPRNG. These root certification authorities do not issue end-user certificates but do issue cross-certificates for the AffirmTrust issuing CAs. The issuance of the implicated cross-certificates was not in practice vulnerable to hash collision attacks for the following reasons:
• The certificate signature algorithm uses the SHA-256 hash algorithm, which currently has 128-bit security strength against collision attacks.
• The input to the cross-certification process is generated within an Entrust Datacard isolated network zone and is thus under Entrust Datacard's control.
• The affected root certification authorities are offline roots and are housed in an isolated network zone. As such, they are used only to generate a small number of certificates.
• The affected root certification authorities are shut down except during issuance of cross-certificates and CRLs.
Thus, although including more CSPRNG output in the certificate serial numbers would be preferable, there was no practical attack known at the time the certificates were issued, and revoking them is therefore unnecessary from a security perspective. In addition, if the cross-certificates were revoked, all customers with certificates issued by the issuing CAs would need to deploy new cross-certificates, posing an unnecessary burden on those customers.

Entrust Datacard is working to revoke the single end entity certificate that was impacted before March 22.

Entrust Datacard will also revoke the non-production intermediate certificate with no active end-entity certificates (https://crt.sh/?id=98714371) before March 22.

Comment 2

[Jonathan Rudenberg]

March 13, 2019: News about the EJBCA serial number issue was reported and confirmed

Can you explain what you mean by "reported"? The serial number entropy issue was discussed in detail on MDSP at the end of February, and the issue was then linked to EJBCA by Google in their email with the subject "Google Trust Services and EJBCA serial number behavior" on March 1st.

Comment 3

[Jonathan Rudenberg]

I don't see this "Trend Micro S2 CA" intermediate listed in this report, what was your analysis of its 63-bit serial number and the certificates issued from it? https://crt.sh/?id=5213576

Comment 4

[Ryan Sleevi]

To echo Comment #2:

This matter received its own dedicated thread in [1], on February 28, 2019. Mozilla has required, since April 2017, that CAs monitor mozilla.dev.security.policy, particularly for "certificates that are found to be non-compliant with the CA/Browser Forum's Baseline Requirements or other program requirements."

As part of this incident report, please explain why this monitoring was not performed and what steps are being taken to ensure compliance going forward. Given the significant delays in identifying and acknowledging this issue, this represents a matter of concern.

[1] https://groups.google.com/d/msg/mozilla.dev.security.policy/nlN_QrDwgaw/cg_v-VY0AQAJ
[2] https://wiki.mozilla.org/CA/Communications#April_2017

Comment 5

[Dathan Demone]

In answer to comments 2 and 4:

We do monitor MDSP and we did see the original posts. However, since we do not use EJBCA for any of our issuing CAs, it wasn’t immediately obvious that we may have had a problem. It wasn't until we saw the post mentioned above that we realized that there was a detailed configuration in EJBCA that resulted in 63-bit serial numbers that we needed to check in the offline AffirmTrust roots. In retrospect, perhaps we should have made the connection sooner and we will endeavor to be more vigilant in the future.

In answer to comment 3:

The Intermediate CA “Trend Micro S2 CA” – this CA was created before the cross-certificate issued before the 64-bit serial number requirement was introduced in the BRs, as shown in the crt.sh link: Not Before: Mar 1 23:13:48 2014 GMT.

When looking at certificates issued from that CA, the issuing CA was taken out of production on January 14, 2016, and there is only one end entity certificate that is currently active and affected. It was already included in the incident report - https://crt.sh/?id=306928500 Although the certificate expires on 1 April 2019 it will be revoked on March 22nd.

Comment 6

[Dathan Demone]

The 6 production intermediate certificates mentioned in the original post were re-signed yesterday, March 21st, with serial numbers containing 127 bits from a CSPRNG. The updated intermediate certificates will be distributed with all AffirmTrust issued certificates starting Tuesday, March 26th.

The 1 non-production intermediate certificate mentioned in the original post was revoked yesterday, March 21st. This is reflected here - https://crt.sh/?id=98714371

We also decided to revoke three intermediate certificates from the original post because they are not currently used in production. These certificates were also revoked on March 21st.
https://crt.sh/?id=170271912
https://crt.sh/?id=170271911
https://crt.sh/?id=170271910

The 1 end entity certificate that was mentioned in the original post was revoked today, March 22nd. This is reflected here - https://crt.sh/?id=306928500

Comment 7

[Dathan Demone]

This is a confirmation to state that we have put the re-signed intermediate certificates into production distribution.

We have completed all planned activities associated with this issue.

As mentioned in comment 1, we are not planning to revoke the affected intermediate certificates and we have instead re-signed the certificates to have 127-bit serial numbers.

In agreement with Wayne Thayer, we have determined that an MDSP posting is not required.

Comment 8

[Wayne Thayer]

(In reply to Dathan Demone from comment #7)

In agreement with Wayne Thayer, we have determined that an MDSP posting is not required.

Correct, no posting of an incident to the mozilla.dev.security.policy list is required by policy, although I consider it a good, if not "best practice" to do so. I see nothing special in this incident to cause me to ask the CA to post to the list, or to post myself.

It appears that all questions and remediation actions have been addressed.