1597950 - Sectigo: CCADB failed ALV - Ensured Root CA

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Robin Alden]

Kathleen's email alerted us to the recent 'Failed ALV results' report in CCADB.
This indicated that we had a total of 9 Intermediate CA Certificates with Failed ALV Results.
This bug addresses 2 of those 9:

Ensured Document Signing CA
https://crt.sh/?id=48140521

Ensured Root CA
https://crt.sh/?id=12721494 (now revoked)

This older version of the Ensured Root CA was re-issued (as https://crt.sh/?id=49537563) later in 2016 and this original version was not used after 2016. It has recently been revoked, which has removed both of the above listed CAs from the Failed ALV report.

I will follow up in this ticket with an incident report in Mozilla's preferred format.

Comment 1

[Robin Alden]

We apologize for the delay in our response.

1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.

Kathleen sent an email to all CAs on the mozilla-dev-security-policy list at 08/OCT/2019 20:50 BST, entitled
"Audit Letter Validation (ALV) on intermediate certs in CCADB"
https://www.mail-archive.com/[e-mail]/msg12730.html

The email pointed out a new summary item in the CCADB home page for CAs that listed (for Sectigo):
"Intermediate Certs with Failed ALV Results: 9"

2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.

18 November 2014 A cross-signed CA certificates was issued for "USERTrust RSA Certification Authority"->"Ensured Root CA"
https://crt.sh/?id=12721494

18 November 2014 A subordinate CA certificates was issued for "Ensured Root CA"->"Ensured Document Signing CA"
https://crt.sh/?id=48140521

New CA keys were selected so that the immediately following "Ensured Root CA" and "Ensured Document Signing CA" are not the same CA as the immediately preceding ones.

26 October 2016 A subordinate CA certificates was issued for "Ensured Root CA"->"Ensured Document Signing CA"
https://crt.sh/?id=49618903

04 November 2016 A cross-signed CA certificates was issued for "USERTrust RSA Certification Authority"->"Ensured Root CA"
https://crt.sh/?id=49537563

On 9th October 2019 we started evaluating options to fix the failed ALV results.

On 6th November 2019 we revoked https://crt.sh/?id=12721494, which also had the effect of removing trust from https://crt.sh/?id=48140521.
https://crt.sh/?id=12721494 has also been added to OneCRL.

3. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.

NA

4. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.

There are two CA certificates.
These CA certificates were issued on 17-Nov-2014 and 18-Nov-2014.

5. The complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem.

https://crt.sh/?id=12721494 "USERTrust RSA Certification Authority"->"Ensured Root CA"
https://crt.sh/?id=48140521 "Ensured Root CA"->"Ensured Document Signing CA"

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

The cause of the issue here is that the above referenced CA certificates were issued and used briefly but then re-issued in 2016 with new keys.
The original CA certificates were included in an initial audit reports, but subsequent audit reports included the replacement CAs and not the originals.

7. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.

We will continue to ensure that all issued CA certificates are known to CCADB, as required by Mozilla policy.
We will ensure that we include all certificates over a CA's public key, where we control the corresponding private key, in our own audit regardless of which other audit reports include it.

Comment 2

[Ben Wilson]

Why hasn't the Ensured Document Signing CA (https://crt.sh/?id=48140521) issued on 18-Nov-2014 been revoked and/or added to OneCRL as well?

Comment 3

[Rob Stradling]

Ben, please help me to understand why you're asking this question. I don't see how https://crt.sh/?id=48140521 is of interest to Mozilla at this point.

AFAIK OneCRL is still only used by Firefox, and Firefox won't accept any certificates that chain up to https://crt.sh/?id=48140521 due to the EKU OIDs in that intermediate certificate. So what would be the purpose of adding it to OneCRL?

Also, the only trust path (see https://crt.sh/?graph=48140521&opt=nometadata) for that certificate goes through https://crt.sh/?id=12721494, which has already been revoked and added to OneCRL. Isn't that sufficient? On what grounds would Mozilla desire or require https://crt.sh/?id=48140521 to be revoked?

Comment 4

[Ben Wilson]

I was at first confused and then curious, because it was an anomaly and it wasn't that easy to follow the sequence of events and how there were two CA certificates with the same name but different keys, but I can move on. Thanks.

Comment 5

[Rob Stradling]

Ben, can this bug be closed now?

Comment 6

[Ben Wilson]

Hi Kathleen, the CCADB still shows 2 ALV failures for Sectigo Intermediate CAs: COMODO ECC Certification Authority and COMODO RSA Certification Authority. How should this be handled?

Comment 7

[Rob Stradling]

(In reply to Ben Wilson from comment #6)

Hi Kathleen, the CCADB still shows 2 ALV failures for Sectigo Intermediate CAs: COMODO ECC Certification Authority and COMODO RSA Certification Authority. How should this be handled?

Hi Ben. Both of those ALV failures are false positives. https://crt.sh/?id=2545965608 and https://crt.sh/?id=2545966120 were both issued on 6th March 2020, and I submitted them to the CCADB and several CT logs the same day.

The notBefore dates were deliberately backdated (to 2004-01-01) to influence certificate path building behaviour in Microsoft CryptoAPI, specifically to ensure that an EV-capable chain is selected in Chrome on Windows. The "COMODO ECC Certification Authority" and "COMODO RSA Certification Authority" roots are in Chrome's EV root list, but the "AAA Certificate Services" root is not; and it is my understanding that Chrome on Windows still uses CryptoAPI for path building.

When I created the CCADB records for these 2 certificates, I followed Kathleen's guidance (see https://www.mail-archive.com/[e-mail]/msg12137.html) and wrote the following in the "Public Comments" section:
"Issued after the Audit Period End Date (March 31st 2019). Will be included in the next applicable Audit Statement(s)."

So once Sectigo's current CCADB Audit Case is processed (see bug #1648593 for coverage of its delay), these false positives should disappear.