1436173 - DigiCert: SCEE / Justica: Non-BR-Compliant Certificate Issuance

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Wayne Thayer]

In bug 1432608, Jonathan Rudenberg pointed to a large number of technical compliance issues with certificates issued by the ECCE 001 subordinate CA operated by SCEE / EC Raiz Estado and cross-signed by the DigiCert Baltimore CyberTrust root. A list of issues from 2017-to-present can be found at https://crt.sh/?caid=5132&opt=cablint,zlint,x509lint&minNotBefore=2017-01-01

Jonathan also reported a number of additional technical compliance issues with the Justica subordinate that also chains through a cross-signed SCEE root to the Baltimore CyberTrust root. A list of issues from 2017-to-present can be found at https://crt.sh/?caid=606&opt=cablint,zlint,x509lint&minNotBefore=2000-01-01

For the problem listed above, please provide an incident report as described here:

https://wiki.mozilla.org/CA/Responding_To_A_Misissuance#Incident_Report

Comment 1

[Firefox Bug Husbandry Bot]

Changing QA contact per https://bugzilla.mozilla.org/show_bug.cgi?id=1438254

Comment 2

[Ben Wilson]

Sorry for the delay in getting this Incident Report posted.  As you'll see below, we have been working to resolve this issue and we were hoping to have a full and final disposition to report.

How DigiCert first became aware of the problem:

DigiCert first became aware of the problem when Jonathan Rudenberg posted to Bug 1432608 (January 26,  2018) https://bugzilla.mozilla.org/show_bug.cgi?id=1432608#c3 regarding DigiCert’s request to place the cross certificate issued by the Baltimore Cybertrust Root issued to the EC Raiz Estado ("the Portuguese Root") included on One CRL.  DigiCert received an email on February 6, 2018, when this bug was opened on Bugzilla.

The Bug 1432608 post from J. Rudenberg cited “the sheer amount of misissuance” under the Portuguese Root as a good reason to add   it to OneCRL. However, a review of all certificates issued since 2000 in this Incident Report would be overly burdensome and unproductive. In this bug, Wayne Thayer cites to issues from 2017-to-present, which we will attempt to address in this Incident Report.  W. Thayer also mentioned the issuing CA “Justiça” (Department of Justice under the Portuguese Root), https://crt.sh/?caid=606, which had already been revoked at the time this bug was filed.  This Incident Report will also address the Justiça CA to a limited extent.

Timeline of actions (unless otherwise indicated, all date references are to 2018):

Note:  At all times relevant over at least the past 7 months through the present, DigiCert has held 2-3 internal meetings per week to discuss remediation of its external CAs that are not technically constrained.  During such meetings, DigiCert compliance staff, executive management, and customer relations personnel discuss progress towards bringing such externally operated CAs into full compliance with Mozilla Policy and the Baseline Requirements.

January 23 – Concerned generally about Baseline-Requirements-Mozilla-Policy compliance of the EC Raiz Estado and its subordinate issuing CAs, DigiCert filed a bug to add four cross certificates issued by the Baltimore Cybertrust Root to the Portuguese Root to prevent issuing CAs from being used for SSL/TLS server authentication. (Prior to 26-Jan, DigiCert had not specifically run a  crt.sh query similar to the one by J. Rudenberg.)

January 25 – The relevant departments of the Portuguese government held a meeting to discuss DigiCert’s request to place the Portuguese Root on OneCRL.

January 26 – The Portuguese government responded to our request to place the Portuguese Root on OneCRL, stating there would be “severe impacts … on the image of Portugal, its Government and its public services (all public websites of Portuguese public services - like IRS, Health Services and Social Security - work with SSL certificates issued under EC Raiz).”  Portugal proposed revocation of the CA certificate issued to Justiça and requested further discussion about options for a transition period so that the impact for Portugal and its public services could be minimized.  DigiCert filed https://bugzilla.mozilla.org/show_bug.cgi?id=1432608#c2 to Bug 1432608 requesting that the request to add the Portuguese Root to OneCRL be held in abeyance while certain issues were worked out.

January 29 – Portugal indicates that all SSL/TLS server certificates issued under Justiça would be revoked by 30-Jan and that they would revoke the CA certificate issued to Justiça on 2-Feb.

February 2 – The Portuguese Root CA revokes the CA certificate issued to Justiça.

February 6 – This bug is filed and DigiCert forwards the resulting Bugzilla notice to the Portuguese Government.  

February 8 - Portuguese Government responds that it is analyzing the issues and deciding upon a course of action.

March 1 – DigiCert asks for an update on the status of the Portuguese Government’s response to this bug.  They respond that they believe that their SSL/TLS Server certificate profile has been appropriately updated and that all certificates are being properly issued.  For instance, errors with control characters in CertificatePolicies and UserNotice strings had been corrected, among others.  

March 23 – Portugal indicates that it continues to work through the issues identified. 
 
April 6 – DigiCert indicates that it intends to revoke the cross certificates issued to the Portuguese Root on 10-April.  Portugal responds that it is concerned about the effects that revocation will have on non-SSL/TLS certificates.  Specifically, a few sub CAs (including https://crt.sh/?id=341674988) only issue certificates for the Portuguese national eID cards, which Portuguese Citizens use to authenticate to public services websites. DigiCert responds that it had asked that the listing of the Portuguese Root on OneCRL had been postponed because the ECCE 001 sub CA had issued approximately 400 SSL/TLS server certificates, which were still valid, and so problems with that CA needed to be resolved.

April 8 – DigiCert proposes replacing the cross certificate with one that only allows client authentication.

April 9 – DigiCert holds a conference call with the Portuguese government.  Because of the number of valid SSL/TLS certificates still being used on G2C websites (e.g.  Portuguese Tax Authority), the Portuguese government asks DigiCert about its threatened revocation of the cross certificates on 10-April.  The Portuguese government agrees to begin obtaining SSL/TLS server certificates through DigiCert’s commercial service offering, and DigiCert agrees to postpone revocation.  The parties to explore options for the eID certificates. An audit report for the ECCE 001 sub CA is delivered and posted to Bugzilla (https://bug1433326.bmoattachments.org/attachment.cgi?id=8966451).  

April 10 through May 10 – Portuguese government makes efforts to replace issuance of all SSL/TLS server certificates previously issued by ECCE 001 with issuance by a new CA and to revoke all active SSL/TLS certificates.

June 13 - Portugal advises DigiCert that it is working on an agreement with a third party to provide it with SSL/TLS server certificates.  

June 21 – Portugal advises DigiCert that it is experiencing difficulties meeting revocation deadlines set by DigiCert and needs an extension of 2-3 weeks’ time to be prepared for revocation of the cross certificates issued to the Portuguese Root.  

July 2 – DigiCert sets revocation date for the cross certificates for on or about 14-August.

Summary of problematic certificates, complete certificate data, and whether Portugal has stopped issuing certificates with the specified problems:

Full information of the historic issues for the ECCE 001 CA, all affected certificates, and  complete certificate data can be obtained by running the following query:  https://crt.sh/?caid=5132&opt=cablint,zlint,x509lint&minNotBefore=2017-01-01 
That query reveals that approximately 440 certificates in the set of certificates issued since January 1, 2017, had errors of “Control character found in String in CertificatePolicies”, “Control character found in String in UserNotice,” and “ERROR: Constraint failure in UserNotice: ASN.1 constraint check failed: VisibleString: constraint failed (DisplayText.c:100).”  As mentioned above, as of 1-March the Portuguese government had resolved these three issues.

A query of the ECCE 001 issuing CA for certificates issued on or after 1-March 

(https://crt.sh/?caid=5132&opt=cablint,zlint,x509lint&minNotBefore=2018-03-01) indicates that only one certificate, issued on 1-March, had the issues identified above.  This was the certificate with cert.sh ID# 349763357 – ERROR: Control character found in String in CertificatePolicies; ERROR: Constraint failure in UserNotice: ASN.1 constraint check failed: VisibleString: constraint failed (DisplayText.c:100); ERROR: Control character found in String in UserNotice; WARNING: explicitText is not using a UTF8String; AND WARNING: Compliant certificates should use the utf8string encoding for explicitText.

Since 1-March, another six (6) certificates were issued with the following errors:

cert.sh ID# 393673775 -- ERROR: Incorrectly encoded TeletexString in X520OrganizationalUnitName; ERROR: Incorrectly encoded TeletexString in Certificate; ERROR: Fails decoding the characterset; WARNING: organizationalUnitName is using deprecated TeletexString; WARNING: commonName is using deprecated TeletexString; AND WARNING: The name entry contains something that is not a PrintableString or UTF8String;

cert.sh ID# 381632233 -- ERROR: BR certificates must not contain directoryName type alternative name; ERROR: Invalid type in SAN entry; AND ERROR: The Subject Alternate Name extension MUST contain only 'dnsName' and 'ipaddress' name types;
cert.sh ID# 431048576 -- ERROR: DNSName is not in preferred syntax; ERROR: Unknown TLD in SAN; ERROR: Characters in labels of DNSNames MUST be alphanumeric, - , _ or *; AND ERROR: DNSNames must have a valid TLD;

cert.sh ID#s 347273839 and 347273836 -- ERROR: Constraint failure in X520OrganizationalUnitName: ASN.1 constraint check failed: UTF8String: constraint failed (X520OrganizationalUnitName.c:174); ERROR: organizationalUnitName too long; AND ERROR: The 'Organizational Unit Name' field of the subject MUST be less than 64 characters; AND

cert.sh ID# 349763357 --  ERROR: Fails decoding the characterset.

Others had warnings:

cert.sh ID# 383391239 --  WARNING: commonName is using deprecated TeletexString AND WARNING: The name entry contains something that is not a PrintableString or UTF8String; 

cert.sh ID# 347317340 –  WARNING: The domain SHOULD NOT have a bare public suffix; 

AND also 27 certificates with "WARNING: Name has deprecated attribute emailAddress", AND 88 certificates with "WARNING: Policy information has qualifier other than CPS URI".


The following search query at Censys.IO “parsed.issuer.common_name: "ecce 001" AND parsed.validity.start: [2018-05-01 TO *]” indicates that the last certificate issued by the ECCE 001 CA was on 18-May.

How and why mistakes were made and how they avoided detection:

DigiCert cannot speculate on the exact reasons that the issuer of the certificates in question did not meet current SSL/TLS server certificate profile requirements or why errors avoided detection. 

Steps DigiCert is taking to resolve the situation:

As indicated above, DigiCert will revoke the Portuguese Root on or about 14-August.  DigiCert continues working with other externally operated CAs to ensure future compliance with applicable industry requirements.  These steps include weekly review meetings, regular correspondence with third party CAs, revocation of non-compliant CAs, and agreement amendments to include newly adopted requirements and other supervisory provisions imposed by DigiCert on such third party CAs.

Comment 3

[Wayne Thayer]

Thanks for the detailed update Ben. I assume you're planning to revoke all 4 EC Raiz Estado cross-certificates on 14-August?

Once that happens, please mark them revoked in CCADB so that we can get them added to OneCRL.

Comment 4

[Steve Medin]

CCADB has been updated with the four revocations completed yesterday. We granted a one week extension to help the government work through replacing certificates for a few critical services that were not ready for 14-August revocation.

Comment 5

[Wayne Thayer]

Confirmed that the four ECRaizEstado cross-certificates signed by the Baltimore CyberTrust root are marked as revoked in CCADB and will be added to OneCRL. I believe this removes all trust in ECRaizEstado from Mozilla products.