1597947 - Sectigo: CCADB failed ALV - Network Solutions Certificate Authority

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Robin Alden]

Kathleen's email alerted us to the recent 'Failed ALV results' report in CCADB.
This indicated that we had a total of 9 Intermediate CA Certificates with Failed ALV Results.
This bug addresses 6 of those 9:

In all 6 cases, CN=Network Solutions Certificate Authority
The first three were issued from UTN-USERFirst-Hardware:
https://crt.sh/?id=12715817
https://crt.sh/?id=1880
https://crt.sh/?id=597715
The last three were issued from AddTrust External CA Root:
https://crt.sh/?id=1789
https://crt.sh/?id=24679
https://crt.sh/?id=7715768

In each case the CA Certificate is included in Sectigo's WebTrust audits, but CCADB is configured to refer to Web.com's WebTrust audits for these CAs and Web.com's audit reports do not include these (now unused) CAs.
This arose as an inadvertent consequence of the changes made in Bug #1567060.

We will rework the fix for Bug #1567060 to also address the ALV failure.

Comment 1

[Robin Alden]

We apologize for the delay in our response.

1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.

Kathleen sent an email to all CAs on the mozilla-dev-security-policy list at 08/OCT/2019 20:50 BST, entitled
"Audit Letter Validation (ALV) on intermediate certs in CCADB"
https://www.mail-archive.com/[e-mail]/msg12730.html

The email pointed out a new summary item in the CCADB home page for CAs that listed (for Sectigo):
"Intermediate Certs with Failed ALV Results: 9"

2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.

18 July 2019 Bug #1567060 was opened highlighting an apparent inconsistency between the self-signed and the cross-signed versions of this CA https://crt.sh/?spkisha256=32d180ed31c935589ec9dbbb722123b883b5fc2dc10f9fca3a95d77e1bfcb534

13 September 2019 (from Bug #1567060) We updated the CCADB records for the cross-certificates we had issued to Web.com so that the Audit and CP/CPS details match what Web.com have disclosed for their self-signed CA.

08 October 2019 Kathleen's email alerted us to the issue of these issuing CAs now failing ALV

9 October 2019 We identified that Web.com's audit reports didn't include the SHA-256 fingerprints for the six intermediate CA certs that we issued years ago from "UTN-USERFirst-Hardware" and "AddTrust External CA Root".

20 November 2019 We posted to Bug #1597947 that we would reverse the nature of the change we made in Bug #1567060.

We subsequently realized that the reversal of the change we made for Bug #1567060 would not be a complete solution so we did not do it.

3. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.

4. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.

There are 6 certificates listed. These are all CA certificates for the same CA, i.e. they all have the same values for the subjectDN and subjectKeyIdentifier extensions.
The earliest was issued on 01 December 2006
The latest was issued on 07 May 2015

5. The complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem.

https://crt.sh/?id=1789
https://crt.sh/?id=1880
https://crt.sh/?id=24679
https://crt.sh/?id=597715
https://crt.sh/?id=7715768
https://crt.sh/?id=12715817

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

The problem we face with these 6 CA certificates is:
a) These 6 'cross-signed' CA certificates were included in the audit report for Sectigo, but not that of Web.com
b) The self-signed certificate for this CA, https://crt.sh/?id=7677, was included in the audit report for Web.com, but not that of Sectigo.
c) As discussed briefly in Bug #1567060, and in [1], Sectigo runs some "white label" services for Web.com and this includes the operation of this CA.
d) Bug #1567060 concerned an apparent inconsistency in disclosure of which CPS each CA certificate was operated under.
We genuinely tried to include the CA certificates in the CPSs and Audits where we thought they were most apt.

With the benefit of hindsight we should have had included all of the CA certificates in the audit reports for both Network Solutions and Sectigo.

7. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.

These CA certificates were all disclosed.
These CA certificates were all covered by the intersection of the audit reports of Sectigo and NetSol.

Both Network Solutions and Sectigo will include these 6 issuing CAs in their next audit report. These reports are expected to be published by 30th June 2020.

[1] https://www.mail-archive.com/[e-mail]/msg12210.html

Comment 2

[Ben Wilson]

See 2020 Audits uploaded to Bug 1650199. I will need to check to see the status of these using the ALV process in the CCADB.

Comment 3

[Ben Wilson]

When will Sectigo have its WebTrust seal files / CPA Canada URL links?

Comment 4

[Ryan Sleevi]

Nick: Another example of Bug 1563579 where Sectigo is failing to provide timely updates.

Comment 5

[Nick France]

Thanks, Ryan.

The auditors had stated mid-August for the seals and URLs for our sites, but I've just contacted them again (today, July 27th) to request an update and see if the process can be expedited.

Comment 6

[Nick France]

We're still pushing the auditors for updates on the WebTrust seals, we're told they're still waiting on CPA Canada and other internal staff to get them ready. I'll update again next week.

Comment 7

[Nick France]

Still waiting on a further update from our auditors - I have chased them again this today.

Comment 8

[Nick France]

We have no further update at this time. We continue to communicate with our auditors on this issue.

Comment 9

[Nick France]

We have had a good number of further useful exchanges with our auditors.
Our ability to get resolution of this has been impaired by holiday schedules beyond our control.
We now feel we are on the home straight and expect a September release.

Comment 10

[Nick France]

In the last week we have made good progress. We continue to expect a September release.

Comment 11

[Rob Stradling]

We have no further update, but see bug #1648593 for some related Q&A.

Comment 12

[Rob Stradling]

We have no further update, but see bug #1648593 for some related discussion.

Comment 13

[Rob Stradling]

We've received the updated set of audit letters from E&Y. I've attached them to bug #1472993 and updated the links in our Audit Case accordingly.

Comment 14

[Rob Stradling]

We have no further update.

Comment 15

[Rob Stradling]

We have no further update.

Comment 16

[Rob Stradling]

We've received the Seal URLs. I've updated our CCADB Audit Case accordingly.

Comment 17

[Rob Stradling]

Ben, can this bug be closed now?

Comment 18

[Ben Wilson]

Yes - I'll close this on or about 30-October-2020.