1726333 - Network Solutions: All test CA test website certificates are expired

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Matthias]

All certificates on the Test Websites in CCADB (as seen in [0]) for the Network Solutions CA [1] are expired, even those that should be Valid and Revoked. This means that the recorded test websites do not comply with BR s2.2.

I communicated this issue on 13 Aug 2021, 21:52 CEST, and received a receipt confirmation at 14 Aug, 01:25 CEST from the Sectigo SSL Abuse and Malware Team (some 4 hours later), stating "Thank you for bringing this to our attention. We will begin an investigation immediately and inform you of any updates or actions."

I have since not been informed of any updates or actions.

The test websites registered in CCADB do not seem to have been updated since, nor have they received new certificates: the 'valid' website still hosts https://crt.sh/?id=1620527116, and the 'revoked' website still hosts https://crt.sh/?id=1620492216 as of the moment of writing. For Firefox, the revoked certificate does (due to its expiry) not show the 'this certificate is revoked' status, and therefore is not useful as a test for 'revoked' certificates.

Particularly noteworthy is that these certificates expired on the 28th of june, meaning that these test websites have been incorrectly configured for over 6 weeks without detection from the CA.

[0] https://ccadb-public.secure.force.com/mozilla/IncludedCACertificateReport
[1] https://crt.sh/?caid=157

Comment 1

[Matthias]

I have since not been informed of any updates or actions.

I just received an update from Sectigo: "Test certificates were issued and have been installed by Network Solutions.", which now indeed seems to be the case.

Comment 2

[Charles Wang]

In your next update i hope you can explain more about the questions below:
Could you explain more about what actions was taken to prevent similar case being repeated again?
As a certification authority why you failed to monitor the expiration status of all your certificates deployed?

Comment 3

[Matthias]

(In reply to Charles Wang from comment #2)

In your next update i hope you can explain more about the questions below:

For your information, I am not affiliated with Network Solutions (other than being a Relying Party). I filed this issue to ensure that the community would be notified that this issue existed at that point in time.

My subsequent updates were provided to update the community to the best of my knowledge on the status of the issue, because Network Solutions itself hadn't provided updates on this forum.

Could you explain more about what actions was taken to prevent similar case being repeated again?
As a certification authority why you failed to monitor the expiration status of all your certificates deployed?

I believe that answering those questions follow naturally from the requirement to provide an incident report.

Forwarding needinfo to Keith McKenney to provide an incident report and answer the questions asked in comment 2.

Comment 4

[Keith McKenney]

Network Solutions and Sectigo have decided to transition the Network Solutions and Web.com branded digital certificate businesses entirely to a managed CA model, managed by Sectigo. We intend to announce a target transition date and other relevant details by the end of this month.

We are working on a full incident report for this bug.

Comment 5

[Keith McKenney]

We have targeted a full transition to the managed service on or before November 8, 2021.

Comment 6

[Matthias]

Keith, I think Network Solutions still needs to provide an incident report: Although the CA keys might be transfered soon, this is still an incident with Network Solutions which the community might learn from through an incident report.

Comment 7

[Keith McKenney]

1. How your CA first became aware of the problem

On June 24 one of our employees for the public CA business received an email from a Sectigo contact stating that our test website certificates were soon to expire. Although a few emails went back and forth with Sectigo, this employee didn’t socialize that we had this need to others on the Network Solutions team and soon left the company.
On August 10 additional staff inside Network Solutions received contact from Sectigo informing us that these certificates were expired and action was required.

2. Timeline

March 15, 2021
The previous program owner leaves Network Solutions, creating a process and accountability gap for many public CA operations. The scope and nature of his gap remains unknown for some months.

June 24
Sectigo informs its technical contact by email that these certificates are soon to expire. For reasons unknown, although the contact acknowledges the communication, no action is taken. This individual leaves the company shortly thereafter.

August 10
Sectigo contacts new designated Network Solutions team members to inform them that these certificates are expired and remain unaddressed.

August 10 to August 18
Current Network Solutions team members investigate what is required and acquire new test website certificates.

August 13
An outside third party emails Network Solutions notifying us that there is an issue with our test website certificates.

August 18
This bug is opened.

August 19
New test website certificates are deployed.

3. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem.

This is not a case of certificate misissuance. Test website certificates are now active and deployed.

4 & 5. A summary of the problematic certificates and affected certificates

This is not a case of certificate misissuance.

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

The Network Solutions employee primarily responsible for test website certificates left the company. That and other personnel changes in our CA practice left a knowledge gap in replacing CA test website certificates. There was no focus on various CA practices for some time. We did not become aware of this knowledge gap until the time had come to renew these certificates. The issuance of test website certificates was also handled partly manually, so once we were aware of the need it still took some time to have the certificates issued and installed.

7. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future.

Network Solutions and Sectigo have decided to transition the Network Solutions and Web.com branded digital certificate businesses entirely to a managed CA model, managed by Sectigo.
Sectigo has a robust process for renewing test website certificates including an alerting mechanism to prevent problems like this one.

Comment 8

[Keith McKenney]

Are there any additional questions on this bug?

Comment 9

[Keith McKenney]

Ben,

We believe we have addressed this bug, including our go-forward strategy for avoiding this problem again. There don’t appear to be any more points of discussion from the community. Should we close this bug?

Comment 10

[Ben Wilson]

I will close this bug on or about Thursday, 4-Nov-2021.