1742657 - GoDaddy: Failure to Revoke Subscriber Certificates within 24 hours

Assignee

Description

•

2 years ago

User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36

Brittany Randall

Assignee

Comment 1

•

2 years ago

Preliminary Incident Report

Issue Summary

As of 10AM AZ MST on November 23, 2021, approximately 310K subscriber certificates were not revoked within 5 days of when we identified their private keys were exposed. This is a violation of the Baseline Requirements for the Issuance and Management of Publicly-Trusted Certificates, Section 4.9.1.1 “Reasons for Revoking a Subscriber Certificate” which states that “a CA SHOULD revoke a certificate within 24 hours and MUST revoke a Certificate within 5 days if the CA is made aware of a demonstrated or proven method that exposes the Subscriber’s Private Key to compromise.”

Background

On November 17, 2021, GoDaddy discovered unauthorized third-party access to GoDaddy’s Managed WordPress hosting environment. On November 18, 2021, at 11AM MST, GoDaddy confirmed that subscriber private keys associated with Managed WordPress customer accounts were exposed. (Refer here for additional details https://aboutus.godaddy.net/newsroom/company-news/news-details/2021/GoDaddy-Announces-Security-Incident-Affecting-Managed-WordPress-Service/default.aspx).

GoDaddy’s Managed WordPress hosting environment is a consumer of subscriber certificates from the GoDaddy CA. During the incident remediation process, GoDaddy’s Managed WordPress hosting engineering team worked with the GoDaddy PKI engineering team to systematically reissue impacted subscriber certificates and subsequently revoke certificates once the replacement certificates had been issued and installed. We chose to issue and install new certificates prior to revoking certificates based on the available security information, the security remediation efforts, and the importance of keeping customer sites up and running. The GoDaddy CA is both physically and logically separated from the managed WordPress environment and was not compromised as a result of this security incident.

Revocation Progress

While good faith efforts were made to cycle through all subscriber certificates exposed by the Managed WordPress incident before the deadline of Tuesday, November 23, 2021, at 10:59AM MST, we made the decision to exceed the 5 day revocation requirement for a subset of the impacted certificates.

Some of the factors that informed this approach included the following:

Based on our investigation, subscriber private keys in the Managed WordPress environment are not the target of this incident (Note: investigations are ongoing)
Additional security layers on the Managed WordPress hosting environment help to minimize the exposure risk.
Managed WordPress Hosting customers include small businesses and individuals whose websites may be critical to their businesses, and we wanted to minimize downtime during Thanksgiving week.
Systematic issuance and revocation are continuing through to completion.

Plan of Record and Next Steps

Reissuance and revocations are currently underway. We intend to revoke all impacted certificates within the next 72 hours.

A detailed incident report, which includes specific numbers, additional information, and relevant timeline information will follow in the coming week.

Ryan Sleevi

Comment 2

•

2 years ago

The BRs require revocation within 24 hours if:

The CA obtains evidence that the Subscriber’s Private Key corresponding to the Public Key in the Certificate suffered a Key Compromise;

Will the incident report contain sufficient detail to assure relying parties that this provision does not apply? The text quoted by GoDaddy was intended for situations closer to Heartbleed or remote timing channels, as opposed to the above requirement, which was intended for compromised systems, as this appears to be.

Some of the reasons for delaying revocation sound, on the substance, remarkably similar to the example called out in the expectations, namely:

Responses similar to “we do not deem this non-compliant certificate to be a security risk” are not acceptable.

This isn’t to say that mitigating factors don’t exist, but rather, to ensure there is sufficient detail to objectively evaluate that.

Hopefully this incident report will address in sufficient details why the Managed WordPress solution was not designed with the Baseline Requirements in mind. It would seem that, as a managed hosting provider, it should be trivial to be able to replace these certificates in a timely fashion with zero disruption or outages. This has been demonstrated by other CAs and cloud providers that are customers of CAs, and is well understood as minimal industry expectations, so it will be useful to both understand why GoDaddy’s systems were not designed with this in mind, and to ensure timely remediation of failing to meet those baselines, such as through automation and robust systems management.

Assignee: bwilson → brittany

Status: UNCONFIRMED → ASSIGNED

Type: defect → task

Ever confirmed: true

Comment 3

•

2 years ago

Thanks, Ryan! Appreciate your feedback. We will definitely provide a thorough incident report in the coming week which provides additional details as well as the list of ALL certificates for visibility.

Brittany Randall

Assignee

Comment 4

•

2 years ago

Just wanted to provide an update since we originally estimated completion at 72 hours. The teams are still actively working through the issuance and revocation processes.

To Ryan’s point in Comment #2, as we work through compiling our full incident report and identifying a thoughtful remediation strategy, we will be focused on automation and robust systems management in addition to better upstream guidance for our larger certificate consumers. We are learning from this experience, including how we can enable our consumers to better leverage our existing product features in addition to how we can add processing speed and throughput to prevent future, similar events.

We take the revocations of these certificates very seriously and the teams are working diligently to complete the remaining certificate revocations associated with this incident.

Ryan Dickson

Comment 5

•

2 years ago

[Posting on behalf of Google Chrome]

Hi Brittany,

We appreciate GoDaddy's dedication to providing the community with a detailed and complete response related to this incident; however, we'd benefit from some interim feedback to ensure we're doing our best to manage risk on behalf of our users. Along those lines, can you comment when the next update will be available?

In terms of specific areas of interest:

Complete certificate data for all affected certificates
Describe the role of the provisioning system and its relationship with the CA and/or other PKI components
The maximum level of access or functional privilege the unauthorized third party could have exercised through the attack, especially concerning the issuing CA or other PKI components (e.g., could the adversary use their access to request new certificates, perform certificate management functions, remotely manage network-attached HSMs, etc.?)
The detected access and activities performed by unauthorized third party with any implication to the issuing CA or other PKI components
Whether the compromised account and password also existed on the issuing CA or other PKI component.
Can GoDaddy describe the "additional security layers on the Managed WordPress hosting environment help to minimize the exposure risk" in more detail?
What evidence can GoDaddy offer supporting the conclusion that subscriber private keys were not the target of the breach?
Is there any evidence to refute the assumption that the adversary exfiltrated subscriber private keys?

Thanks,
Ryan

Brittany Randall

Assignee

Comment 6

•

2 years ago

Hi Ryan! The next update should be by the end of this week. We will be posting the full list of revoked certs (i.e. all of the certificates that were impacted which were subsequently revoked). We are also targeting Friday for the formal incident report.

Thanks for sharing the specific areas of interest, we will see where we can provide some additional information relative to those inquiries.

Brittany Randall

Assignee

Comment 8

•

2 years ago

Attached file revoked_certs_serials.txt — Details

Brittany Randall

Assignee

Comment 9

•

2 years ago

Revocation progress update: All certificates (Total: 457,911 certificates) associated with the incident have been revoked. See attached file, “revoked_certs_serials.txt”, for the serial numbers. Full certificate data to follow.

Brittany Randall

Assignee

Comment 10

•

2 years ago

Hi Ryan,

Wanted to provide some answers to your questions from comment 5:

Complete certificate data for all affected certificates

We are expecting to post this by the end of the week.

Describe the role of the provisioning system and its relationship with the CA and/or other PKI components

The provisioning system is part of the Managed WordPress hosting environment and acts as a consumer of certificates from the GoDaddy CA by making certificate requests and receiving issued certificates from the GoDaddy CA for Managed WordPress hosting customers. It is logically and physically separate from the GoDaddy CA.

The maximum level of access or functional privilege the unauthorized third party could have exercised through the attack, especially concerning the issuing CA or other PKI components (e.g., could the adversary use their access to request new certificates, perform certificate management functions, remotely manage network-attached HSMs, etc.?)

The compromised credentials were not ones that would have been used in conjunction with anything PKI related.

The detected access and activities performed by unauthorized third party with any implication to the issuing CA or other PKI components

The CA environment, issuing CAs, and the general PKI environment were not impacted and could not have been impacted by the credentials obtained as mentioned above.

Whether the compromised account and password also existed on the issuing CA or other PKI component.

No. We can confirm that the account and password that were compromised did not exist in the issuing CA or any other PKI components.

Can GoDaddy describe the "additional security layers on the Managed WordPress hosting environment help to minimize the exposure risk" in more detail?

For security reasons we don’t disclose the measures we have taken in detail, however, we can confirm that we have added access hardening to the previously exposed API end point.

What evidence can GoDaddy offer supporting the conclusion that subscriber private keys were not the target of the breach?

To date we’ve seen instances of SEO poisoning (“Blackhat SEO”). We have no other reason to believe that subscriber keys were the target.

Is there any evidence to refute the assumption that the adversary exfiltrated subscriber private keys?

It is possible that the adversary had access to private keys, however, there is no evidence to suggest they were after these specifically. To date, we’ve only seen instances of SEO poisoning.

One thing that may be noteworthy, related to our formal incident report, we will be amending our previously provided issue summary to measure against the 24 hour revocation timeline within the BRs. Namely, subscriber certificates were not revoked within 24 hour of key compromise, which is a violation of the Baseline Requirements for the Issuance and Management of Publicly-Trusted Certificates, Section 4.9.1.1 “Reasons for Revoking a Subscriber Certificate” which states that “the CA SHALL revoke a Certificate within 24 hours if the CA obtains evidence that the Subscriber’s Private Key corresponding to the Public Key in the Certificate suffered a Key Compromise”

We will continue providing updates, working toward the full incident report, and monitoring for questions. Hope this helps in the meantime.

Best,

Brittany

Brittany Randall

Assignee

Comment 11

•

2 years ago

Just wanted to provide an update. Given the scope of this incident, we are still working on the incident report to ensure we are including the right level of detail.

Thanks,
Brittany

Ryan Dickson

Comment 12

•

2 years ago

Hi Brittany,

GoDaddy has now failed its commitment to offer a timely response related to this issue several times (i.e., meeting the 72-hour revocation commitment, posting full certificate data of affected certificates, and sharing substantive updates that certificate consumers can meaningfully use to protect the interests of their user community). The continued delays, compounded with the additional ~150,000 revoked certificates compared to the initial post in Comment #1, are worrying.

When can we expect an update from GoDaddy?

Thanks,
Ryan

Brittany Randall

Assignee

Comment 13

•

2 years ago

Attached file MWPCertificates.zip — Details

Brittany Randall

Assignee

Comment 14

•

2 years ago

Attached are the full list of certificates with the full cert data ("MWPCertificates.zip"). Formal incident report should be posted by end of day.

Thanks,

Brittany

Brittany Randall

Assignee

Updated

•

2 years ago

Summary: GoDaddy: Failure to Revoke Subscriber Certificates within 5 days → GoDaddy: Failure to Revoke Subscriber Certificates within 24 hours

Brittany Randall

Assignee

Comment 15

•

2 years ago

Incident Report

1. How your CA first became aware of the problem (e.g., via a problem report submitted to your Problem Reporting Mechanism, a discussion in the MDSP mailing list, a Bugzilla bug, or internal self-audit), and the time and date.

Problem Summary:

NOTE: This is an amended problem summary. We shifted our problem statement from the preliminary incident report to focus on revocations against the 24-hour timeline.

457,911 subscriber certificates were not revoked within 24 hour of key compromise. This is a violation of the Baseline Requirements for the Issuance and Management of Publicly-Trusted Certificates, Section 4.9.1.1 “Reasons for Revoking a Subscriber Certificate” which states that “the CA SHALL revoke a Certificate within 24 hours if the CA obtains evidence that the Subscriber’s Private Key corresponding to the Public Key in the Certificate suffered a Key Compromise”

Discovery Details:
On November 17, 2021, GoDaddy discovered unauthorized third-party access to GoDaddy’s Managed WordPress hosting environment. On November 18, 2021, at 11:00 MST, GoDaddy confirmed that subscriber private keys associated with Managed WordPress customer accounts were exposed. (Refer here for additional details https://aboutus.godaddy.net/newsroom/company-news/news-details/2021/GoDaddy-Announces-Security-Incident-Affecting-Managed-WordPress-Service/default.aspx).

2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.

Of note, additional meetings and touchpoints were held as part of the overall security incident management process that are outside the scope of SSL, such as the password rotations, etc. Meetings included incident milestone tracking, ad hoc engineering calls, and briefings to senior leadership. In the case of this timeline, we’ve focused on the SSL milestones specifically with other events that are relevant to understanding the incident.

DD/MM/YYYY HH:MM (Times are all MST)

11/17/2021 12:16 GoDaddy discovers unauthorized third-party access to GoDaddy’s Managed WordPress hosting environment.
11/18/2021 11:00 GoDaddy’s InfoSec confirms that subscriber private keys associated with Managed WordPress customer accounts were exposed.
11/18/2021 12:13 GoDaddy’s InfoSec identifies incident management workstreams.
11/18/2021 20:45 External IT Forensic firm engaged
11/19/2021 11:00 Meeting: Incident milestone tracking. SSL tracking totals: 21,000 certs rotated. 0 certs revoked.
11/20/2021 11:00 Meeting: Incident milestone tracking. SSL tracking totals: 44,443 certs rotated. 17,300 certs revoked.
11/21/2021 12:00 Meeting: Incident milestone tracking. SSL tracking totals: 85,573 certs rotated. 47,000 certs revoked.
11/21/2021 13:17 GoDaddy decides to replace existing EV certificates with free DV certificates to expedite the reissuance and revocation process.
11/23/2021 09:48 The revocation event queue is backed up with 90K certificates. Team pauses on reissuance to investigate.
11/23/2021 10:38 Investigation completed, and fix implemented. Revocation and reissuance events resume processing. Developers continue to monitor CA throughput and look for ways to increase concurrency.
11/23/2021 10:56 GoDaddy posts preliminary incident report in Bug 1742657.
11/23/2021 13:49 Development team escalates that up to 125k certificates that had been rotated were inadvertently revoked.
11/23/2021 15:45 Development teams pause reissuance and revocation of compromised certificates and focus efforts on reissuing certificates that were inadvertently revoked to minimize customer impact.
11/24/2021 11:00 Meeting: Incident milestone tracking. SSL tracking totals: 92,230 inadvertently revoked certs have been reissued.
11/24/2021 11:23 Developers continue to monitor systems and look for ways to improve CA performance.
11/24/2021 14:08 All certificates that were inadvertently revoked have been reissued.
11/25/2021 16:40 Team finalizes list of 300K to remove certificates that have already been revoked and performs extra checks to confirm revocations should be completed. Begin processing revocations.
11/26/2021 10:03 GoDaddy provides an update to Bug 1742657 that estimated completion time will not be met.
11/29/2021 09:30 Meeting: Incident milestone tracking. SSL tracking totals: Certificate reissuance complete. 94.57% certs revoked.
11/30/2021 17:00 Meeting: Incident milestone tracking. SSL tracking totals: 457,878 revoked
12/1/2021 22:45 Revocations of last impacted certificates completed.
12/2/2021 08:30 Meeting: review the list of serials and confirm details for bug update.
12/02/2021 10:15 GoDaddy comments on Bug 1742657 and provides an aggregate total of certificates revoked (457,911) as well as an attachment of the certificate serial numbers.
12/02/2021 16:00 GoDaddy responds to questions from comment 2 on Bug 1742657.

3. Whether your CA has stopped, or has not yet stopped, certificate issuance or the process giving rise to the problem or incident. A statement that you have stopped will be considered a pledge to the community; a statement that you have not stopped requires an explanation.

The exposure of subscriber private keys was limited to the Managed WordPress hosting environment and all certificates associated with the incident have been revoked. Subscriber certificate issuance was not impacted.

4. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.

457,911 subscriber certificates were not revoked within 24 hours of notification of compromise. Although not revoked within 24 hours, all certificates have since been revoked. Subscriber certificate issuance was not impacted.

5. The complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem.

See attached file, “MWPCertificates.zip”, for list of impacted certificates (crt.sh links)

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

In the case of this security incident, remediation of the Managed WordPress hosting environment was required prior to certificate revocation. This was to safeguard against additional exposure. The Managed WordPress provisioning system executed rotations of passwords and certificates (including requesting and installation), which consumed the necessary upstream system resources. Password rotation was completed prior to certificate reissuance and revocation.

While GoDaddy CA systems generally process issuance and revocation events in a timely manner, an incident of this scale (400k+ revocations) is extremely infrequent, and our systems were not able to scale as needed. Notwithstanding the delay in revocation, even if we began revocations at the zero-hour mark, it would have taken approximately 50+ hours to complete the revocation processing through the system.

In addition to the items listed above, in our efforts to expedite the revocation process, an administrative error resulted in the revocation of approximately 125k newly reissued certificates. This error was identified on November 23, 2021 at 13:49 and required additional time and resources to reissue certificates to address customer downtime. This error further delayed the reissue and revocation process.

7. List of steps your CA is taking to resolve the situation and ensure that such situation or incident will not be repeated in the future, accompanied with a binding timeline of when your CA expects to accomplish each of these remediation steps.

Our remediation efforts are focused on the ability to scale the CA such that large scale events will be trivial from a customer and system perspective. Additionally, we will reinforce best practices with upstream partners to assist in preventing future events.

Below is a list of specific remediation actions and the expected completion time:

Revoke all certificates associated with the incident (Completed)
Internal After-Action Security Review (Target completion: 12/31/2021): We will be a stakeholder in a security-led review over the event in totality which will give us a better understanding of where we may be able to build out additional guidelines for upstream teams.
Targeted Consumer Guidelines (Target Completion: 3/31/2022): Based on the after-action review, we will better understand details about the subscriber key exposure. We’ll use this information to reinforce expectations across all existing, large consumers.
Increase CA servers (Target completion: 3/31/2022): We will add more nodes to the existing infrastructure allowing the team to increase the speed of event processing. Additionally, as part of the process, we will generate a node generation playbook which will allow the teams to reuse the process and scale up faster if needed.

GoDaddy leadership and stakeholder teams are committed to scaling and automating these systems. In addition to the targeted items above, we are also evaluating the overall design of our CA/PKI systems.

Rob Stradling

Comment 16

•

2 years ago

The provisioning system is part of the Managed WordPress hosting environment and acts as a consumer of certificates from the GoDaddy CA by making certificate requests and receiving issued certificates from the GoDaddy CA for Managed WordPress hosting customers. It is logically and physically separate from the GoDaddy CA.

Hi Brittany. Are users of GoDaddy’s Managed WordPress hosting environment able to "bring their own" certificates issued by other CAs? Or does the platform exclusively provision certificates from the GoDaddy CA?

Also, do the users have direct access to their own private keys at any time? (i.e., do the users generate their own keys and upload them to the platform, or can they download them from the platform?)

(At Sectigo we're planning to add all of the affected keys in attachment 9254193 [details] to our compromised key blocklist, but we're keen to understand if there's any other way that our customers might have been affected by this breach. Thanks!)

Flags: needinfo?(brittany)

Aman Bhutani

Comment 17

•

2 years ago

I wanted to add a short statement from GoDaddy as well. Starting with me and our senior leaders, at all levels we understand that this was a serious incident. We are committed to getting our processes and systems right. We are working across the company to provide the appropriate teams with the support needed to prevent such events and keep the data for our customers secure.

Thanks,
Aman

Brittany Randall

Assignee

Comment 18

•

2 years ago

(In reply to Rob Stradling from comment #16)

Thanks for your questions, Rob. Answers are below.

Are users of GoDaddy’s Managed WordPress hosting environment able to "bring their own" certificates issued by other CAs?

No.

Does the platform exclusively provision certificates from the GoDaddy CA?

Yes. The platform provisions certificates specifically from a GoDaddy CA.

Also, do the users have direct access to their own private keys at any time? (i.e., do the users generate their own keys and upload them to the platform, or can they download them from the platform?)

No. Users do not have direct access to their private keys. The keys are generated by the Managed WordPress hosting provisioning system and are not readily available to customers.

(At Sectigo we're planning to add all of the affected keys in attachment 9254193 [details] to our compromised key blocklist, but we're keen to understand if there's any other way that our customers might have been affected by this breach. Thanks!)

Best,

Brittany

Flags: needinfo?(brittany)

Rob Stradling

Comment 19

•

2 years ago

Thanks Brittany!

Jeremy Rowley

Comment 20

•

2 years ago

Is there a way to get a complete list of the certs in a spreadsheet or something like that? We'd like to extract the keys from the certs and add them to our blocklist.

Jeremy Rowley

Comment 21

•

2 years ago

Specifically, we're hoping to get the certs instead of a long list of serial numbers. That way we don't have to pull the certs from Censys, extract the keys, and then add them.

Brittany Randall

Assignee

Comment 22

•

2 years ago

Hi Jeremy. It was too large to upload the certificate data file to the bug directly. Here is a link to a shared location: https://drive.google.com/file/d/1XqMMzRLOwtphlK-vvODrs0dlapVmnP0q/view?usp=sharing

Best,

Brittany

michael.b

Comment 23

•

2 years ago

Hi, our site is in "revoked_certs_serials.txt" but still facing "Error code: SEC_ERROR_REVOKED_CERTIFICATE" message in Firefox(95.0latest) . Also sometimes error occours in Chrome browser too!
Is tht mean our godaddy hosted wordpress site is still need to be revoked ssl certificate?

Brittany Randall

Assignee

Comment 24

•

2 years ago

(In reply to michael.b from comment #23)

Hi, our site is in "revoked_certs_serials.txt" but still facing "Error code: SEC_ERROR_REVOKED_CERTIFICATE" message in Firefox(95.0latest) . Also sometimes error occours in Chrome browser too!
Is tht mean our godaddy hosted wordpress site is still need to be revoked ssl certificate?

I would recommend reaching out to GoDaddy’s customer support (https://www.godaddy.com/contact-us). They will be able to troubleshoot any specific issues you’re having with your GoDaddy products. Thanks, Brittany

Phil Porada

Comment 25

•

2 years ago

Could GoDaddy provide signed CSRs with CN=Proof of Key Compromise (e.g. like DigiCert did: https://bugzilla.mozilla.org/show_bug.cgi?id=1675684 )

We can conclude with pretty good confidence that a) GoDaddy generated & controlled the private keys (as the hosting status of most affected sites is visible); b) GoDaddy issued the certificates; and c) a GoDaddy representative confirmed the compromise. It seems like a good practice to enable CAs to confirm compromise via CSRs with 100% confidence, and not have to go through this kind of analysis.

Brittany Randall

Assignee

Comment 26

•

2 years ago

Hi Phil, Thanks for the question. This is not something we are able to provide as the private keys have been destroyed. However, this is a great idea of something we may consider in the future. We do see the value in being able to share key compromise information across CAs on an ongoing basis, and are open to working on this together with the community.

Brittany Randall

Assignee

Comment 27

•

2 years ago

Update on progress made against #7 – actions taken by the CA

Internal After-Action Security Review (Original target completion: 12/31/2021, updated target completion: 2/11/2022) – Target completion date updated, and preliminary feedback sessions held with compliance. During the December PKI compliance meetings, the team reviewed the incident and collected targeted feedback from PKI dev, PKI product, and RA who were impacted by the incident downstream in preparation for formal security AAR scheduled for January 2022. Change in target completion date accounts for scheduling across multiple stakeholder teams and holidays.

There are no additional updates at this time. We will continue to monitor for questions. Hope everyone has a great holiday!

Brittany Randall

Assignee

Comment 28

•

•

2 years ago

(In reply to Brittany Randall from comment #30)

(In reply to zach from comment #29)

I've been doing some research into Godaddy injections "in the wild" that could be related to this ongoing security incident - details @ https://victorymedium.com/godaddy-global-issues-canadian-pharmacy-injections/ // the domains in my research don't seem to align to the domains released by GoDaddy as having their certificates revoked -- it's unclear to me why that is the case but just wanted to point out that minor conflict.

Thanks for any comments/notes on the research and the ongoing updates on this thread.

Thank you for sharing this with us. We've reviewed and confirmed that the information in your post is not related to the Managed WordPress incident we've referenced within this bug report. Please feel free to send any future GoDaddy product security concerns to security@godaddy.com (Link for reference: https://www.godaddy.com/legal/agreements/coordinated-vulnerability-disclosure).

My research is about websites on your Managed Wordpress hosting, attacking .gov resources as recently as 5 days ago. You're saying publicly right now, that the breach that GoDaddy reported to the SEC, is "not related to the Managed WordPress incident we've referenced within this bug report."

That's a VERY interesting position to be taking.

This means that there is a second ongoing breach - my 10k words is NOT accounted for in the SEC breach details. Is that correct?

GoDaddy doesn't reply to messages sent to that email, and ya'll famously have no bug bounty program. And are in the middle of an active breach that was reported to the SEC, and your domains are actively attacking .gov infrastructure through attacks on the Googlebot.

Is there a second notice to the SEC coming very soon about this apparently unrelated issue impacting gov resources? Or does GoDaddy believe that the SEC doesn't require any notice about this type of attack on the U.S. government?

Chris Hauser

Comment 32

•

2 years ago

(In reply to zach from comment #31)

(In reply to Brittany Randall from comment #30)

(In reply to zach from comment #29)

I've been doing some research into Godaddy injections "in the wild" that could be related to this ongoing security incident - details @ https://victorymedium.com/godaddy-global-issues-canadian-pharmacy-injections/ // the domains in my research don't seem to align to the domains released by GoDaddy as having their certificates revoked -- it's unclear to me why that is the case but just wanted to point out that minor conflict.

Thanks for any comments/notes on the research and the ongoing updates on this thread.

Thank you for sharing this with us. We've reviewed and confirmed that the information in your post is not related to the Managed WordPress incident we've referenced within this bug report. Please feel free to send any future GoDaddy product security concerns to security@godaddy.com (Link for reference: https://www.godaddy.com/legal/agreements/coordinated-vulnerability-disclosure).

My research is about websites on your Managed Wordpress hosting, attacking .gov resources as recently as 5 days ago. You're saying publicly right now, that the breach that GoDaddy reported to the SEC, is "not related to the Managed WordPress incident we've referenced within this bug report."

That's a VERY interesting position to be taking.

This means that there is a second ongoing breach - my 10k words is NOT accounted for in the SEC breach details. Is that correct?

GoDaddy doesn't reply to messages sent to that email, and ya'll famously have no bug bounty program. And are in the middle of an active breach that was reported to the SEC, and your domains are actively attacking .gov infrastructure through attacks on the Googlebot.

Is there a second notice to the SEC coming very soon about this apparently unrelated issue impacting gov resources? Or does GoDaddy believe that the SEC doesn't require any notice about this type of attack on the U.S. government?

Zach,

Brittany brought your post to my attention. While this isn't the right forum to dive deep into your article, there's a few things I want to make clear:

We will not be filing another SEC incident about a breach any time soon
You are seeing Black Hat SEO tactics on several hosting accounts, across different platforms, including several platforms that were completely unrelated to our recent Managed WordPress incident.
wsimg.com was not meant to and no longer resolves
Customers are responsible for the content of their websites. We’re reaching out to the customers you mentioned in your post and are alerting them to the presence of Black Hat SEO on their websites and are recommending steps to clean up their website and protect themselves from further attacks.

Thank you again for taking the time to raise this and please reach out if you have any other questions. Since this is unrelated to the SSL incident, I would ask that you contact me directly.

Thanks,

Chris

zach

Comment 33

•

2 years ago

Howdy GoDaddy,

Thanks for replying.

Oddly worded!
I wrote 10k words with 110 pages of screenshots to prove the connection to Godaddy infrastructure - I'm calling ~BS on what you're saying here. My receipts were clear -- your statement is not and the public needs to understand what is happening on your infrastructure.
Okay cool, so as per my research, you owe the public an explanation about what has been happening on that domain since 2017.
Customers are responsible for the content of their websites... unless the data controller (GoDaddy) makes a client-wide hosting decision to inject javascript from a domain in 2017, and then "loses control" of that domain years later, without removing the injected javascript code. As GoDaddy just noted in #3, the domain I focused on in my research was "not meant to and no longer resolves" - aka, why was that happening, what risks did it create, and how has this been exploited to inject malicious redirects on your customers websites?

I emailed ya'll... so please email me back if you'd like correspondence outside of public channels... on twitter @ thezedwards if you need to DM for an email, cheers

Chris Hauser

Comment 34

•

2 years ago

(In reply to zach from comment #33)

Howdy GoDaddy,

Thanks for replying.

Oddly worded!

I wrote 10k words with 110 pages of screenshots to prove the connection to Godaddy infrastructure - I'm calling ~BS on what you're saying here. My receipts were clear -- your statement is not and the public needs to understand what is happening on your infrastructure.

Okay cool, so as per my research, you owe the public an explanation about what has been happening on that domain since 2017.

Customers are responsible for the content of their websites... unless the data controller (GoDaddy) makes a client-wide hosting decision to inject javascript from a domain in 2017, and then "loses control" of that domain years later, without removing the injected javascript code. As GoDaddy just noted in #3, the domain I focused on in my research was "not meant to and no longer resolves" - aka, why was that happening, what risks did it create, and how has this been exploited to inject malicious redirects on your customers websites?

I emailed ya'll... so please email me back if you'd like correspondence outside of public channels... on twitter @ thezedwards if you need to DM for an email, cheers

Thank you for emailing us through the proper channel at security@godaddy.com. Since this is unrelated to this SSL incident, we will consider this thread closed on this forum, but will follow up through our security response process.

Ben Wilson

Updated

•

2 years ago

Whiteboard: [ca-compliance] → [ca-compliance] Next update 2022-02-11

Updated

•

1 year ago

Product: NSS → CA Program

David Lawrence [:dkl]

Updated

•

1 year ago

Whiteboard: [ca-compliance] [delayed-revocation-leaf] Next update 2022-03-31 → [ca-compliance] [leaf-revocation-delay]

revoked_certs_serials.txt 2 years ago Brittany Randall 7.39 MB, text/plain		Details
MWPCertificates.zip 2 years ago Brittany Randall 2.71 MB, application/x-zip-compressed		Details
Starfield RKGC Indp Audit Opinion and Mgmt Assertion June 2022 - FINAL.pdf 2 years ago daryn 469.80 KB, application/pdf		Details
GoDaddy WTEV SSL Indp Acct Opinion and Mgmt Assertion June 2022 FINAL.pdf 2 years ago daryn 529.59 KB, application/pdf		Details
GoDaddy WTCS BR Indp Acct Opinion and Mgmt Assertion June 2022 - FINAL.pdf 2 years ago daryn 508.70 KB, application/pdf		Details
GoDaddy WTCA Indp Acct Opinion and Mgmt Assertion June 2022 - FINAL.pdf 2 years ago daryn 543.17 KB, application/pdf		Details
GoDaddy WTBR Indp Acct Opinion and Mgmt Assertion June 2022 - FINAL.pdf 2 years ago daryn 538.71 KB, application/pdf		Details
Auditor Qualifications WTCA Engagement 2022 - FINAL.pdf 2 years ago daryn 147.19 KB, application/pdf		Details