1330482 - GoDaddy: New GoDaddy incorrect issuance bug appears to be regression of 2010 issue

Status: RESOLVED INCOMPLETE

(CA Program :: CA Certificate Compliance, task)

Description

[fred]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36

Steps to reproduce:

The new incorrect issuance with non-200 responses appears to be the same as one of two issues I reported (and was fixed) in 2010:

I reported two issues to GoDaddy back in 2010 (included after timeline) - the second one appears to be the same bug as the new incident being discussed at https://groups.google.com/forum/?hl=en#!msg/mozilla.dev.security.policy/Htujoyq-pO8/uRBcS2TmBQAJ - timeline:

2010-01-09: I wanted an SSL certificate for my domain; I read GoDaddy’s instructions, and realized that my existing configuration probably matched their requirements. I got my SSL certificate with no changes to my DNS or website.

2010-01-10: Notification sent to godaddy (Incident ID 8143623)

2010-01-20: Second notification sent to godaddy (Incident ID 8214596)

2010-01-21: Confirmed the flaw still exists - I got go daddy to issue me a certificate for a friend’s domain (consent 
given, but no cooperation)

2010-08-26: Reported again (id 9711250)

2010-09-03: Phone call with Todd Redfoot (Chief information security officer)
2010-09-23: Email from Todd Redfoot confirming fix deployed

My report from 2010 is below - the first one is no longer valid given the switch to TXT-records for authentication.

----- 1. CNAME verification ——

 In short: if a domain had a wildcard CNAME, anyone could get a certificate for that domain from GoDaddy.

I have tested this approach, and got a godaddy signed certificate with consent 
but not cooperation (and godaddy not being informed of consent).

1. Find a domain which appears to have a wildcard CNAME record (i.e. "dig 
somegarbage.example.com" returns any CNAME record)

2. Ask for permission from the owner (optional if blackhat….)

3. Create a CSR for the domain

4. Sign up for a godaddy domain verified certificate (referred to on the site by 
a mixture of "Standard SSL" or "Turbo SSL" - same product)

5. Submit your CSR

6. Godaddy will attempt some verification via whois; wait a few hours for this 
to fail. They will email to let you know when this happens

7. Sign in to the web panel, and select "domain-based verification"

8. It will immediately confirm that verification was successful

9. In a few minutes, you can download a certificate for the domain

Godaddy's DNS verification is as follows:
- Create a random code
- Require you to make it so that randomcode.yourdomain.com returns a CNAME 
record
- The value of the CNAME record is irrelevant and not specified

So, you can get a domain-verified SSL certificate for a domain with anything 
like any of the following in the zonefile:

* CNAME @
* CNAME example.com
* CNAME pleasedontsignacertificatebasedonthis.com

----- 2. Web-based verification ------

Instructions as above, except:

1. Find a domain where a not-found page returns a page including the URL 
requested. I'd assume that the site must also be misconfigured to return a 200 
response instead of 404, but this is a fairly common misconfiguration.

7. Select web-based verification

Similar cause; godaddy require that http://example.com/godaddyRandomCode 
returns a page including godaddyRandomCode


Actual results:

In 2010:
 - Got SSL certificate for my domain with no changes to my DNS or configuration
 - Got SSL certificate for a friend's domain without their help (but with verbal permission)

Now: issue reported as new


Expected results:

Now: issue should be reported as a regression

Comment 1

[fred]

If this is a regression, follow-up on the current issue should also cover what measures were put in place after my previous report, and why they failed to catch the regression.

Comment 2

[Daniel Veditz [:dveditz]]

Kathleen: is this a separate issue from the one we're already investigating with GoDaddy? If so we can close it (dupe?), if not please add this to the investigation.

Comment 3

[Kathleen Wilson]

Is this the same issue as the one Wayne reported?
https://groups.google.com/d/msg/mozilla.dev.security.policy/Htujoyq-pO8/uRBcS2TmBQAJ

Comment 4

[Ryan Sleevi]

No, this is a different issue, with a different root cause. Note that it was in 2010 - well before industry norms had coalesced.

Comment 5

[fred]

If that comment is regarding a confidential issue, please ignore this comment :)

Based on public information I can find:
 - the CNAME issue from 2010 does not appear to have been publicly discussed, but is not a current issue
 - the web-based issue from 2010 appears to be a duplicate of the current issue, but the current issue describes it as a new issue in 2016 - it does not address that steps were not put in place to prevent a recurrence of the issue in 2010

Comment 6

[Ryan Sleevi]

The issue reported by Wayne is entirely independent then your issue. It has a different root cause, a different mitigation, and a different set of expectations.

Comment 7

[fred]

From Wayne:

"Our system automatically checks for the presence of that code via an HTTP and/or HTTPS request to the website. If the code is found, the domain control check is completed successfully.  Prior to the bug, the library used to query the website and check for the code was configured to return a failure if the HTTP status code was not 200 (success). A configuration change to the library caused it to return results even when the HTTP status code was not 200. Since many web servers are configured to include the URL of the request in the body of a 404 (not found) response, and the URL also contained the random code, any web server configured this way caused domain control verification to complete successfully. "

From me:

"Find a domain where a not-found page returns a page including the URL 
requested. I'd assume that the site must also be misconfigured to return a 200 
response instead of 404, but this is a fairly common misconfiguration."


While I can't comment on root cause/mitigation, the symptoms and exploit method seem the same.

Comment 8

[Kathleen Wilson]

(In reply to fred from comment #7) 
> While I can't comment on root cause/mitigation, the symptoms and exploit
> method seem the same.

To my knowledge, the errors noticed and resolved in 2010 (reported in this bug) were not previously reported to Mozilla. I cannot find a Bugzilla Bug relating to it. So, I am not aware of a prior requirement from Mozilla for GoDaddy to demonstrate that preventative measures and automated tests had been put in place to avoid recurrence of that mistake.

Even so, it is indeed unfortunate that GoDaddy had not put checks into place to prevent them from repeating a past mistake.

However, I think we should track Wayne's report and the forthcoming postmortem review in a separate, new bug. I will expect the full postmortem review to explain why there were not preventative measures in place to prevent this mistake. And I will expect GoDaddy to provide assurance to demonstrate that preventative measures and automated tests are being put in place to avoid recurrence of the mistake.

Comment 9

[fred]

Yep, sorry, I didn't mean to imply they've broken an obligation - just that the history of the issue may be incomplete, and that they weren't following best-practices (even if they weren't formalized at the time).

Comment 10

[Gervase Markham [:gerv]]

Fred: thanks for this report. Reading it through, it does indeed seem to me that GoDaddy has had a very similar problem before.  I notice this bug is confidential. In order for us to ask GoDaddy the obvious question: "Why were proper tests not put in place following the previous near-identical failure?", then the details of your 2010 experience need to be public. Would you care to post a message like your initial bug report here in m.d.s.policy? Or perhaps on your personal blog?

Gerv

Comment 11

[fred]

Will do; only filed as confidential as I wasn't sure what was appropriate, not out of a particular wish for it to remain so.

Comment 12

[fred]

https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/B5a4iUhabrE

Comment 13

[Kathleen Wilson]

Gerv, do you know what the status of this bug is?
Should it be closed?
Or should it be made non-confidential?

Comment 14

[Gervase Markham [:gerv]]

I don't think this bug is now actionable, given the progress we have made with GoDaddy. Closing the bug and making it public.

Gerv