1489293 - Investigating SSL Outage on extensiontest.com

Status: RESOLVED FIXED

(Infrastructure & Operations :: DNS and Domain Registration, task)

Description

[Potch [:potch]]

extensiontest.com is registered with Mozilla and is hosted on the mozillacorporation Heroku org.

As of the past few days, the site is down with an SSL error. Looking into the site settings, it reports "Incorrect DNS settings", which could either mean

* DNS of extensiontest.com is not pointed to the correct heroku endpoint
* DNSSEC is enabled and has errors.

Looking for help chasing down the particulars of what's going on.

Comment 1

[Potch [:potch]]

per emorley on IRC:

> I would guess it's not set up to use an ALIAS record (or similar)
> the A record for extensiontest.com has two IPs, neither of which match those for www.extensiontest.com.herokudns.com
> see: https://devcenter.heroku.com/articles/custom-domains#add-a-custom-root-domain
> yeah the domain is still pointing at a mozilla service, not Heroku

Do we know when those A records were set up?

Comment 2

[Keegan Ferrando [:fauweh]]

The DNS for this domain is apparently hosted in AWS, perhaps someone in Cloud Infra can help? The entries we have in infoblox would only be referenced by internal IT hosts in this case. 

$ whois extensiontest.com|grep 'Name Server'
   Name Server: NS-1144.AWSDNS-15.ORG
   Name Server: NS-1639.AWSDNS-12.CO.UK
   Name Server: NS-422.AWSDNS-52.COM
   Name Server: NS-660.AWSDNS-18.NET
Name Server: ns-422.awsdns-52.com
Name Server: ns-1144.awsdns-15.org
Name Server: ns-1639.awsdns-12.co.uk
Name Server: ns-660.awsdns-18.net

Comment 3

[Ed Morley [:emorley]]

Might this be another site affected by bug 1485795?

Comment 4

[Potch [:potch]]

(In reply to Ed Morley [:emorley] from comment #3)
> Might this be another site affected by bug 1485795?

I'm unable to see to see that bug, so I'm not sure.

Comment 5

[Potch [:potch]]

(In reply to Keegan Ferrando [:fauweh] from comment #2)
> The DNS for this domain is apparently hosted in AWS, perhaps someone in
> Cloud Infra can help? The entries we have in infoblox would only be
> referenced by internal IT hosts in this case. 

Who can I contact in Cloud Infra to get some help?

Comment 6

[Philippe M. Chiasson (:gozer)]

www.extensiontest.com wasn't included as a SAN for the cert, currently only extensiontest.com

https://github.com/mozilla-it/haul/commit/1461ddf48c073a7a461ca9ead6f05926a2e57771

Comment 7

[Keegan Ferrando [:fauweh]]

Ok so this route53 record is controlled by gozer's team. He can get this added when he is back in the office tomorrow.

To reiterate the request in Slack:

> ericz
> @gozer www should be a cname to www.extensiontest.com.herokudns.com is specifically what is needed

Comment 8

[Potch [:potch]]

We are back online! thanks all for your help.

Comment 9

[Daniel Hartnell [:danielh]]

Just for reference: fixed through PR: https://github.com/mozilla-it/haul/pull/162