1001709 - Update *.services.mozilla.com SSL certificate on PHX1 Zeus

Status: RESOLVED FIXED

(Infrastructure & Operations :: SSL Certificates, task)

Description

[Gene Wood [:gene]]

Related to Bug 993094

It looks like the static-san.mozilla.org certificate which applies to these sites :

static-san.mozilla.org
addons.mozilla.com
autoconfig-live.mozillamessaging.com
autoconfig.thunderbird.net
broker-live.mozillamessaging.com
live.mozillamessaging.com
live.thunderbird.net
nightly.mozilla.org
getfirefox.com
www.getfirefox.com
opensearch-live.mozillamessaging.com
dnt.mozilla.org
support.live.mozillamessaging.com
firefox.com
www.firefox.com
gaming.mozillalabs.com
apps.mozillalabs.com
webmaker.mozillalabs.com
support.mozillamessaging.com
heatmap.mozillalabs.com
videos-cdn.mozilla.net
videos.mozilla.org
planet.mozilla.org
publicsuffix.org
www.publicsuffix.org
static.mozilla.com
mozilla.com
www.mozilla.com
activations.mozilla.com
activations.mozilla.org

was not updated post heartbleed (as the cert notBefore date is February 6th)
and has now been revoked

If indeed this keypair is from before heartbleed and therefore potentially leaked, please issue a new cert

Please deploy this new cert on all sites using it as currently they are down due to the certificate being revoked

Comment 1

[Brandon Burton [:solarce]]

The certificate shouldn't have been revoked, the cluster it's used for is behind Zeus and so was not vulnerable to heartbleed

I am looking into this

Comment 2

[Brandon Burton [:solarce]]

It looks like one instance of the certificate was revoked back in January, https://www.digicert.com/enterprise/order-details.php?order_id=00455181 , but a new one was ordered and is the one that's currently in place, https://www.digicert.com/enterprise/order-details.php?order_id=00480320

The current certificate was just re-issued yesterday as additional SANs were added.

The current live certificate serial matches between what Digicert shows, what's in Zeus's UI, and what I get via 'openssl' on the CLI, 

* Digicert: 08948EEB6BE3ECC2E971EB102176EBC5
* Zeus: Serial:  	08:94:8e:eb:6b:e3:ec:c2:e9:71:eb:10:21:76:eb:c5
* openssl: 
 -> % openssl s_client -connect firefox.com:443 2>&1|openssl x509 -noout -serial
 serial=08948EEB6BE3ECC2E971EB102176EBC5

Can you provide additional details on which domain you're seeing as revoked and where?

Comment 3

[Gene Wood [:gene]]

The site that exhibits this is https://docs.services.mozilla.com/

It's possible that his is an issue with only this one site since I haven't seen it when I browse to the other sites on this multi-san cert

Comment 4

[Brandon Burton [:solarce]]

(In reply to Gene Wood [:gene] from comment #3)
> The site that exhibits this is https://docs.services.mozilla.com/
> 
> It's possible that his is an issue with only this one site since I haven't
> seen it when I browse to the other sites on this multi-san cert

Ah! that site uses the *.services.mozilla.com certificate

* Server certificate:
* 	 subject: C=US; ST=CA; L=Mountain View; O=Mozilla Foundation; CN=*.services.mozilla.com
* 	 start date: 2014-02-24 00:00:00 GMT
* 	 expire date: 2017-02-28 12:00:00 GMT
* 	 subjectAltName: docs.services.mozilla.com matched
* 	 issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
* 	 SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
> HEAD / HTTP/1.1
> User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8y zlib/1.2.5
> Host: docs.services.mozilla.com

It looks like the copy in Zeus didn't get updated, I'll update it

Comment 5

[Brandon Burton [:solarce]]

https://docs.services.mozilla.com/ is now working as expected!

Comment 6

[Gene Wood [:gene]]

Thanks solarce! Sorry for the confusing ticket (I mis-diagnosed what was going on)