1369143 - buy new primary and backup cert for aus5.mozilla.org

Status: RESOLVED FIXED

(Cloud Services :: Operations: Product Delivery, task)

Description

[[e-mail] (:bhearsum)]

I had mentioned that the backup was expiring, but it turns out the primary is expiring as well. The primary from DigiCert expires on July 28th and the backup from Thawte expires on August 10th.

The new certs must be issued by intermediaries that have:
* CN=DigiCert SHA2 Secure Server CA,O=DigiCert Inc,C=US
* CN=thawte SSL CA - G2,O=thawte, Inc.,C=US

Comment 2

[Shyam Mani [:fox2mike]]

I'd also filed Bug 1366031, feel free to dupe this if needed.

Comment 3

[Daniel Thorn [:relud]]

ni :bhearsum to confirm, but I believe we are only using the digicert ssl cert actively, and the thawte cert is only a backup. I can only find configurations on my end for that, but sometimes there are pieces I don't know about.

:fox2mike in Bug 1366031 you offered to renew the thawte cert and hand if off, that would be great if you could do that, so long as :bhearsum agrees it is backup only. My gpg fingerprint is in phonebook.

I can handle getting a digicert one reissued, and updating the cert will expire the old one in 48 hours, so I will handle that one on my end on the 17th when I return from PTO (rather than changing the SSL cert right before leaving).

Comment 4

[Daniel Thorn [:relud]]

I can't read, and bhearsum stated in comment 0 that thawte is the backup.

Comment 5

[[e-mail] (:bhearsum)]

Is it possible to renew the cert in a way where the old one doesn't automatically expire? I'm slightly concerned that the new one may end up with different issuer information (that would break our pinning). It would be good to have more than 48 hours to deal with that on the off chance it happens.

Comment 6

[Daniel Thorn [:relud]]

The standard way to issue a new cert in cloudops is to generate a new private key and CSR, and have digicert approve that. if we re-use the old private key, then the old cert doesn't get expired, but we aren't allowed to do that without a very good reason.

:ulfr what do you think, should we rotate the key and accept the risk of losing the old cert after 48 hours when something goes wrong? I'm leaning towards accepting the risk. We update certs often enough that it's not an unfamiliar task, and I feel very confident that the CN for the intermediate will be correct.

Comment 7

[Julien Vehent]

I share Ben's concerns. We should make an exception for these certs, not rotate the key pairs, and simply renew the certificates (which won't revoke the old ones).

Breaking AUS is one of those things we're so worried about we relax security rules for it.

Comment 8

[Daniel Thorn [:relud]]

okay, we'll re-use then

Comment 9

[Shyam Mani [:fox2mike]]

:relud,

We don't do anything special with Thawte, I'm wondering if you guys should go ahead and take over the Thawte cert as well, since everything else here is managed by cloudops. Thoughts?

Comment 11

[Daniel Thorn [:relud]]

if all the thawte certs are owned by cloudops, then i don't see a reason why we shouldn't take it over. I'll contact you later this week to see about getting creds, and make sure my manager is okay with us taking that on.

Comment 12

[Daniel Thorn [:relud]]

the new primary cert is bought, and tested. it will go live on thursday.

Comment 13

[Daniel Thorn [:relud]]

new primary cert is live. reassigning to :ulfr so he can handle taking over thawte and getting a new backup, or whatever he's doing there.

Comment 14

[Julien Vehent]

I'd like to remove the dependency to thawte. This isn't a CA we have an active relationship with, and isn't one of the CA we pin to in Firefox (Digicert and LE are pinned).

Is there any objection to removing this pin? If a backup is needed, LE is the CA we should use to complement Digicert.

Comment 15

[[e-mail] (:bhearsum)]

(Apologies - I wrote this up last week but forgot to submit it.)

(In reply to Julien Vehent [:ulfr] from comment #14)
> I'd like to remove the dependency to thawte. This isn't a CA we have an
> active relationship with, and isn't one of the CA we pin to in Firefox
> (Digicert and LE are pinned).
> 
> Is there any objection to removing this pin? If a backup is needed, LE is
> the CA we should use to complement Digicert.

I think it's important to buy a new backup from Thawte for now because GMP has been pinned to DigiCert + Thawte SHA-2 servers since 42.0 (https://wiki.mozilla.org/Balrog/Client_Domains#Pinning_Requirements has details on all of our pinning). If something happens to the DigiCert root while pinning is still enabled, we would have no way to ship GMP to these clients.

I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1330393 recently to try to get us to remove that. I would love if we could push forward on it, but I don't think it means we can do without a backup in the short term.

Comment 16

[[e-mail] (:bhearsum)]

Any progress on the backup cert? The current one expires in 5 days.

Comment 17

[Julien Vehent]

We haven't tried, because we were focused on bug 1340880. In that bug, :relud got a cert issued by /C=US/O=thawte, Inc./CN=thawte SSL CA - G2, which is the intermediate we want for aus5, so it looks like we can obtain a backup without going through the trouble we had with aus3.

Comment 18

[[e-mail] (:bhearsum)]

AFAIK this is done now? Relud gave me details of the new backup cert that I added to https://wiki.mozilla.org/Balrog/Client_Domains#SSL_Certificates.