Closed Bug 780316 Opened 12 years ago Closed 12 years ago

generate a new Mozilla CA cert

Categories

(Infrastructure & Operations Graveyard :: WebOps: Other, task, P1)

All
Other

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: nmaul, Assigned: nmaul)

References

Details

(Whiteboard: [triaged 20120824][deadline 20121001][2012q3])

CC'ing people who seem most likely to have a good handle on how this ought to work.

Due to bug 650355, Firefox no longer accepts signatures that are signed by a CA that uses MD5+RSA as the signing algorithm. This applies to current-Aurora (Firefox 16) and newer.

Unfortunately, this includes our current Mozilla Root CA.

First, we need to make a new root CA and get it distributed such that the instructions for using it are still valid:

http://wiki.mozilla.org/MozillaRootCertificate

Perhaps that should encompass both our current root and the new root... or maybe it should no longer reference the current root at all, I don't know.

The scripts on ssl1.private.phx1:/root/root-ca should all be updated or deprecated to "do the right thing". Some will be unaffected, but not all. At least one is ostensibly used to create the root CA itself, and at least 7 other such scripts are supposedly used for signing things. 2 of them are referenced here:

https://mana.mozilla.org/wiki/display/SYSADMIN/SSL+Certificates


Then we need to generate new keys and CSRs for anything affected by this, and sign them with the new CA. It *might* be possible to re-use the existing CSRs for some things, but we'd probably be better off to just remake them- some are quite old, and probably use a smallish key size or are themselves using MD5 as the signing hash.

Bugs for each individual affected site/cert should be blocked by this bug.
Blocks: 777812
In my testing, the current new-root-ca.sh script does sign with sha1 now... must be a default change in OpenSSL at some point since our CA was generated. However, it does generate what a too-small key and cert (1024 bits).

If we want this CA to be usable for 10 years (2022), it needs to be at least 2048 bits, and by some measurements even bigger. To err on the side of caution, I'm going to modify this script to generate a CA with a key length of 4096 bits.

Note that we cannot simply generate a new cert signed by the old one and then start using that. Firefox will still reject the top-level one.


I have generated what I believe to be a suitable key and cert. They are in ~/jakem/ on ssl1.private.phx1. I simply duplicated the information in the current cert, and used the same key passphrase (which should probably be changed).

If this looks good to people, I recommend we move the copy in root-ca/ to root-ca/old/, replace the new-root-ca.sh script with my changed copy (in ~/jakem/), and move my generated key and crt into ~/root-ca/. It's possible no other scripts on this system will need updated.

Then we can start work on deploying this new CA to people, by making sure the stuff on the wiki page in comment 0 is still valid.

*Then* we can start replacing certs in any blocked bugs (at least ringring, probably many others).
The root CA affects more than just WebOps, so moving this to the main ServerOps component.
Assignee: server-ops-webops → server-ops
Component: Server Operations: Web Operations → Server Operations
QA Contact: cshields → jdow
Adding Joe S... OpSec should probably hear about this as well.
Assignee: server-ops → server-ops-webops
Component: Server Operations → Server Operations: Web Operations
QA Contact: jdow → cshields
cc: me, as i'm responsible for the svc-oob subCA certificate, which will need to be re-signed by the new root CA when it's available.

:jakem - please add a note to https://wiki.mozilla.org/MozillaRootCertificate linking to this bug, and please reconsider marking this bug as private.
To list all certificates on a Zeus server that are signed by root-ca:

find /usr/local/zeus/{admin/etc,zxtm/conf}/ -name '*.public' -type f | xargs -n1 -I{} bash -c 'openssl x509 -noout -in "{}" -text 2>&1 | grep -q md5WithRSAEncryption && echo "{}";'
(In reply to Richard Soderberg [:atoll] from comment #5)
> To list all certificates on a Zeus server that are signed by root-ca:

Confirmed that all Services Zeus clusters (zlb*{,.pub}.{phx1,scl2,mtv1,*.stage}.svc.m.c) are using certificates with Signature Algorithm: sha1WithRSAEncryption.
Group: infra
Whiteboard: [pending triage]
Priority: -- → P2
Whiteboard: [pending triage] → [triaged 20120824][deadline 20121001]
Whiteboard: [triaged 20120824][deadline 20121001] → [triaged 20120824][deadline 20121001][2012q3]
This is now a Q3 goal (due to the deadline set by Firefox 16 launch), per our webops meeting today. Bumping to P1.
Priority: P2 → P1
Other zeus clusters checked:

pp-zlb* (PHX1 public and private clusters) are clean

www.zlb.ops.scl3 (SCL3 public) has 3 Mozilla CA certs:
    *.mozilla-europe.org
    *.bugzilla-stage-tip.mozilla.org
    bugzilla-stage-tip.mozilla.org

internal.zlb.ops.scl3 (SCL3 internal) is clean

ams-zlb01.nl (AMS1 public) is clean

zlb1.vips.pek1 (PEK1 public) is clean



I will edit the wiki page in comment 0 and comment 4, make a note on Yammer regarding the new CA, and send an email to all@. That should cover the bases fairly well.

Once all that is done, it should be safe to start re-signing CSRs with the new CA key.
Assignee: server-ops-webops → nmaul
Depends on: 789321
Whiteboard: [triaged 20120824][deadline 20121001][2012q3] → [triaged 20120824][deadline 20121001][2012q3][waiting][webdev]
(In reply to Richard Soderberg [:atoll] from comment #4)
> cc: me, as i'm responsible for the svc-oob subCA certificate, which will
> need to be re-signed by the new root CA when it's available.

The svc-oob subCA has been resigned with the new root certificate. Existing certs signed by svc-oob have been verified to continue working once the certificate is swapped out in Firefox.
The wiki page is updated with current information. I'm sending out emails/posts now to inform people of the change. Individual certs that need updating can be handled in separate bugs (there's already one for ringring).
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Whiteboard: [triaged 20120824][deadline 20121001][2012q3][waiting][webdev] → [triaged 20120824][deadline 20121001][2012q3]
Component: Server Operations: Web Operations → WebOps: Other
Product: mozilla.org → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.