1139589 - Use one-time deploy passwords

Status: RESOLVED WONTFIX

(Infrastructure & Operations :: RelOps: Puppet, task)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

Currently we have a single shared deploy password that can be used to get a key for any hostname you can get into reverse DNS.  We can do better.

The riskiest points for deploy passwords are when they're injected into a newly-instantiated host.  PXE and DeployStudio don't have much in the way of secure crypto, and anyway the new host doesn't have any unique identity yet.

Instead of using a single, shared password, we could set up a system whereby we provide authentication and a hostname to an API and get back a limited-duration password that will only work for that hostname.  When we're manually reimaging hosts with PXE, we'd just type that password into the kernel command line.  aws_create_instance could have its own credentials (limited to only certain hostnames) that it uses to create new passwords every night.  I'm not sure how we'd do DeployStudio.

Comment 1

[Amy Rich [:arr] [:arich]]

We should move away from having any sort of interactive requirement (especially on a crappy sometimes-works java console) to install new machines. Linux in the datacenter is the only place we do this, and it makes it much more difficult to do them in an automated fashion or convince other people to self serve.

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

An alternative we've discussed in the past is to require external confirmation for every cert request.  So the host would come up and request a cert (probably with a shared secret, but a relatively un-protected one, just to make spoofing harder).  Then whatever system issues the certs would require an external, authenticated confirmation that the request was valid before actually issuing the cert.

For automated systems like aws_create_instance, the automation could do the confirmation.  For other systems (manual reinstalls), we could build a basic web UI that displays pending confirmations with "YES" and "NO" buttons (and some bulk-approve option for the really big reimages).

Comment 3

[Amy Rich [:arr] [:arich]]

We should be moving away from deploy passwords all together because it blocks easy automation.