660318 - revise POSIX puppet/buildbot startup process

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P5)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

You can see how slaves are started now:
  https://wiki.mozilla.org/ReleaseEngineering/Buildslave_Startup_Process

I'd like to do for *all* slaves what I've proposed for macs in bug 658678:

 * a universal run-puppet-until-it's-done shell script
   * reboot after N failures (N=10 now)
 * block system startup on the script's completion
   * already done on Centos
   * bug 658678 shows how to do it on macs
   * easy to do on Fedora
 * Startup buildbot either at graphical login or via another initscript
 * ensure SSH logins are enabled before puppet starts

This:

 1. separates puppet startup from buildbot startup
 2. works the same way everywhere

Finally, along with a basic install of Buildbot, we can include this system in our "clean" reference images (a project that will be making progress in the new relops lab) for all of these platforms.  Then a newly-imaged system will immediately come up and try to configure itself from puppet, with no login required.  Just sign the cert on the puppet master (or let autosign do it..).

Comment 1

[Chris Cooper [:coop] (he/him)]

Dustin: is there anything left to do here after the migration to PuppetAgain? I don't think we really tackled the startup process as part of that move.

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

This pretty much got implemented.