764658 - puppetize.sh needs to work properly in darwin

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: General, task)

Description

[Jake Watkins [:dividehex]]

Currently, puppetize.sh does not work 'out of the box' on darwin.
We will need to either add darwin compatibility or create a separate script just for darwin.

Darwin incompatibles:
/root does not exist by default
wget is not installed by default; curl should be used in place
/etc/rc.d/ does not exist
shred is not installed by default
puppet on darwin wants ssl certs in /etc/puppet/ssl/ not /var/lib/puppet/ssl/

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

(In reply to Jake Watkins [:dividehex] from comment #0)
> Darwin incompatibles:
> /root does not exist by default
This can be a variable that's set based on [ -d /root ] or a facter check.

> wget is not installed by default; curl should be used in place

If we can change everything to use curl, that'd be great.

> /etc/rc.d/ does not exist

I think we'll need to do that kind of stuff in a conditional, since setting something up to start at boot is *completely* different in Darwin.

> shred is not installed by default

We can probably pass on this, at least in a first approximation.

> puppet on darwin wants ssl certs in /etc/puppet/ssl/ not /var/lib/puppet/ssl/

I'd *really* like for this not to be the case, but again, it can be solved with a simple conditional.

I'm not totally opposed to writing a Darwin script, but there may be a bit of pain involved keeping multiple scripts coordinated if we change the puppetization process at all.  This has already been painful for the kickstart and ganeti setup scripts, which share a lot of code.

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

For the record, I installed puppet as follows:

r5-puppetagain-2:~ administrator$ curl http://downloads.puppetlabs.com/mac/puppet-2.7.12.dmg > puppet-2.7.12.dmg   
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1037k  100 1037k    0     0   428k      0  0:00:02  0:00:02 --:--:--  508k
r5-puppetagain-2:~ administrator$ inst
install        install-info   installer      instmodsh      instmodsh5.10  instmodsh5.12  instruments    
r5-puppetagain-2:~ administrator$ inst
install        install-info   installer      instmodsh      instmodsh5.10  instmodsh5.12  instruments    
r5-puppetagain-2:~ administrator$ inst
install        install-info   installer      instmodsh      instmodsh5.10  instmodsh5.12  instruments    
r5-puppetagain-2:~ administrator$ hdiutil attach puppet-2.7.12.dmg 
Checksumming Driver Descriptor Map (DDM : 0)…
     Driver Descriptor Map (DDM : 0): verified   CRC32 $F84A44D5
Checksumming Apple (Apple_partition_map : 1)…
     Apple (Apple_partition_map : 1): verified   CRC32 $290A09BD
Checksumming disk image (Apple_HFS : 2)…
............................................................................................................................................................................................................................................................................................................................................................................................
          disk image (Apple_HFS : 2): verified   CRC32 $F1CA2AF2
Checksumming  (Apple_Free : 3)…
                    (Apple_Free : 3): verified   CRC32 $00000000
verified   CRC32 $B07B1192
/dev/disk3              Apple_partition_scheme          
/dev/disk3s1            Apple_partition_map             
/dev/disk3s2            Apple_HFS                       /Volumes/puppet-2.7.12
r5-puppetagain-2:~ administrator$ sudo installer -pkg /Volumes/puppet-2.7.12/puppet-Password:
installer: Package name is puppet-2.7.12
installer: Installing at base path /
installer: The install was successful.
r5-puppetagain-2:~ administrator$ 


(puppetize.sh assumes puppet is already installed)

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

And it took me way too long to figure out I needed to do the same with facter (1.6.7).

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: bug764658.patch

tested in its final form on both r5-puppetagain-2 and relabs08, both interactively and with a deploypass.

This expects puppet and facter to be installed before its runs.  I've removed the requirement for wget.  The system python for either Linux or Darwin is sufficient.

Python will save the world.  I don't know how, but it will.

Comment 5

[Jake Watkins [:dividehex]]

Comment on attachment 642137 [details] [diff] [review]
bug764658.patch

+    python <<EOF
+import urllib2, getpass
+deploypass="$deploypass"
+if not deploypass:
+    deploypass = getpass.getpass('deploypass: ')
+password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
+password_mgr.add_password(None, 'https://puppet', 'deploy', deploypass)
+handler = urllib2.HTTPBasicAuthHandler(password_mgr)
+opener = urllib2.build_opener(handler)
+data = opener.open('https://puppet/deploy/getcert.cgi').read()
+open("$ROOT/certs.sh", "w").write(data)
+EOF

I think this is acceptable. I love the the use of pthyon for this but I'm not a fan of mixing shell script and python (or any other lang).  In fact, I wouldn't object to the entire puppetize.sh becoming puppetize.py :-)  A later date maybe.


+if [ $OS = "Linux" ]; then
+    # make sure the time is set correctly, or SSL will fail, badly.
+    ntprunning=`ps ax | grep ntpd | grep -v grep`
+    [ -n "$ntprunning" ] && /sbin/service ntpd stop
+    /usr/sbin/ntpdate pool.ntp.org
+    [ -n "$ntprunning" ] && /sbin/service ntpd start
+fi

if we want to test for the ntp daemon running on darwin I would suggest using launchctl to handle this.  Also, this doesn't really make sure the time is set correctly, only that launchd has the ntpd config loaded (its up to launchd to make sure it is actually running and up to ntp to sync the systems clock to the correct (or incorrect?) time).  There are a lot of assumptions being made here.

eg. 

if ! launchctl list org.ntp.ntpd > /dev/null 2>&1 ; then 
    launchctl load /System/Library/LaunchDaemons/org.ntp.ntpd.plist
fi


+        else
+            echo "removing deploypass"
+            rm $ROOT/deploypass || hang
+        fi

I am review- this because we should be using 'srm' for secure file removal on a darwin system.  srm is an osx "shred" like tool.

Comment 6

[Dustin J. Mitchell [:dustin] (he/him)]

(In reply to Jake Watkins [:dividehex] from comment #5)
> +if [ $OS = "Linux" ]; then
> +    # make sure the time is set correctly, or SSL will fail, badly.
> +    ntprunning=`ps ax | grep ntpd | grep -v grep`
> +    [ -n "$ntprunning" ] && /sbin/service ntpd stop
> +    /usr/sbin/ntpdate pool.ntp.org
> +    [ -n "$ntprunning" ] && /sbin/service ntpd start
> +fi
> 
> if we want to test for the ntp daemon running on darwin I would suggest
> using launchctl to handle this.  Also, this doesn't really make sure the
> time is set correctly, only that launchd has the ntpd config loaded (its up
> to launchd to make sure it is actually running and up to ntp to sync the
> systems clock to the correct (or incorrect?) time).  There are a lot of
> assumptions being made here.
> 
> eg. 
> 
> if ! launchctl list org.ntp.ntpd > /dev/null 2>&1 ; then 
>     launchctl load /System/Library/LaunchDaemons/org.ntp.ntpd.plist
> fi

Yep, thanks for the invocation!  The idea is to stop ntp long enough to run ntpdate, then start it back up again.

> +        else
> +            echo "removing deploypass"
> +            rm $ROOT/deploypass || hang
> +        fi
> 
> I am review- this because we should be using 'srm' for secure file removal
> on a darwin system.  srm is an osx "shred" like tool.

Perfect - nobody else knew that when I asked around :)

Comment 7

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: bug764658.patch

Comment 8

[Jake Watkins [:dividehex]]

Comment on attachment 642932 [details] [diff] [review]
bug764658.patch

Looks good :-)