Closed Bug 917768 Opened 11 years ago Closed 11 years ago

[captain] Create puppet manifest/templates for shove daemon

Categories

(Infrastructure & Operations Graveyard :: WebOps: Socorro, task, P1)

Product:
Infrastructure & Operations Graveyard
Component:
WebOps: Socorro
Platform:
x86
macOS
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: bburton, Assigned: bburton)

Details

* shove should be managed by supervisord
* settings.py should be a template managed by Puppet
* secrets should be in Hiera
-> % svn add shove.pp
A         shove.pp
bburton@althalus [05:49:56] [~/code/mozilla/sysadmins/puppet/trunk/modules/webapp/manifests]
-> % svn diff
Index: shove.pp
===================================================================
--- shove.pp	(revision 0)
+++ shove.pp	(revision 0)
@@ -0,0 +1,13 @@
+class webapp::shove {
+
+  package {
+    'python-captain-shove':
+      ensure => present;
+  }
+
+  supervisord::service {
+    'shove':
+      command => "/usr/bin/shove",
+      user => 'root';
+  }
+}
Status: NEW → ASSIGNED
Testing out

Index: manifests/nodes/captain.pp
===================================================================
--- manifests/nodes/captain.pp	(revision 75245)
+++ manifests/nodes/captain.pp	(working copy)
@@ -1,5 +1,6 @@
 node /^captainadm.private.scl3.mozilla.com$/ {
     include webapp::admin::captain
+    include webapp::shove

 }
bburton@althalus [05:50:24] [~/code/mozilla/sysadmins/puppet/trunk]
-> % svn ci -m "initial shove daemon manifest, testing out supervisord bits on captainadm.p.scl3, bug 917768"
Sending        trunk/manifests/nodes/captain.pp
Adding         trunk/modules/webapp/manifests/shove.pp
Transmitting file data ..
Committed revision 75248.

bburton@althalus [06:02:21] [~/code/mozilla/sysadmins/puppet/trunk/modules/webapp/manifests]
-> % svn ci -m "initial shove daemon manifest, testing out supervisord bits on captainadm.p.scl3, bug 917768"
Sending        manifests/shove.pp
Transmitting file data .
Committed revision 75249.
-> % svn diff
Index: modules/webapp/manifests/shove.pp
===================================================================
--- modules/webapp/manifests/shove.pp	(revision 75249)
+++ modules/webapp/manifests/shove.pp	(working copy)
@@ -1,4 +1,6 @@
-class webapp::shove {
+class webapp::shove(
+  $project_list,
+  ) {

   package {
     'python-captain-shove':
@@ -9,6 +11,17 @@
     'shove':
       command => "/usr/bin/shove",
       app_dir => '/tmp/',
+      environ => 'SHOVE_SETTINGS_FILE="/etc/shove/settings.py"',
       user    => 'root';
   }
+
+  file {
+
+    '/etc/shove':
+      ensure => 'directory';
+
+    "/etc/shove/settings.py":
+    content => template("webapp/shove/settings.py.erb")
+
+  }
 }
Index: modules/webapp/templates/shove/settings.py.erb
===================================================================
--- modules/webapp/templates/shove/settings.py.erb	(revision 0)
+++ modules/webapp/templates/shove/settings.py.erb	(revision 0)
@@ -0,0 +1,33 @@
+##################################
+# This file is managed by Puppet #
+##################################
+
+# This file configures the behavior of the shove daemon.
+import os
+
+# Change this to point to the root directory under which your projects are stored to make
+# generating file paths a little easier.
+#ROOT = os.path.dirname(__file__)
+ROOT= <%= @web_root %>
+
+def path(*args):
+    """Utility for making file paths relative to ROOT."""
+    return os.path.join(ROOT, *args)
+
+# RabbitMQ connection settings.
+RABBITMQ_HOST = '<%= @rabbitmq_host %>'
+RABBITMQ_PORT = '<%= @rabbitmq_port %>'
+RABBITMQ_VHOST = '<%= @rabbitmq_vhost %>'
+RABBITMQ_USER = '<%= @rabbitmq_user %>'
+RABBITMQ_PASS = '<%= @rabbitmq_pass %>'
+QUEUE_NAME = '<%= @rabbitmq_queue %>'
+
+# Map of projects that shove can run commands for. Keys are IDs that captain needs to know about,
+# values are the path to the repo root directory. Shove expects to find a file under
+# `bin/commands.procfile` in each directory here.
+PROJECTS = {
+    #'test_project': path('tests', 'test_project'),
+    <% @projects.sort.each do |project_name, v| %>
+        '<%= project_name %>': path('<%= v['code_directory'] %>', '<%= v['Procfile] %>'),
+    <% end %>
+}
Index: manifests/nodes/captain.pp
===================================================================
--- manifests/nodes/captain.pp	(revision 75249)
+++ manifests/nodes/captain.pp	(working copy)
@@ -1,7 +1,11 @@
 node /^captainadm.private.scl3.mozilla.com$/ {
-    include webapp::admin::captain
-    include webapp::shove
+  include webapp::admin::captain
+  include webapp::shove

+  $projects_list =  {
+    project1 => { 'code_directory' => '/test/path1', 'Procfile' => 'deploy.py' },
+    project2 => { 'code_directory' => '/test/path2', 'Procfile' => 'deploy.py' },
+  }
 }

 node "captain1.dev.webapp.scl3.mozilla.com" {
bburton@althalus [07:18:40] [~/code/mozilla/sysadmins/puppet/trunk]
-> % svn ci -m "testing shove settings template, bug 917768"
Sending        trunk/manifests/nodes/captain.pp
Sending        trunk/modules/webapp/manifests/shove.pp
Adding         trunk/modules/webapp/templates/shove
Adding         trunk/modules/webapp/templates/shove/settings.py.erb
Transmitting file data ...
Committed revision 75254.
-> % svn diff
Index: modules/webapp/manifests/shove.pp
===================================================================
--- modules/webapp/manifests/shove.pp	(revision 75254)
+++ modules/webapp/manifests/shove.pp	(working copy)
@@ -1,5 +1,5 @@
 class webapp::shove(
-  $project_list,
+  $projects_list,
   ) {

   package {
Index: modules/webapp/templates/shove/settings.py.erb
===================================================================
--- modules/webapp/templates/shove/settings.py.erb	(revision 75254)
+++ modules/webapp/templates/shove/settings.py.erb	(working copy)
@@ -27,7 +27,7 @@
 # `bin/commands.procfile` in each directory here.
 PROJECTS = {
     #'test_project': path('tests', 'test_project'),
-    <% @projects.sort.each do |project_name, v| %>
+    <% @projects_list.sort.each do |project_name, v| %>
         '<%= project_name %>': path('<%= v['code_directory'] %>', '<%= v['Procfile] %>'),
     <% end %>
 }
bburton@althalus [08:48:06] [~/code/mozilla/sysadmins/puppet/trunk]
-> % svn ci -m "testing shove settings template, fixing some naming, bug 917768"
Sending        trunk/modules/webapp/manifests/shove.pp
Sending        trunk/modules/webapp/templates/shove/settings.py.erb
Transmitting file data ..
Committed revision 75258.
-> % svn diff
Index: manifests/nodes/captain.pp
===================================================================
--- manifests/nodes/captain.pp	(revision 75254)
+++ manifests/nodes/captain.pp	(working copy)
@@ -1,11 +1,15 @@
 node /^captainadm.private.scl3.mozilla.com$/ {
   include webapp::admin::captain
-  include webapp::shove

   $projects_list =  {
     project1 => { 'code_directory' => '/test/path1', 'Procfile' => 'deploy.py' },
     project2 => { 'code_directory' => '/test/path2', 'Procfile' => 'deploy.py' },
   }
+
+  class {
+    'webapp::shove':
+      projects_list => $projects_list;
+  }
 }

 node "captain1.dev.webapp.scl3.mozilla.com" {
bburton@althalus [09:08:55] [~/code/mozilla/sysadmins/puppet/trunk]
-> % svn ci -m "testing shove settings template, fixing some naming, bug 917768"
Sending        trunk/manifests/nodes/captain.pp
Transmitting file data .
Committed revision 75261.
bburton@althalus [04:30:41] [~/code/mozilla/sysadmins/puppet/trunk]
-> % svn ci -m "secrets for captain shove rabbitmq connections, bug 917768" hiera/secrets/site.yaml
Sending        hiera/secrets/site.yaml
Transmitting file data .
Committed revision 75338.

-> % svn ci -m "adding rabbitmq configs to shove.pp and new manifest with captain cluster configs, bug 917768"
Sending        trunk/manifests/nodes/captain.pp
Adding         trunk/modules/webapp/manifests/shove
Adding         trunk/modules/webapp/manifests/shove/captain.pp
Sending        trunk/modules/webapp/manifests/shove.pp
Transmitting file data ...
Committed revision 75339.
Done and working nicely! \o/
Status: ASSIGNED → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.