1264620 - monitor hgweb11-14

Status: RESOLVED FIXED

(Developer Services :: Mercurial: hg.mozilla.org, defect)

Description

[Kendall Libby [:fubar] (he/him)]

add new hgweb nodes to nagios

Comment 1

[Kendall Libby [:fubar] (he/him)]

added standard monitoring for blades, hgweb nodes. :gps, which of these will eventually be running kafka/zookeeper?

Comment 2

[Gregory Szorc [:gps]]

They will all eventually be running kafka/zookeeper. I plan to light those up once these hosts are in service for real - possibly later today. I suppose you could enable the checks but ack them for the time being.

The plan is to have kafka/zookeeper running on hgweb[11-14] and the master server (currently hgssh3). So we'll drop kafka/zookeeper on hgssh2 and hgweb1, 9, and 10 once hgweb[11-14] are online.

Comment 3

[Kendall Libby [:fubar] (he/him)]

ok. if this happens while I'm out, someone just needs to add the hg-kafka and hg-zookeeper hostgroups to hgweb11-14 in /puppet/trunk/modules/nagios/manifests/hosts/scl3.pp, and remove from the others...

Comment 4

[Ludovic Hirlimann [:Usul]]

[ludo@Oulanl hosts]$ svn diff
Index: scl3.pp
===================================================================
--- scl3.pp	(revision 117025)
+++ scl3.pp	(working copy)
@@ -7887,6 +7887,8 @@
                 'hp-servers',
                 'generic',
                 'zeus-scl3-http',
+		'hg-kafka',
+		'hg-zookeeper',
                 'hg-web',
             ]
         },
[ludo@Oulanl hosts]$ svn commit -m "https://bugzilla.mozilla.org/show_bug.cgi?id=1264620#c3"
Sending        scl3.pp
Transmitting file data .done
Committing transaction...
Committed revision 117026.
[ludo@Oulanl hosts]$

Comment 5

[Gregory Szorc [:gps]]

To be explicit, the nagios actions we want to change are:

* Add hg-kafka and hg-zookeeper to hgweb11-14
* Remove hg-kafka and hg-zookeeper from hgweb{1,9,10} and hgssh2

Comment 6

[Ludovic Hirlimann [:Usul]]

[ludo@Oulanl hosts]$ svn diff
Index: scl3.pp
===================================================================
--- scl3.pp	(revision 117026)
+++ scl3.pp	(working copy)
@@ -7752,8 +7752,6 @@
                 'generic',
                 'zeus-scl3-http',
                 'hg-web',
-                'hg-kafka',
-                'hg-zookeeper',
                     ]
                 },
         'hgweb2.dmz.scl3.mozilla.com' => {
@@ -7834,8 +7832,6 @@
                 'generic',
                 'zeus-scl3-http',
                 'hg-web',
-                'hg-kafka',
-                'hg-zookeeper',
             ]
         },
         'hgweb10.dmz.scl3.mozilla.com' => {
@@ -7846,8 +7842,6 @@
                 'generic',
                 'zeus-scl3-http',
                 'hg-web',
-                'hg-kafka',
-                'hg-zookeeper',
             ]
         },
         'hgweb11.dmz.scl3.mozilla.com' => {
@@ -7857,6 +7851,8 @@
                 'hp-servers',
                 'generic',
                 'zeus-scl3-http',
+		'hg-kafka',
+		'hg-zookeeper',
                 'hg-web',
             ]
         },
@@ -7867,6 +7863,8 @@
                 'hp-servers',
                 'generic',
                 'zeus-scl3-http',
+		'hg-kafka',
+		'hg-zookeeper',
                 'hg-web',
             ]
         },
@@ -7877,6 +7875,8 @@
                 'hp-servers',
                 'generic',
                 'zeus-scl3-http',
+		'hg-kafka',
+		'hg-zookeeper',
                 'hg-web',
             ]
         },
@@ -7910,8 +7910,6 @@
                 'hp-servers',
                 'generic',
                 'hg-ssh',
-                'hg-kafka',
-                'hg-zookeeper',
             ]
         },
         'pvtbuilds2.dmz.scl3.mozilla.com' => {
[ludo@Oulanl hosts]$ svn commit -m" helping out fubar/gps on bug 1264620"
Sending        scl3.pp
Transmitting file data .done
Committing transaction...
Committed revision 117027.

Comment 7

[Gregory Szorc [:gps]]

I /think/ these hosts are fully monitored in nagios now.

fubar: please confirm and close out the bug.

Comment 8

[Kendall Libby [:fubar] (he/him)]

all good