421421 - Tell nagios to forget about mothballed machines

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P1)

Description

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

The following machines are currently monitored by nagios. Can you please have nagios stop monitoring:

bl-bldlnx03
moz180-win32-tbox
tb180-win32-tbox
xserve04

...before we power-down these machines?

Comment 1

[Reed Loden [:reed]]

This is all in CVS, so IT doesn't have to be involved here at all, as the files are on a cronjob to update. See the files in the tools/tinderbox-configs/monitoring/ directory under /cvsroot.

Write a patch yourself to remove them? :)

Comment 2

[Dave Miller [:justdave]]

(In reply to comment #1)
> This is all in CVS, so IT doesn't have to be involved here at all, as the files
> are on a cronjob to update. See the files in the
> tools/tinderbox-configs/monitoring/ directory under /cvsroot.

Umm... no.  He's talking about the machine monitoring, not the tinderbox status monitoring.  That's all in the actual nagios config on bm-admin01, not in the public CVS.

Comment 3

[Reed Loden [:reed]]

(In reply to comment #2)
> Umm... no.  He's talking about the machine monitoring, not the tinderbox status
> monitoring.  That's all in the actual nagios config on bm-admin01, not in the
> public CVS.

Ah, n/m then. My apologies.

Comment 4

[Jeremy Orem [:oremj]]

Don't see the following in the nagios configs:

bl-bldlnx03
moz180-win32-tbox
tb180-win32-tbox

I removed: 

host:bm-xserve04.build:bm-admin01:build

Comment 5

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Thanks Dave, Jeremy; perfect!

Comment 6

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

I just powered off bl-bldlnx03, and got the following nagios alert?!? Where in nagios is this coming from?


-------- Original Message --------
Subject: ** PROBLEM alert - surf/tree - Firefox is CRITICAL **
Date: Mon, 17 Mar 2008 20:48:56 -0700 (PDT)
From: [e-mail] (nagios)
To: [e-mail]

***** Nagios  *****
Notification Type: PROBLEM
Service: tree - Firefox
Host: surf
Address: 10.2.74.116
State: CRITICAL
Date/Time: 03-17-2008 20:48:56
Additional Info:
CRITICAL: missing bl-bldlnx03 - perf test

Comment 7

[Dave Miller [:justdave]]

[root@surf tinderbox-monitoring]# grep -n bl-bldlnx03 *
Tier1_Firefox.txt:2:bl-bldlnx03|perf test|fx-linux-tbox
[root@surf tinderbox-monitoring]# cvs stat Tier1_Firefox.txt 
===================================================================
File: Tier1_Firefox.txt	Status: Up-to-date

   Working revision:	1.12
   Repository revision:	1.12	/cvsroot/mozilla/tools/tinderbox-configs/monitoring/Tier1_Firefox.txt,v

That's on your end.

Comment 8

[Dave Miller [:justdave]]

so reed was part-right in comment 1, except it was both. :)

Comment 9

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Attachment: remove mothballed machines from Tier1_Firefox.txt

Remove the mothballed machines from Tier1 page, to stop nagios paging about them. This is in addition to the work already done telling Nagios to ignore these machines.

Comment 10

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: [as checked in] remove mothballed machines from Tier1_Firefox.txt

I also added the windows and mac debug builds which are also tier 1 on Build:Farm.

Checking in Tier1_Firefox.txt;
/cvsroot/mozilla/tools/tinderbox-configs/monitoring/Tier1_Firefox.txt,v  <--  Tier1_Firefox.txt
new revision: 1.13; previous revision: 1.12
done