Closed Bug 920667 Opened 11 years ago Closed 11 years ago

New host group with in Nagios

Categories

(mozilla.org Graveyard :: Server Operations, task)

Product:
mozilla.org Graveyard
Component:
Server Operations
Platform:
x86_64
Windows 7
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: markco, Assigned: ashish)

References

Details

We are going to be transitioning to different OS build for Windows 2008R2. However, this transition will be incremental. I was wondering if we could create an addition host group to windows 2008R2 64 bit build hosts (w64-ix-slaves) that will not monitor the E disk? If the host group can be create could the following machines be added: w64-ix-slave110 w64-ix-slave111 w64-ix-slave112 w64-ix-slave113 w64-ix-slave114 w64-ix-slave115 w64-ix-slave116 w64-ix-slave117 w64-ix-slave118 w64-ix-slave119 Thank you.
Blocks: 781277
w64-ix-slave10 will need to be added to this list.
I'm moving this bug to the IT triage queue (product mozilla.org, component Server Operations) so that the appropriate team can take action. I apologize for the one week delay in recognizing that it was misfiled; this happens a few times a week and once in a while a bug is missed in the noise. Please use the IT triage queue when filing these requests - I understand it's virtually impossible to discover organically, but it ensures that requests are handled in a timely manner. 'mozilla.org :: Server Operations' works in the enter_bug.cgi quicksearch, and the exact match should be first in the list.
Assignee: infra → server-ops
Component: Infrastructure: Monitoring → Server Operations
Product: Infrastructure & Operations → mozilla.org
QA Contact: jdow → shyam
For the lack of a more descriptive name, I chose w64r2-ix-slaves but would really like something more obvious. FWIW, monitoring on E: for these machines have been removed. Please do drop in with a better name :)
Status: NEW → RESOLVED
Closed: 11 years ago
Flags: needinfo?(mcornmesser)
Resolution: --- → FIXED
As far as the name, I will put some thought into the name. For more obvious are we just talking about to differentiate between the two builds or additional data in the name? Also could you add these machines to the group as well: w64-ix-slave120 w64-ix-slave121 w64-ix-slave122 w64-ix-slave123 w64-ix-slave124 w64-ix-slave125 w64-ix-slave126 w64-ix-slave127 w64-ix-slave128 w64-ix-slave129 w64-ix-slave130 This is going to be an ongoing transition. Should we open additional bugs for each group, or should we keep this one bug open until the transition is complete?
Flags: needinfo?(ashish)
(In reply to Mark Cornmesser [:markco] from comment #4) > As far as the name, I will put some thought into the name. For more obvious > are we just talking about to differentiate between the two builds or > additional data in the name? > Oh this is just in Nagios. I do not touch any other part of the system for releng/ops. > Also could you add these machines to the group as well: > > w64-ix-slave120 > w64-ix-slave121 > w64-ix-slave122 > w64-ix-slave123 > w64-ix-slave124 > w64-ix-slave125 > w64-ix-slave126 > w64-ix-slave127 > w64-ix-slave128 > w64-ix-slave129 > w64-ix-slave130 > Done > This is going to be an ongoing transition. Should we open additional bugs > for each group, or should we keep this one bug open until the transition is > complete? > Let's reuse this bug.
Status: RESOLVED → REOPENED
Flags: needinfo?(ashish)
Resolution: FIXED → ---
ashish: I'm not sure theres a more obvious name to be had. Once we finish converting them all, you can go back to calling them w64-ix if you like (and just remove the E: check form the old group, change the all the hosts back to the old group, and delete the new group). Having two sets is a temporary state while we reimage batches of them.
Flags: needinfo?(mcornmesser)
:arr no problem, I'm not terribly particular especially if this is all temporary.
Assignee: server-ops → ashish
Status: REOPENED → ASSIGNED
Can you move w64-ix-slave30 to the w64r2-ix-slaves host group as well, please?
Flags: needinfo?(ashish)
Done!
Flags: needinfo?(ashish)
Please move these to the w64r2-ix-slaves host group: w64-ix-slave06 w64-ix-slave07 w64-ix-slave08 w64-ix-slave09 w64-ix-slave10 w64-ix-slave30 w64-ix-slave31 w64-ix-slave32 w64-ix-slave33 w64-ix-slave34
as well as these: w64-ix-slave11 w64-ix-slave12 w64-ix-slave13 w64-ix-slave14 w64-ix-slave15 w64-ix-slave35 w64-ix-slave36 w64-ix-slave37 w64-ix-slave38 w64-ix-slave39
Flags: needinfo?(ashish)
Done!
Flags: needinfo?(ashish)
Please move these to the w64r2-ix-slaves host group: w64-ix-slave16 w64-ix-slave17 w64-ix-slave18 w64-ix-slave19 w64-ix-slave20 w64-ix-slave40 w64-ix-slave41 w64-ix-slave44 w64-ix-slave45 w64-ix-slave46
as well as these: w64-ix-slave21 w64-ix-slave23 w64-ix-slave24 w64-ix-slave25 w64-ix-slave26 w64-ix-slave47 w64-ix-slave48 w64-ix-slave49 w64-ix-slave50 w64-ix-slave51
Flags: needinfo?(ashish)
Done!
Flags: needinfo?(ashish)
Please move these to the w64r2-ix-slaves host group: w64-ix-slave27 w64-ix-slave28 w64-ix-slave29 w64-ix-slave42 w64-ix-slave43 w64-ix-slave52 w64-ix-slave53 w64-ix-slave54 w64-ix-slave55 w64-ix-slave56
Flags: needinfo?(ashish)
as well as these: w64-ix-slave75 w64-ix-slave76 w64-ix-slave77 w64-ix-slave78 w64-ix-slave79 w64-ix-slave57 w64-ix-slave58 w64-ix-slave59 w64-ix-slave60 w64-ix-slave61
Done!
Flags: needinfo?(ashish)
Please move these to the w64r2-ix-slaves host group: w64-ix-slave81 w64-ix-slave82 w64-ix-slave83 w64-ix-slave84 w64-ix-slave87 w64-ix-slave62 w64-ix-slave63 w64-ix-slave64 w64-ix-slave65 w64-ix-slave66
Flags: needinfo?(ashish)
these too: w64-ix-slave88 w64-ix-slave89 w64-ix-slave90 w64-ix-slave91 w64-ix-slave92 w64-ix-slave67 w64-ix-slave68 w64-ix-slave69 w64-ix-slave70 w64-ix-slave71
Done!
Flags: needinfo?(ashish)
Please move these to the w64r2-ix-slaves host group: w64-ix-slave72 w64-ix-slave73 w64-ix-slave74
Flags: needinfo?(ashish)
Done!
Flags: needinfo?(ashish)
Please move these to the w64r2-ix-slaves host group: w64-ix-slave04 w64-ix-slave22 w64-ix-slave80
Flags: needinfo?(ashish)
(In reply to John Hopkins (:jhopkins) from comment #24) > Please move these to the w64r2-ix-slaves host group: > > w64-ix-slave04 > w64-ix-slave22 > w64-ix-slave80 - Just saw this come through from watching the component. Figured I would knock it out. Moved these 3 hosts to w64r2-ix-slaves Hostgroup.
Flags: needinfo?(ashish)
Please move this to the w64r2-ix-slaves host group: w64-ix-slave03
These as well: w64-ix-slave93 w64-ix-slave94 w64-ix-slave95 w64-ix-slave96 w64-ix-slave97
Flags: needinfo?(ashish)
Done!
Flags: needinfo?(ashish)
Please move these to the w64r2-ix-slaves host group: w64-ix-slave98 w64-ix-slave99 w64-ix-slave100 w64-ix-slave101 w64-ix-slave102
Flags: needinfo?(ashish)
Also these: w64-ix-slave103 w64-ix-slave104 w64-ix-slave105 w64-ix-slave106 w64-ix-slave107
Done!
Flags: needinfo?(ashish)
Please move these to the w64r2-ix-slaves host group: w64-ix-slave108 w64-ix-slave109 w64-ix-slave131 w64-ix-slave132 w64-ix-slave133
Flags: needinfo?(ashish)
Done!
Flags: needinfo?(ashish)
At this point, I think we should move all of the hosts to one unified group that does not have a check for e: (we never ever see alerts for e: anyway) and go back to calling it by the original name again.
Blocks: 918414
No longer blocks: 781277
I'd slightly prefer to keep drive e: monitoring active on rev1's to keep murphy's law from biting us. But if it greatly simplifies the nagios configs, probably worth the tradeoff since e: rarely/never alerts. Please move these to the w64r2-ix-slaves host group: w64-ix-slave134 w64-ix-slave135 w64-ix-slave136 w64-ix-slave137 w64-ix-slave138
Flags: needinfo?(ashish)
Alright I'm gonna keep things as they are. slave[134-138] added to w64r2-ix-slaves.
Flags: needinfo?(ashish)
Hi Ashish, can you please add the new rev2 nagios checks for these machines? Currently they have the old Win64 rev1 checks. Should we change the default nagios checks for Win64 machines to be the ones for rev2 machines? Thanks for your help! w64-ix-slave159.build.scl1.mozilla.com w64-ix-slave160.build.scl1.mozilla.com w64-ix-slave161.build.scl1.mozilla.com w64-ix-slave162.build.scl1.mozilla.com w64-ix-slave163.build.scl1.mozilla.com w64-ix-slave164.build.scl1.mozilla.com w64-ix-slave165.build.scl1.mozilla.com w64-ix-slave166.build.scl1.mozilla.com w64-ix-slave167.build.scl1.mozilla.com w64-ix-slave168.build.scl1.mozilla.com w64-ix-slave169.build.scl1.mozilla.com w64-ix-slave170.build.scl1.mozilla.com
Flags: needinfo?(ashish)
w64-ix-slave[159-170] added to w64r2-ix-slaves hostgroup. :armenzg There are still a handful of slaves that are still in the old hostgroup, w64-ix-slave[139-157] for instance. rev2 in Nagios only removes the E: check and yes, that can be set as default for all. I'm on PTO for the reminder of this week and can make significant changes on Monday (Sunday evening Pacific). However, others in SRE should be able to add hosts to w64r2-ix-slaves on short notice.
Flags: needinfo?(ashish)
Please move these to the w64r2-ix-slaves host group: w64-ix-slave139 w64-ix-slave140 w64-ix-slave141 w64-ix-slave142 w64-ix-slave143
Flags: needinfo?(ashish)
Done!
Flags: needinfo?(ashish)
Amy said earlier: At this point, I think we should move all of the hosts to one unified group that does not have a check for e: (we never ever see alerts for e: anyway) and go back to calling it by the original name again. I'm reimaging the last of the rev1 w64-ix-slaves so I think we should go ahead with that plan whenever is convenient.
One last request in comment 40, Ashish. Thanks!
Flags: needinfo?(ashish)
I'll clean this up and remove the w64r2-ix-slaves hostgroup and exclude the E: disk check
Completed in sysadmins r84631. I've retained the w64-ix-slaves hostgroup and removed w64r2-ix-slaves and the E: disk check.
Status: ASSIGNED → RESOLVED
Closed: 11 years ago → 11 years ago
Flags: needinfo?(ashish)
Resolution: --- → FIXED
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.