997226 - prep new nagios information for scl1->scl3 move

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Amy Rich [:arr] [:arich]]

As part of each scl1->scl3 move day, we'll need to modify nagios so that it starts using the new dns information for each host we move. Right now the move trains are mostly solidified, but we expect some changes here and there as we fine tune things.

Old hostname -> new hostname translations can be found in 5 spreadsheets (note that each has a number of tabs):

Move Train A: https://docs.google.com/a/mozilla.com/spreadsheets/d/1QuB69Aor4YK9TWeBKWRtgGF8i_PZfXmL8uAnneqaw74
Move Train B: https://docs.google.com/a/mozilla.com/spreadsheets/d/1O38M2fbvDR95PTW1I7q829OUW5YCQbcFUsC24ePRAKY
Move Train C: https://docs.google.com/a/mozilla.com/spreadsheets/d/1Pz0wSIhht1xlVLRe5Tl676iLyNAQtVqwKA5Z3rSkpmE
Move Train D: https://docs.google.com/a/mozilla.com/spreadsheets/d/1CSbCNtuY6pZMHtAs6KmOUgPholm4zMsE9SssWhZ0TfI

Panda infrastructure (these will also be split out into move trains at a later date, but will be moved as entire racks, either in sets of 2 or 3): https://docs.google.com/a/mozilla.com/spreadsheets/d/1Z_1fGFD3lg53YiRlIFti_GV6RGOgKRzlPoepNBVYloU

Move train dates will be tracked at: https://mana.mozilla.org/wiki/display/DC/SCL1+Decommission

Comment 1

[Ashish Vijayaram [:ashish]]

The only prerequisite is that the new hosts should be in DNS (and a couple random ones I picked indeed were). There are only a handful that are migrated in each train, so I don't see this taking up a whole lot of effort. I'll be around on the dates for all the trains and could move the affected hosts before/at the beginning of each train.

Keeping this bug unassigned for easy escalations.

Comment 2

[Adrian J Fernandez [:Aj]]

"Train A" ready and active in nagios.

Quick note on extract info; export sheets as cvs then;
echo 'sed 's/,/ /g' "$FILE" | awk '{print $1,$2,$10}' | sed s'/\:.*.//g''

then either manually update... or script it!

Comment 3

[Adrian J Fernandez [:Aj]]

As per hwine, the "new" Train A nodes (94) have been downtimed in releng nagios for 2days.

Comment 4

[Amy Rich [:arr] [:arich]]

(In reply to Adrian J Fernandez [:Aj] from comment #2)

Different sheets (system types) will have different numbers of fields, so just want to make sure that you're capturing the parents accurately for all types of machines.

Comment 5

[Adrian J Fernandez [:Aj]]

sysadmins/moc/scripts/dc-move.sh was created to streamline this process (separate e-mail was sent out).
script has information on usage etc.

Closing this bug as the "updating" will be done via the individual train move Bugs:  1018302, 1018303