660099 - install and configure nrpe on foopies

Status: RESOLVED WONTFIX

(Release Engineering :: General, defect)

Description

[Amy Rich [:arr] [:arich]]

Aki, can you please install and configure nrpe on the foopies (or add then to puppet, even better) so we can add additional monitoring?

Comment 1

[Aki Sasaki (not active)]

/me doesn't know puppet and has near-zero interest in learning in the near future.

Depending on the timing for the new manageable linux foopies, I may wontfix by inaction.

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

If these are production systems, they need to be monitored.

Comment 3

[Aki Sasaki (not active)]

new linux foopies: eta to ship, ~2 weeks
get one or two, test load on them, order more
eta for retiring these rev2 mini foopies: ~4-6 weeks maybe.

Definitely more worth installing nrpe now than if we were turning them off in 2 weeks.

Bear has offered to put puppet + nrpe on these while he's setting them up for clientproxy and sut_tools; I may take him up on that, but would like to see what else he's working on next week.

Keeping this open and assigned to me for the moment.

Comment 4

[Amy Rich [:arr] [:arich]]

FYI this bug is also for the existing foopies foopy01 - foopy03 (so I can institute the ganglia checks on them, since it was requested that these be monitored with ganglia).

Comment 5

[Aki Sasaki (not active)]

I'm going to push to retire bm-foopy and foopy01-04 (is 04 not on your radar?) as soon as we have foopy05-foopy11 online and all tegras switched over.

TTL for bm-foopy, foopy01-04: <1-2 weeks hopefully
TTL for foopy05-11: <6 weeks hopefully

It makes more sense to me to put in the work to monitor foopy05-11 than bm-foopy + foopy01-04, but if we get puppet set up sooner we may do everything.

Comment 6

[Amy Rich [:arr] [:arich]]

Ah, foopy04 was being pinged by nagios, but it wasn't requested that ganglia monitor it.  Should ganglia be monitoring it as well?

Comment 7

[Aki Sasaki (not active)]

(In reply to comment #6)
> Ah, foopy04 was being pinged by nagios, but it wasn't requested that ganglia
> monitor it.  Should ganglia be monitoring it as well?

Yes, *but* nrpe isn't on it and Zandr says he may rack foopy05-11 early next week, which means foopy01-04 may not be long for this world.

Comment 8

[Amy Rich [:arr] [:arich]]

Since we're decommissioning the old foopies, do the new foopies have nrpe and ganglia installed and configured?

Comment 9

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

per irc w/aki, bear: this doesnt block android becoming tier1.

Comment 10

[Chris Cooper [:coop] (he/him)]

Not an issue with the newer foopys.