tons of UNREACHABLE :NRPE: Unable to read output errors

RESOLVED FIXED

Status

--
critical
RESOLVED FIXED
6 years ago
4 years ago

People

(Reporter: bhearsum, Assigned: atoll)

Tracking

Details

(Reporter)

Description

6 years ago
14:47 < nagios-releng> Mon 11:46:36 PST [416] builddata.pub.build.mozilla.org is UNREACHABLE :NRPE: Unable to read output
14:47 < nagios-releng> Mon 11:46:36 PST [418] pxe1.build.scl1.mozilla.com is UNREACHABLE :NRPE: Unable to read output
14:47 < nagios-releng> Mon 11:46:36 PST [419] slavealloc.pvt.build.mozilla.org is UNREACHABLE :NRPE: Unable to read output
14:47 < nagios-releng> Mon 11:46:36 PST [421] ns1.infra.scl1.mozilla.com is UNREACHABLE :NRPE: Unable to read output
14:47 < nagios-releng> Mon 11:46:36 PST [423] admin1.infra.scl1.mozilla.com is UNREACHABLE :NRPE: Unable to read output
14:47 < nagios-releng> Mon 11:46:36 PST [424] rabbit2.build.scl1.mozilla.com is UNREACHABLE :NRPE: Unable to read output
14:47 < nagios-releng> Mon 11:46:36 PST [425] kms1.ad.mozilla.com is UNREACHABLE :NRPE: Unable to read output
14:47 < nagios-releng> Mon 11:46:46 PST [427] relengweb1.dmz.scl3.mozilla.com is DOWN :NRPE: Unable to read output
14:47 < nagios-releng> Mon 11:47:06 PST [428] buildbot-master31.srv.releng.scl3.mozilla.com is DOWN :NRPE: Unable to read output
(Reporter)

Comment 1

6 years ago
Callek suggested that this might be related to bug 810827.
Assignee: server-ops-releng → server-ops
Component: Server Operations: RelEng → Server Operations

Updated

6 years ago
Assignee: server-ops → rsoderberg
moving the class include for "base::puppetclient" from node-info.pl (the external nodes script) to manifests/site.pp, prior to the $::lib definition used by nrpe checks, resulted in a broken $::lib definition.

that change has been reverted and should propagate to the clients over the next while.

it's not clear why this occurred; perhaps ordering is relevant in site.pp with puppet 2.7. we're going to migrate $::lib to a fact anyways, which should prevent this from occurring in the future.
Status: NEW → ASSIGNED
Please don't change the default QA on ANY server-ops bugs. If you move bugs between components, please make sure that the reset QA assignee box is checked.  Thanks!
QA Contact: release → shyam
No further occurrences of the NRPE: Unable issue since 12:45pm PST (-0800).
Status: ASSIGNED → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.