Puppet freshness nagios alerts

RESOLVED FIXED

Status

Infrastructure & Operations
RelOps: Puppet
--
major
RESOLVED FIXED
9 months ago
9 months ago

People

(Reporter: nthomas, Assigned: arr)

Tracking

Details

(Reporter)

Description

9 months ago
These have persisted for a 2 to 3 days, oldest last:

releng-puppet1.srv.releng.scl3.mozilla.com   Puppet freshness   Agent lockfile /var/lib/puppet/state/agent_catalog_run.lock 189993 seconds old 
releng-puppet1.srv.releng.use1.mozilla.com   Puppet freshness   Agent lockfile /var/lib/puppet/state/agent_catalog_run.lock 276872 seconds old 
releng-puppet1.srv.releng.usw2.mozilla.com   Puppet freshness   Agent lockfile /var/lib/puppet/state/agent_catalog_run.lock 276484 seconds old 
releng-puppet2.srv.releng.scl3.mozilla.com   Puppet freshness   Agent lockfile /var/lib/puppet/state/agent_catalog_run.lock 297342 seconds old
(Assignee)

Comment 1

9 months ago
The following host had wedged puppet runs, so I killed them off and ran puppet manually:
releng-puppet1.srv.releng.use1.mozilla.com
releng-puppet1.srv.releng.usw2.mozilla.com
releng-puppet2.srv.releng.scl3.mozilla.com

The issue started 10d 17h 59m 43s ago according to nagios, so no puppet changes had been picked up by those puppet masters since then. This may result in some unexpected behavior since any hosts talking to those puppet masters would have been 10 days behind in changes.
Assignee: relops → arich
Status: NEW → RESOLVED
Last Resolved: 9 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.