1202745 - puppent agent hung on buildbot-master83

Status: RESOLVED WONTFIX

(Release Engineering :: General, defect)

Description

[[e-mail] (:bhearsum)]

Noticed this today:
root      1628  0.0  0.0 117324  1044 ?        Ss   Aug29   0:06 crond
root     26180  0.0  0.0 140208  1012 ?        S    Aug29   0:00  \_ CROND
root     26182  0.0  0.0 106092  1012 ?        Ss   Aug29   0:00      \_ /bin/sh -c . /usr/local/bin/proxy_reset_env.sh && /usr/bin/puppet agent --enable >/dev/null 2>/dev/null && /usr/bin/puppet agent --onetime --no-daemonize --logdest=syslog >/dev/null 2>&1
root     26197  0.0  5.4 226408 104220 ?       S    Aug29   0:10          \_ /usr/bin/ruby /usr/bin/puppet agent --onetime --no-daemonize --logdest=syslog

I think it started during the last TCW. I killed the leaf process and it seems to have shut down:
Sep  8 10:31:02 buildbot-master83.bb.releng.scl3.mozilla.com puppet-agent[29804]: Enabling Puppet.
Sep  8 10:31:03 buildbot-master83.bb.releng.scl3.mozilla.com puppet-agent[29809]: Run of Puppet configuration client already in progress; skipping  (/var/lib/puppet/state/agent_catalog_run.lock exists)
Sep  8 10:32:58 buildbot-master83.bb.releng.scl3.mozilla.com puppet-agent[26197]: Caught TERM; calling stop


I ran the agent by hand and it synced up without issue. Not sure if it's worth tracking the hang, so maybe this should just be closed.

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

I dream of a world where every TCP connection has keepalives enabled.  Likely whatever puppet was hitting went away, and it just sat around waiting for a packet.