1038299 - asterisk - manager on pbx1.voip.mtv2.mozilla.com is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds.

Status: RESOLVED FIXED

(Infrastructure & Operations :: Telecom, task)

Description

[MOC Nagios API]

Automated alert report from nagios1.private.scl3.mozilla.com:

Hostname: pbx1.voip.mtv2.mozilla.com
Service:  asterisk - manager
State:    CRITICAL
Output:   CHECK_NRPE: Socket timeout after 30 seconds.

Runbook:  http://m.allizom.org/asterisk+-+manager

Comment 1

[Rick Bryce [:rbryce]]

Timeout has persisted for the past 30 minutes.  This is the only error I see in the asterisk console.

[Jul 14 18:53:48] WARNING[9136]: chan_iax2.c:3509 __attempt_transmit: Max retries exceeded to host 10.248.40.4 on IAX2/dundiexternal-1672 (type = 6, subclass = 11, ts=2269944, seqno=97)

Escalating to netops

Comment 2

[Rick Bryce [:rbryce]]

This doesn't appear to be affecting users at the moment

Comment 3

[Dave Miller [:justdave]]

(In reply to Rick Bryce [:rbryce] from comment #2)
> This doesn't appear to be affecting users at the moment

The system was pretty much dead by the time I got involved in the troubleshooting, we had to do a full restart.

See also bug 1038328

Comment 4

[Dave Miller [:justdave]]

I just updated the runbook page for this.  In the future this one should probably be escalated immediately.