1017479 - HP Health on node6.bagheera.metrics.scl3.mozilla.com is UNKNOWN: UNKNOWN - hanging hpasmdcli processes

Status: RESOLVED FIXED

(Infrastructure & Operations :: DCOps, task)

Description

[MOC Nagios API]

Automated alert report from nagios1.private.scl3.mozilla.com:

Hostname: node6.bagheera.metrics.scl3.mozilla.com
Service:  HP Health
State:    UNKNOWN
Output:   UNKNOWN - hanging hpasmdcli processes

Runbook:  http://m.allizom.org/HP+Health

Comment 2

[Ludovic Hirlimann [:Usul]]

  Using Proliant Standard
 	IPMI based System Health Monitor
  Starting Proliant Standard
 	IPMI based System Health Monitor (hpasmlited): 
hpasmlited: Not able to initialize HP iLO Management Controller: Device or resource busy

Means we probably need to reboot the host , tmary would that be ok ?

Comment 3

[T [:tmary] Meyarivan]

Looks like the host (intentional ?) rebooted..

No longer able to SSH into this host (stuck just after "Last login:....")

--

Comment 4

[Ludovic Hirlimann [:Usul]]

(In reply to T [:tmary] Meyarivan from comment #3)
> Looks like the host (intentional ?) rebooted..
> 
> No longer able to SSH into this host (stuck just after "Last login:....")

I got to the console - missed the opportunity to login (had an old password file). Then lost the screen on the virtual console. Tried to reset the box but that failed.

waiting 10 minutes before trying again.

Comment 5

[Ludovic Hirlimann [:Usul]]

After reboot I was able to ssh to it again.