685389 - relabs06-mgmt broken

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: General, task)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

The iLO accepts and then hangs on ports 443 and 22.  The OS itself is still functional, but I'd like to install Ubuntu on it for bug 685257.

Matt has tried power-cycling by cord-yank, to no avail.

Amy suggests
 http://www.normanbauer.com/2011/03/11/how-to-reset-hp-ilo-integrated-lights-out

Comment 1

[Amy Rich [:arr] [:arich]]

I grabbed the firmware from http://ftp.hp.com/pub/softlib2/software1/doc/p2025081362/v49549/filehandler.html?path=/pub/softlib2/software1/sc-linux-fw-ilo/p1255562964/v64722&files=CP014002.scexe&i=MTX-7d8c347071cd448fbcba30012e

I had to do a yum install glibc.i686

Then I ran the firmware installer:

sh ./CP014002.scexe -f


It's alive again.

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

It died again overnight.  I re-ran the above, and it lived -- briefly, but long enough for me to open the remote console.  Then the web and ssh UI's failed again, although remote console is still working.

For the moment, I'll keep using this as a test system, but it will need some TLC before going into production.

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

OK, well, that didn't last long.  Let's open an HP call on this system.

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

Case ID: 	4634505017
Case title: 	iLO functions only for short periods
Product serial number: 	2M21210359
Product number: 	628691-001
Submitted: 	Sep 19, 2011 12:59:50 PM
Last updated: 	Sep 19, 2011 12:59:50 PM
Source: 	Web
Case status: 	Received by HP

Comment 5

[Dustin J. Mitchell [:dustin] (he/him)]

They recommended I upgrade the FW to 1.26, but pointed me to a windows installer.  The latest I can find for rhel6 is 1.20 (which is what arr has attached in mana)

Comment 6

[Dustin J. Mitchell [:dustin] (he/him)]

I was pointed to:

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc=us&prodTypeId=15351&prodSeriesId=5075933&prodNameId=5075937&swEnvOID=4006&swLang=8&mode=2&taskId=135&swItem=MTX-eab39401868843399011c667cf

But the machine isn't ssh-able anymore, so that doesn't really help me :(

Matt, can you put a monitor on this system and try installing 1.26 from
  https://mana.mozilla.org/wiki/display/SYSADMIN/HP+iLO

Comment 7

[Dustin J. Mitchell [:dustin] (he/him)]

It's back up

Comment 8

[Dustin J. Mitchell [:dustin] (he/him)]

Success!  I'll leave this bug open for a bit to see if it fails again.  I'll comment and close the HP bug when this is solved, and open a new one to upgrade (all) the other HP's.

Comment 9

[Dustin J. Mitchell [:dustin] (he/him)]

This is still running now, so let's call it good.