1278563 - iLO issues on hgweb blades

Status: RESOLVED FIXED

(Infrastructure & Operations :: DCOps, task)

Description

[Kendall Libby [:fubar] (he/him)]

+++ This bug was initially created as a clone of Bug #1277840 +++

while checking hgweb nodes to make sure iLo textcons worked (cf bug 1277870), I found a few issues that I need someone to fix:

hgweb11 and 12 report 'iLO Advanced License required for this functionality'
hgweb13 hangs when running 'connect server 1' from the chassis.
hgweb14 is missing it's name in 'show server names' and also reports missing license.

it'd also be great if the set up docs for new blades ensured that textcons worked before handoff. :-D

thanks!

Comment 1

[Peter Radcliffe [:pir]]

Sadly we don't have an automated way to make sure advanced licenses get installed.
If you come across a machine that doesn't have one installed and need it you can grab them here and install them via the web interface or the cli:

https://mana.mozilla.org/wiki/display/SYSADMIN/HP+iLO+licenses

Comment 2

[Kendall Libby [:fubar] (he/him)]

I understand it not being something one can automate (ugh), but if that's the case I would argue even more strongly that it should be part of the initial blade setup and kickstart to ensure that it's working.

in any case, this needs to be done by someone sooner than I will otherwise be able to get to it (i.e. mid-June)

Comment 3

[Peter Radcliffe [:pir]]

If you got to the point of discovering that it doesn't work them it's one more command to install the licenses.

I've installed advanced licenses on hgweb11 and hgweb12.

hgweb13 will probably need to be hard reset (e-fuse) from those symptoms, so will need to be shut down first.

server names can be edited, though I believe they usually come from the OS.

Do you have setup docs for new blades, dcops?

Comment 4

[Kendall Libby [:fubar] (he/him)]

(In reply to Peter Radcliffe [:pir] from comment #3)
> If you got to the point of discovering that it doesn't work them it's one
> more command to install the licenses.

Sure, but I'm under a time crunch today to get a bunch of more-time-important things done before 6 hours of meetings and then disappearing on PTO. Which is why I threw it over the wall to a group that knew how to do it better/faster and could more easily find the time to do it, without worrying about breaking something even further.

> I've installed advanced licenses on hgweb11 and hgweb12.

Thanks! 

> hgweb13 will probably need to be hard reset (e-fuse) from those symptoms, so
> will need to be shut down first.

If someone has time to do this, just coordinate with :gps. If not, I'll deal after London.

Comment 5

[Van Le [:van]]

>Do you have setup docs for new blades, dcops?

no set up docs but i know how to install iLO keys and set up the hostname. i believe this should fall under the MOC check list for the following reasons however:

1)dcops doesn't always know the hostname prior to passing bug and the request sometimes requires MOC to work with the requester.
2)MOC installs the OS through iLO anyway, adding the license and configuring hostname is just an extra copy and paste.
3)can be done remotely, no reason for dcops to circle back and log on to iLO to configure hostname which can cause confusion/annoyance if things change last minute. can also be easily forgotten as the bug is no longer in our queue.
4)MOC is last to validate host(s) before passing to developers.

let me know what you think.

Comment 6

[Van Le [:van]]

>hgweb13 will probably need to be hard reset (e-fuse) from those symptoms, so will need to be shut down first.

i can work with :gps on this server

Comment 7

[Amy Rich [:arr] [:arich]]

(In reply to Van Le [:van] from comment #5)

I think this sounds reasonable. Linda, is this part of the existing proceedure? If not, can we get that amended so that at hand off time we have a functioning, labeled serial console?

Comment 8

[Van Le [:van]]

iLO advanced license  has been installed.

User:OAtmp-mozillaadmin-57586 logged-in to ILO2M261204G1.(10.22.2.128 / FE80::7210:6FFF:FE47:3FB4)

iLO Standard Blade Edition 2.40 at  Dec 02 2015
Server Name: host is unnamed
Server Power: On


</>hpiLO-> oemhp_licenseinstall /map1/oemhp_license1 XXXX

status=0
status_tag=COMMAND COMPLETED
Wed Jun  8 18:36:39 2016

New license key installed.


</>hpiLO-> show /map1/oemhp_license1

status=0
status_tag=COMMAND COMPLETED
Wed Jun  8 18:36:44 2016



/map1/oemhp_license1
  Targets
  Properties
    oemhp_name1=iLO Advanced
    oemhp_key1=xxxxxxxxxxxxxxxxxxxxTNHB2
    oemhp_name2=None
    oemhp_key2=0
    oemhp_name3=None
    oemhp_key3=0
  Verbs
    cd version exit show oemhp_licenseinstall

Comment 9

[Linda Ypulong [:unixfairy]]

(In reply to Amy Rich [:arr] [:arich] (PTO) from comment #7)
> (In reply to Van Le [:van] from comment #5)
> 
> I think this sounds reasonable. Linda, is this part of the existing
> proceedure? If not, can we get that amended so that at hand off time we have
> a functioning, labeled serial console?

Amy we are working on a larger process to make sure we have console access for all supported devices and this will include during handoff

We are working this organically Q3 and it if we cannot get easy resolution then it will be a formal deliverable in Q4