lost IPMI connections during power outage

RESOLVED FIXED

Status

Infrastructure & Operations
DCOps
RESOLVED FIXED
5 years ago
4 years ago

People

(Reporter: arr, Unassigned)

Tracking

Details

(Reporter)

Description

5 years ago
We lost the ipmi connections to the ix machines during the power outage.  I was able to get back the ones on the linux machines with ipmitoo, but the windows machines will need to be manually reset.

t-w864-ix-098-mgmt.inband.releng.scl3.mozilla.com
t-w864-ix-099-mgmt.inband.releng.scl3.mozilla.com
t-w864-ix-100-mgmt.inband.releng.scl3.mozilla.com

Updated

5 years ago
colo-trip: --- → scl3

Comment 1

5 years ago
These too:
ix-mn-w732-001-mgmt.inband.releng.scl3.mozilla.com
ix-mn-w732-002-mgmt.inband.releng.scl3.mozilla.com
ix-mn-w732-003-mgmt.inband.releng.scl3.mozilla.com

ix-mn-xp-003-mgmt.inband.releng.scl3.mozilla.com

Comment 2

5 years ago
All hosts have been rebooted and IPMI verified working.

ix-mn-w732-001-mgmt.inband.releng.scl3.mozilla.com address not found
t-w864-ix-098-mgmt.inband.releng.scl3.mozilla.com is alive
t-w864-ix-099-mgmt.inband.releng.scl3.mozilla.com is alive
t-w864-ix-100-mgmt.inband.releng.scl3.mozilla.com is alive
ix-mn-w732-002-mgmt.inband.releng.scl3.mozilla.com is alive
ix-mn-w732-003-mgmt.inband.releng.scl3.mozilla.com is alive
ix-mn-xp-003-mgmt.inband.releng.scl3.mozilla.com is alive


"ix-mn-w732-001-mgmt.inband.releng.scl3.mozilla.com address not found"  <---- wrong DNS name.  But it's pingable by IP address of 10.26.16.11

$ host 10.26.16.11
11.16.26.10.in-addr.arpa domain name pointer ix-mn-w832-001-mgmt.inband.releng.scl3.mozilla.com.

Comment 3

5 years ago
DNS updated to match:

vhua$ host 10.26.16.11
11.16.26.10.in-addr.arpa domain name pointer ix-mn-w732-001-mgmt.inband.releng.scl3.mozilla.com.
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.