Closed Bug 847947 Opened 11 years ago Closed 11 years ago

IPMI unreachable

Categories

(Infrastructure & Operations :: DCOps, task)

x86
macOS
task
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: arich, Assigned: vinh)

References

Details

IPMI is unreachable on the following machines in build.scl1.mozilla.com that need to be reinstalled.  Please power cycle the boxes until IPMI stays up:

bld-linux64-ix-027
bld-linux64-ix-028
bld-linux64-ix-031
bld-linux64-ix-032
bld-linux64-ix-038
bld-linux64-ix-039
bld-linux64-ix-040
bld-linux64-ix-043
bld-linux64-ix-044
bld-linux64-ix-046
colo-trip: --- → scl1
Blocks: 847529
Hosts were power cycled and IPMI is now reachable.


$ for i in `cat pingtest`; do sudo fping $i-mgmt.build.scl1.mozilla.com; done
bld-linux64-ix-027-mgmt.build.scl1.mozilla.com is alive
bld-linux64-ix-028-mgmt.build.scl1.mozilla.com is alive
bld-linux64-ix-031-mgmt.build.scl1.mozilla.com is alive
bld-linux64-ix-032-mgmt.build.scl1.mozilla.com is alive
bld-linux64-ix-038-mgmt.build.scl1.mozilla.com is alive
bld-linux64-ix-039-mgmt.build.scl1.mozilla.com is alive
bld-linux64-ix-040-mgmt.build.scl1.mozilla.com is alive
bld-linux64-ix-043-mgmt.build.scl1.mozilla.com is alive
bld-linux64-ix-044-mgmt.build.scl1.mozilla.com is alive
bld-linux64-ix-046-mgmt.build.scl1.mozilla.com is alive
Assignee: server-ops-dcops → vhua
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
I found another batch that aren't responding:

bld-linux64-ix-001
bld-linux64-ix-003
bld-linux64-ix-007
bld-linux64-ix-008

and bld-linux64-ix-027 went offline again.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Additional hosts were power cycled and IPMI is now reachable.

bld-linux64-ix-027-mgmt.build.scl1.mozilla.com is alive
bld-linux64-ix-001-mgmt.build.scl1.mozilla.com is alive
bld-linux64-ix-003-mgmt.build.scl1.mozilla.com is alive
bld-linux64-ix-007-mgmt.build.scl1.mozilla.com is alive
bld-linux64-ix-008-mgmt.build.scl1.mozilla.com is alive
Status: REOPENED → RESOLVED
Closed: 11 years ago11 years ago
Resolution: --- → FIXED
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.