1241645 - replace iX chassis

Status: RESOLVED FIXED

(Infrastructure & Operations :: DCOps, task)

Description

[Van Le [:van]]

currently one of the slots in an iX chassis is bad. we'll need to downtime 3 hosts that are in the same chassis to swap it out. please give me about 2 hours to do the physical swapping.

13722 - t-w732-ix-161.wintest.releng.scl3.mozilla.com
13771 - t-w732-ix-162.wintest.releng.scl3.mozilla.com
13769 - t-w732-ix-160.wintest.releng.scl3.mozilla.com

Comment 1

[Van Le [:van]]

bug in place for tracking. will sync with relops for downtime.

Comment 2

[Van Le [:van]]

i'd like to schedule this at 11am PT 2/11/2016 with 2 hours of downtime to unrack the chassis, replace blades and recable. 

if the above time doesn't bode well, i'm available at the data center M-F 9am-6pm although i prefer not to schedule things too early in the morning due to california traffic.

Comment 3

[Kim Moir [:kmoir] ET]

I'll graceful the machines when I arrive at 9am ET/6am PT.

Comment 4

[hwine]

Van: you're clear to go. Please let us know when the machines are back online so we can restore them to service!

Comment 5

[Kim Moir [:kmoir] ET]

machines have been disabled in slavealloc and notes indicate bug number

Comment 6

[Van Le [:van]]

chassis swap complete, hosts are back online.

mozillas-MacBook-Air-2:~ vle$ fping t-w732-ix-16{0..2}.wintest.releng.scl3.mozilla.com
t-w732-ix-160.wintest.releng.scl3.mozilla.com is alive
t-w732-ix-161.wintest.releng.scl3.mozilla.com is alive
t-w732-ix-162.wintest.releng.scl3.mozilla.com is alive