800061 - please rack and cable 73 mobile chassis in the scl1 expansion

Status: RESOLVED FIXED

(Infrastructure & Operations :: DCOps, task)

Description

[Amy Rich [:arr] [:arich]]

Please rack and cable 73 mobile chassis in the scl1 expansion.  Will adjust blocker bug when we have one to figure out how these are being assembled.

Comment 1

[Van Le [:van]]

The first 3 racks are complete. Please let me know if there is anything missing from inventory (this should include the pandas, pdu, foopies, imager, switches). 

https://inventory.mozilla.org/en-US/systems/racks/?location=13&status=&rack=248&allocation=
https://inventory.mozilla.org/en-US/systems/racks/?location=13&status=&rack=249&allocation=
https://inventory.mozilla.org/en-US/systems/racks/?location=13&status=&rack=250&allocation=

Comment 2

[Van Le [:van]]

We were able to complete an additional 3 racks this week (Racks 201-[456]).
Please let me know if there is anything missing from inventory (this should include the pandas, pdu, foopies, imager, switches). 

This brings a total of 48/73 panda chassis racked/cabled/inventoried in the new cage. 

Can someone confirm these numbers? We also have 4 more chassis on site so we *should* be expecting 21 more from D.J. Grey.

Thanks,
Van

Comment 3

[Jake Watkins [:dividehex]]

I was at SCL1 yesterday and noticed the chassis in rack 201-6 do not have the top cover correctly installed.  This is pushing on the back panel on the chassis and putting stress on it.  We need to put at least one screw in the rear of the top panel for proper structural integrity of the chassis.  I know that we have been putting in only one screw to secure the top cover for convenient internal access but we really should use more screws.

I would suggest 1 screw in the top rear center and 2 side screws.  One on each side towards the rear.  This should be the bare minimum.

Comment 4

[Jake Watkins [:dividehex]]

Also noticed, some of the chassis aren't powered on.  Is there a reason?  If not, can we get them powered up. Thanks.

Comment 5

[Jake Watkins [:dividehex]]

Can we have panda-relay-042.p5.releng.scl1.mozilla.com looked at?  I cannot ping it.

Comment 6

[Van Le [:van]]

>We need to put at least one screw in the rear of the top panel for proper structural integrity of the chassis.  I know that we have been putting in only one screw to secure the top cover for convenient internal access but we really should use more screws.

The reason this was done is because we are currently on back order for 4' IEC power cables. We only have 3' power cables and the current work around to get them all powered on was to use 1 or 2 screws on the side that we can get to and slide the cover off without having to pull the chassis out all the way. We can swap the power cables once they come in but it's going to be a while before they come in. I'll try to get an update.

Comment 7

[Vinh Hua [:vinh]]

Jake,
Panda-relay-042.p5.releng.scl1.mozilla.com should now be pingable.

vhua$ sudo fping panda-relay-042.p5.releng.scl1.mozilla.com
panda-relay-042.p5.releng.scl1.mozilla.com is alive

Comment 8

[Van Le [:van]]

we received all the panda chassis today. I have sent Amy the inventory spread sheet for another rack of pandas. We have 2 racks (16 chassis) left to install, which should be completed by end of next unless something unexpected comes up.

Van

Comment 9

[Amy Rich [:arr] [:arich]]

We still have one chassis left to be delivered.  DJGrey is waiting on RMAed panda boards.

Comment 10

[Amy Rich [:arr] [:arich]]

panda-relay-061 is not pingable.  Please power cycle it.

Comment 11

[Amy Rich [:arr] [:arich]]

Just checking in to see if you guys can get the info for p8 and p9 to me by EOB Thursday so I can get all of the info imported into inventory before I got on PTO for the rest of the year.  If not, let me know, and I can make other arrangements.

Comment 12

[Van Le [:van]]

panda-relay-061 has a bad Digi ME Ethernet module. 

:dividehex, do you have any spares? 

:arr, we definitely can.

Comment 13

[Van Le [:van]]

73 chassis (72@scl1, 1@scl3) have been racked, cabled, uboot image installed and inventoried.

Remaining issues:

-pending 1 chassis to be delivered by DJGrey with Panda boards that have been RMA'd (any ETA?)
-panda-relay-061 has a bad Digi Ethernet module (waiting for Jake's response)
-panda-0662 isn't booting up and has no serial output (we have a spare panda we pulled from chassis 6, let me know if we should swap)

Comment 14

[Jake Watkins [:dividehex]]

:van, contact NCD and see if you can get a replacement ME module without sending back the entire board.  It will be time consuming to replace the entire board due to the wiring.  If they need the entire board for a RMA, then just order a new ME module.

Comment 15

[Van Le [:van]]

:dividehex, I tried googling NCD and I'm not finding the company. Will ordering a module straight from Digi work? I wasn't sure if the modules sold on the Digi site had the correct pin outs.

Comment 16

[Jake Watkins [:dividehex]]

:van,

http://www.controlanything.com/Relay/Relay/CONTACT

Do not order it from Digi directly.  Try to RMA it first through NCD otherwise order a new one from NCD.

Comment 17

[Van Le [:van]]

Unfortunately they claim since they don't manufacture the Digi ethernet module, it's not covered under their warranty. Ordering a replacement module in bug 821419.

Comment 18

[Melissa O'Connor [:melissa]]

Where are we at for racking the remaining chassis?

Comment 19

[Van Le [:van]]

The remaining chassis has not been delivered. I just checked with the SCL1 NOC. 

Van

Comment 20

[Amy Rich [:arr] [:arich]]

What's the status on panda-relay-061?  I saw that the new ethernet module has been delivered, but it doesn't look like it's online yet.

Comment 21

[Amy Rich [:arr] [:arich]]

Have we received the last chassis from DJ Grey at this point (I see the last update on that was 2012-12-17)?

Comment 22

[Amy Rich [:arr] [:arich]]

panda-relay-061 is now up and configured.

Comment 23

[Van Le [:van]]

We received the last chassis from DJ Grey on Friday and we racked, cabled, inventoried it today. The csv has been sent to Amy to batchload.

Van

Comment 24

[Amy Rich [:arr] [:arich]]

Information has been entered for the last chassis.  Please power cycle it so that the relay comes online.

Comment 25

[Vinh Hua [:vinh]]

Panda chassis 079 has been power cycled.

sudo fping panda-relay-079.p10.releng.scl1.mozilla.com
panda-relay-079.p10.releng.scl1.mozilla.com is alive

Comment 26

[Amy Rich [:arr] [:arich]]

The last chassis has been configured, installed with android, and been added to nagios.