Please rack and cable 50 ix chassis for linux and linux64 testing

RESOLVED FIXED

Status

Infrastructure & Operations
DCOps
RESOLVED FIXED
5 years ago
4 years ago

People

(Reporter: arr, Unassigned)

Tracking

Details

(Reporter)

Description

5 years ago
We have 50 new iX chassis arriving at scl3 in the next couple weeks.  There should be stub entries for all hosts in DNS and inventory already. Please make sure that IPMI is configured and all hosts are set to boot from disk by default.

These hosts should be racked/cabled/VLANed as follows:

VLAN 240:

t-w864-ix-099.wintest.releng.scl3.mozilla.com
t-w864-ix-100.wintest.releng.scl3.mozilla.com


VLAN 256:

talos-linux32-ix-002.test.releng.scl3.mozilla.com
..
talos-linux32-ix-100.test.releng.scl3.mozilla.com


talos-linux64-ix-002.test.releng.scl3.mozilla.com
..
talos-linux64-ix-100.test.releng.scl3.mozilla.com


VLAN 216:

t-w864-ix-099-mgmt.inband.releng.scl3.mozilla.com
t-w864-ix-100-mgmt.inband.releng.scl3.mozilla.com

talos-linux32-ix-002-mgmt.inband.releng.scl3.mozilla.com
..
talos-linux32-ix-100-mgmt.inband.releng.scl3.mozilla.com


talos-linux64-ix-002-mgmt.inband.releng.scl3.mozilla.com
..
talos-linux64-ix-100-mgmt.inband.releng.scl3.mozilla.com
(Reporter)

Updated

5 years ago
Summary: Please rack and cable 50 ix nodes for linux and linux64 testing → Please rack and cable 50 ix chassis for linux and linux64 testing

Updated

5 years ago
colo-trip: --- → scl3
9 of these systems will arrive on 3/8, the remainder will arrive on 3/15.

If we can get the first 9 racked and ready to go quickly that would be great. :)

Comment 2

5 years ago
The first 10 iX systems has been racked, cabled and IPMI configured. 

t-w864-ix-099-mgmt.inband.releng.scl3.mozilla.com is alive
t-w864-ix-100-mgmt.inband.releng.scl3.mozilla.com is alive
talos-linux64-ix-002-mgmt.inband.releng.scl3.mozilla.com is alive
..
talos-linux64-ix-039-mgmt.inband.releng.scl3.mozilla.com is alive

Comment 3

5 years ago
switch ports tagged.

vle@switch1.r101-19.console.scl3# show | compare 
[edit interfaces interface-range win8]
-    member-range ge-0/0/0 to ge-0/0/36;
+    member-range ge-0/0/0 to ge-0/0/15;
-    member-range ge-1/0/0 to ge-1/0/36;
+    member-range ge-1/0/0 to ge-1/0/13;
[edit interfaces]
+   interface-range linux64-ix {
+       member-range ge-0/0/16 to ge-0/0/36;
+       member-range ge-1/0/14 to ge-1/0/36;
+       unit 0 {
+           family ethernet-switching {
+               port-mode access;
+               vlan {
+                   members test;
+               }
+           }
+       }
+   }
[edit vlans]
+   test {
+       vlan-id 256;
+   }
(Reporter)

Updated

5 years ago
Blocks: 850502
(Reporter)

Comment 4

5 years ago
t-w864-ix-100.wintest.releng.scl3.mozilla.com is not getting a DHCP lease.  Is there a cable or VLAN problem with it?

Comment 5

5 years ago
Bad cable issue with t-w864-ix-100.wintest.releng.scl3.mozilla.com resolved.
Network hardware for the next three racks arrived today:

========

The EX4200s you ordered on PO # mrz-BCT-490-win8 have been shipped, see shipping information below:
FEDX STD OVR
752259237280
752259237279
752259237268
752259237257
752259237246
752259237290

They were delivered today, March 19, 2013.

The EX2200s and Rack Mounts on PO # mrz-BCT-489-win8 were also shipped, see shipping information below:
FEDX STD OVR
959690088260
959690088293
959690088308
959690088271
959690088282
959690088341
959690088330
959690088320
959690088319
959690088385
959690088374
959690088363
959690088352

These were also delivered today, March 19, 2013.

One item is currently backordered - EX-PWR-320-AC. We will send updated tracking information as soon as we have it for this item.
(Reporter)

Updated

5 years ago
Blocks: 852680

Updated

5 years ago
Depends on: 853249
(Reporter)

Comment 7

5 years ago
Of the ones just installed, the problem children are:

talos-linux64-ix-044 - no IPMI
talos-linux64-ix-057 - WAY SLOW during the kickstart, power cycled, and got blinking cursor
talos-linux64-ix-060 - Reboot and Select proper Boot device
talos-linux64-ix-081 - no DHCP
talos-linux32-ix-008 - no IPMI
(Reporter)

Comment 8

5 years ago
All of these are working now except talos-linux64-ix-057 for which I opened bug 854597 to track that.
(Reporter)

Comment 9

5 years ago
The last batch should be in Friday or Monday.
DCOps how many arrived?

Comment 11

5 years ago
10 arrived Monday, 13 remaining.
(Reporter)

Comment 12

5 years ago
As I sent in email, there should only be three chassis remaining based on the number of systems I already have info for.


I am unable to get to talos-linux32-ix-022-mgmt.inband.releng.scl3.mozilla.com

talos-linux32-ix-036-mgmt.inband.releng.scl3.mozilla.com had the wrong netmask (which I have fixed)

Comment 13

5 years ago
My mistake, it looks like we've received 47 hosts so far.

10 ~ March 8
27 ~ March 25
10 ~ April 8

Pending a delivery of the remaining 3 chassis/12 hosts for
talos-linux32-ix-[088-100].test.releng.scl3.mozilla.com.


Let me know if there are any discrepancies.

Comment 14

5 years ago
IPMI has been configured for the 10 chassis (40 nodes) we received on Monday.

talos-linux32-ix-049-mgmt.inband.releng.scl3.mozilla.com is alive
...
talos-linux32-ix-088-mgmt.inband.releng.scl3.mozilla.com is alive
(Reporter)

Comment 15

5 years ago
Problem hosts:

talos-linux32-ix-022-mgmt.inband.releng.scl3.mozilla.com - unpingable
talos-linux32-ix-085-mgmt.inband.releng.scl3.mozilla.com - can't get DHCP lease
talos-linux32-ix-086-mgmt.inband.releng.scl3.mozilla.com - can't get DHCP lease
The last 3 of these should have been delivered Friday.
The last 3 are in, we do not expect DCOps to rack them until next week.

Comment 18

5 years ago
>talos-linux32-ix-022-mgmt.inband.releng.scl3.mozilla.com - unpingable
reconfigured.
talos-linux32-ix-022-mgmt.inband.releng.scl3.mozilla.com is alive

>talos-linux32-ix-085-mgmt.inband.releng.scl3.mozilla.com - can't get DHCP lease
>talos-linux32-ix-086-mgmt.inband.releng.scl3.mozilla.com - can't get DHCP lease
these 2 nodes are peculiar. eth0 is being seen as eth1 by the BIOS. Changed BIOS setting to PXE boot off eth1 instead of eth0. Kept cable in port that's supposed to be eth0 and updated inventory with new MAC. Let me know if there are any issues. They're both sitting at the OS option screen after PXE booting.

Comment 19

5 years ago
>The last 3 are in, we do not expect DCOps to rack them until next week.

We decided to kick ass and get them all racked/cabled/inventoried anyways.

talos-linux32-ix-089-mgmt.inband.releng.scl3.mozilla.com is alive
...
talos-linux32-ix-100-mgmt.inband.releng.scl3.mozilla.com is alive

Please open individual bugs for any problematic hosts. These 50 iX servers are racked and cabled.

Updated

5 years ago
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
(Reporter)

Comment 20

5 years ago
You guys need to give me the csv for the rest of the machines.  They won't work until I put them in inventory with the k/v store info.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Comment 21

5 years ago
csv sent to amy. please resolve bug if everything is copacetic.
(Reporter)

Updated

5 years ago
Status: REOPENED → RESOLVED
Last Resolved: 5 years ago5 years ago
Resolution: --- → FIXED
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.