expose private Bugzilla vlan to the switches to which the cg-bugs* servers are connected

RESOLVED FIXED

Status

Infrastructure & Operations
NetOps
--
enhancement
RESOLVED FIXED
8 years ago
4 years ago

People

(Reporter: justdave, Assigned: dmoore)

Tracking

Details

Need the private Bugzilla project VLAN on cg-vmware01 exposed to the switches the cg-bugs* servers are connected to, and let me know the resulting vlan ID.  All three of the new physical servers need to be able to connect to it.

Thanks!
Derek says we'll need a short downtime for the existing hosts on this VLAN within ESX, because he can't expose the internal VLAN, only create a new one that has external access and then move them to it.  Should be fairly brief though.

Comment 2

8 years ago
That's fine. As far as my hosts on that system go, you can take them down any time. I think reed also has a host on that ESX server, and so does somebody else, but I forget who at the moment.
(Assignee)

Comment 3

8 years ago
Scheduled for 05/13
Whiteboard: [sched. 05/13]

Updated

8 years ago
Severity: normal → enhancement
Whiteboard: [sched. 05/13]

Updated

7 years ago
Component: Server Operations → Server Operations: Netops
Just an overview of what needs to happen in case anyone else gets to this:

There is an existing private VLAN on a virtual switch entirely internal to cg-vmware01 which has several VMs on it.  We need to connect three physical hosts to this VLAN (listed as cg-bugs01/02/03 in inventory).

In order to do this, we need to:
1) Create a new VLAN on the physical switches
2) Put the switch ports for cg-bugs02 and cg-bugs03 onto that vlan
3) Allow cg-bugs01 to have access to that vlan, but not as its default (it should keep its current default).
4) Allow cg-vmware01 to have access to the vlan on its switch ports.
5) Create a VLAN choice in cg-vmware01's UI to correspond to this VLAN
6) Move the existing hosts connected to the private Bugzilla VLAN on cg-vmware01 to the new VLAN.

Comment 5

7 years ago
Create Vlan23.

interface GigabitEthernet2/9
 description cg-vmware01:N1
 switchport
 switchport trunk encapsulation dot1q
 switchport trunk native vlan 21
 switchport trunk allowed vlan 6,21,23,60
 switchport mode trunk
 no ip address
end

All the VMs are on this Vlan too.

Comment 6

7 years ago
> 2) Put the switch ports for cg-bugs02 and cg-bugs03 onto that vlan

Done  -

interface GigabitEthernet2/6
 description cg-bugs02
 switchport
 switchport access vlan 23
 spanning-tree portfast
end

> 3) Allow cg-bugs01 to have access to that vlan, but not as its default (it
> should keep its current default).

Done - 

interface GigabitEthernet2/5
 description cg-bugs01
 switchport
 switchport trunk native vlan 21
 switchport trunk allowed vlan 21,23
 switchport mode trunk
 spanning-tree portfast
end
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED
The virtual switch created for VLAN 23 in ESX does not appear to be working.  None of the 4 hosts connected to it can see each other.  I've been playing around with it a little already and can't figure out what's wrong with it.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Comment 8

7 years ago
Do I have a login to troubleshoot on any of those hosts?
The oracle box is logged into console as root on nm-ops02, and landfill.bugzilla.org has the standard Ops keyring on it on the root account.
I just kickstarted cg-bugs02 with RHEL6.  Had a fun adventure trying to do it.  After switching its switch ports to vlan6 for the kickstart, it kept dying citing timeouts trying to download random things (sometimes repo info, sometimes packages). If I switched to the shell terminal and starting pinging the router before the connection dropped, then it maintained the connection all the way through the kickstart.  After the kickstart completed, I switched it back to vlan23, and assigned a static IP of 192.168.99.30.  Seems to be up and running at this point.  I brought up the vlan23 interface on cg-bugs01 at 192.168.99.25.  They can ping each other, but neither of them can see landfill or any of the other hosts that are VMs.

Comment 11

7 years ago
wicked is going to be the point person on the Bugzilla side for contacting about this.
OK, this is done.  We apparently had disabled interfaces on both landfill and cg-bugs02.  Enabled those (had to reboot cg-bugs02 to get its back), and everything seems to be working now.  cg-bugs03 kickstart/deployment will be handled on bug 554499.

Best guess on cg-bug02 is NetworkManager was being too smart for its own good and manually disabled the interfaces when it couldn't ping the default gateway (because landfill was the default gateway).
Status: REOPENED → RESOLVED
Last Resolved: 7 years ago7 years ago
Resolution: --- → FIXED
For the record, the dropping interfaces turned out to be https://bugzilla.redhat.com/show_bug.cgi?id=647077

Workaround is to add pcie_aspm=off to the kernel command line.
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.