720553 - community vlan (vlan20) and cg-bugs vlan (vlan23) in scl3

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: NetOps, task)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

We'll need a VLAN similar to the current sjc1 VLAN in scl3.  There are a lot of separate organizations to coordinate, so moving the existing VLAN isn't a good option.

IIRC the space in this VLAN was getting tight in sjc1, so I will leave it to your discretion whether to select a larger subnet this time around.

The current VLAN has incoming connections filtered, and a jumphost.  Will netops be able to build a jumphost for the new network when ESX is up in scl3, or is that someone else's responsibility?

Comment 1

[Teemu Mannermaa (:wicked)]

Since this seems to block the Bugzilla server move bug 719715 as well.. Note that we don't have any jumphost. We have unrestricted direct access to all our hosts and also a private backend net that only allows connections between our hosts.

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

wicked, I'm going to review the network flows for this network as they are, and create something similar for the new network, so I'll take care of that.

As for the "private backend net" - is that an overlaid IP network, or are you using a different VLAN for that?

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

I know that we've historically had both community-build and community-giving.  As far as I'm concerned, that distinction need not be reflected in the VLAN layout.  So I think this only needs one VLAN.

Obviously we can't configure this too soon, but it would be great if I could get allocations for this, so that I can start specifying network flows and setting up DNS.  Community-giving has IPv6 set up, and I'd like to replicate that in scl3.  So:

Can I get IPv4 and IPv6 allocations sooner than later?  /25 and /64 are good.

Comment 4

[Ravi Pina [:ravi]]

> Can I get IPv4 and IPv6 allocations sooner than later?  /25 and /64 are good.

Unlikely.  There is a lot of juggling we need to do and we have limited v4 space at the moment.  It may just move in place.  What is the urgency here?

Comment 5

[Dustin J. Mitchell [:dustin] (he/him)]

Well, the urgency on getting the allocation is just to have the flows, DNS, etc. in place before we start moving machines, so not high yet, but I don't want to block on this for too long.

Moving the IP range in place is not a great option.  First of all, based on information from Derek, I've been planning with the affected parties based on being able to create a new network and migrate (and knowing that bridging is out of the question).  There are a bunch of different owners, with varying degrees of sophistication, so a synchronized move of the whole network is likely to be a very long process, meaning extended downtime for all parties.

The 63.245.208.64/26 network appears to be unallocated.  While I realize that it may not be possible to advertise such a long prefix via BGP, could we use some static routes in sjc1 to locate this network in scl3 during the migration, perhaps opening things up to 63.245.208.0/25 after the move?

Comment 6

[Derek Moore [:dmoore]]

I believe we'll have three /24s available for scl3 (before reclaiming anything from sjc1). Give us some time to allocate out of those subnets.

Comment 7

[Teemu Mannermaa (:wicked)]

(In reply to Dustin J. Mitchell [:dustin] from comment #2)
> As for the "private backend net" - is that an overlaid IP network, or are
> you using a different VLAN for that?

A private 192.168.99.0/24 net which is vlan 23 while our main vlan is 21, I believe. It's the net that's discussed in bug 563848, where it was extended to rest of our servers. I guess we can still make use of it since not all our hosts need public IPs (especially some of our own VMs in cg-bugs02).

Comment 8

[Dustin J. Mitchell [:dustin] (he/him)]

(In reply to Teemu Mannermaa (:wicked) from comment #7)
> A private 192.168.99.0/24 net which is vlan 23 while our main vlan is 21, I

Ah, neat - netops, can we do the same in scl3?

Comment 9

[Dustin J. Mitchell [:dustin] (he/him)]

I know this can't get configured yet, but can I get IP allocations so I can get flow bugs filed?

Comment 10

[Dustin J. Mitchell [:dustin] (he/him)]

I'd like to move these hosts on the B train (3/12).  Will this work be done soon enough in advance that I can request and get flows set up comfortably before that train?

Comment 11

[Derek Moore [:dmoore]]

We've freed up 63.245.223.0/24 to be divided for community use.

I'm proposing the following in scl3:

63.245.223.0/26   - Vlan 20 community_build
63.245.223.64/26  - Vlan 20 (reserved for expansion)
63.245.223.128/27 - Vlan 21 community_giving)
63.245.223.160/27 - Vlan 21 (reserved for expansion)
63.245.223.192/26 - unallocated

Comment 12

[Dustin J. Mitchell [:dustin] (he/him)]

Derek, I had proposed using only a single community VLAN (vlan20), so there's no need to set up vlan21 in this case.  There are 30 hosts slated to go in here now (bug 731053), so a /26 with a /26 for expansion is nice and comfortable.  That means

63.245.223.0/26   - Vlan 20 community_build
63.245.223.64/26  - Vlan 20 (reserved for expansion)
63.245.223.128/25 - unallocated

plus vlan23 (unrouted) in comment 8.

As a reminder, we need this soon enough that server ops: infra can get the jumphost up (bug 720833) before the end of the day on Friday.

Comment 13

[Derek Moore [:dmoore]]

Merging the VLANs works for me.

We've started announcing 63.245.223.0. It is plumbed through to VLAN 20 in SCL3.

The unrouted VLAN 23 has also been created.

Comment 14

[Dustin J. Mitchell [:dustin] (he/him)]

Excellent, thanks!  So I'll get bug 731053 (IP allocs) closed out one way or another today.  I think this bug is complete, then?

Comment 15

[Ravi Pina [:ravi]]

I think this bug is complete, but I'll keep open until we sort everything out with the other bugs.

Comment 16

[Dustin J. Mitchell [:dustin] (he/him)]

This is done