726783 - decommission/move IRC server in sjc1

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

Sand and Concrete are both in sjc1, and will need to go away.  The last plan I've heard is to create a new IRC server in scl3, turn that on, and remove sand/concrete from the rotation.

If sand and/or concrete become unused, please mark them as such per https://mana.mozilla.org/wiki/display/websites/Killing+sjc1+hosts

Comment 1

[Dumitru Gherman [:dumitru]]

concrete is in PHX.
sand is a VM, so why not just move it?

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

(In reply to Dumitru Gherman [:dumitru] from comment #1)
> concrete is in PHX.

inventory updated accordingly

> sand is a VM, so why not just move it?

works for me.

Comment 3

[Shyam Mani [:fox2mike]]

Yes. We can take sand out of rotation, run with gravel and concrete until sand is brought back up in scl3. Dave, ack? (I think we need to move services away from sand though...since that'll go down with this move).

Comment 4

[Dave Miller [:justdave]]

services is configured on concrete and will operate there if we turn it on.  The config for it will need to be synced back there from sand though, because it's way out of date (it's all in berkley DBs, too, so it needs to be shut down before syncing or they'll corrupt in transit).  The config should still be in the main unrealircd.conf files but commented out, as well, so it'll be a matter of uncomment the concrete services line and comment out the sand one.

We'll also need to re-hub, because sand is operating as a hub for the other two servers right now.  gravel will need to be linked with concrete to avoid a netsplit while sand is down.

I just completely removed sand from the geoip rotation on 3crowd, so anyone newly-connecting from outside of Europe/Africa should get concrete from now on, and we'll eventually vacate sand by attrition.

Comment 5

[Dustin J. Mitchell [:dustin] (he/him)]

We'll have ESX running in scl3 soon -- is sand sufficiently drained that we can start syncing config and make that switch?

Comment 6

[Rob Tucker [:rtucker]]

The ESX in scl3 is complete, can this be migrated or can it just be turned off and left?

Comment 7

[Shyam Mani [:fox2mike]]

There was another bug, justdave already moved irc stuff off sand IIRC. This should be good to move. Let me find the other bug.

Comment 8

[Phong Tran [:phong]]

this is done.