863263 - Create VMs to replace kvm signing servers

Status: RESOLVED FIXED

(Infrastructure & Operations :: Virtualization, task)

Description

[Amy Rich [:arr] [:arich]]

The following signing servers need to move off of kvm in scl1 and scl3 and onto vmware in scl3.

signing1.build.scl1.mozilla.com
signing2.build.scl1.mozilla.com
signing3.srv.releng.scl3.mozilla.com

bhearsum: do we have the capability to cross-datacenter signing since we'll still have a number of slaves in scl1?  I thought I remembered someone saying that we didn't do that right now.  Also, is three sufficient capacity, and are there signing servers anywhere else for redundancy?

Comment 1

[[e-mail] (:bhearsum)]

(In reply to Amy Rich [:arich] [:arr] from comment #0)
> The following signing servers need to move off of kvm in scl1 and scl3 and
> onto vmware in scl3.
> 
> signing1.build.scl1.mozilla.com
> signing2.build.scl1.mozilla.com
> signing3.srv.releng.scl3.mozilla.com
> 
> bhearsum: do we have the capability to cross-datacenter signing since we'll
> still have a number of slaves in scl1?  I thought I remembered someone
> saying that we didn't do that right now.  Also, is three sufficient
> capacity, and are there signing servers anywhere else for redundancy?

We avoid it where ever possible right now to reduce the amount of cross-colo bandwidth. It shouldn't be a problem to do cross colo signing though as long as the links are fast enough and hold up to the extra traffic.

Comment 2

[Amy Rich [:arr] [:arich]]

After an issue with signing3 today, we talked a bit more about this, and we're already doing cross datacenter signing from AWS.  So Ben doesn't think this is a technical limitation, just one of perceived performance.  Since the only builders left in scl1 are for windows, I don't think we'd be seeing that much difference in performance by putting the servers in scl3.  I'm going to get the ball rolling here and request 3 new VMWare vms with the following specs:

2 CPU
300G of disk (combined) for / and /boot
4G RAM (and 4G swap)
VLAN 248 in SCL3
CentOS 6.2 (we'll be hooking these up to releng puppet, so please do NOT puppetize with infra puppet)

Hostnames:

signing4.srv.releng.scl3.mozilla.com
signing5.srv.releng.scl3.mozilla.com
signing6.srv.releng.scl3.mozilla.com

Comment 3

[Amy Rich [:arr] [:arich]]

Note that these are disk intensive, so SAS over SATA if possible.

Comment 4

[Greg Cox [:gcox]]

DNS was sitting there already (yay).  Cloned out 3 copies, bumped CPU and RAM and disk, logged MAC in inventory, gave them their hostnames, updated vmware tools.

Did nothing with puppet, per :arr, and did not yum update.  Verified root is remotely available via a password in the gpg file.