650060 - new master VM, dev-master01

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P5)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

We're scrapping any pretense that we do staging.  And staging-master.bmo is old and crufty.  So!

We need a new master VM named dev-master01.build, based on the existing centos5 builder image.

I'd like to proceed as follows:

1. I will create the VM and provide the MAC address
2. Amy will add that to DHCP, DNS, Inventory, and Nagios
  a. Nagios should probably only run the ping check, since it's a dev box
3. I'll spin up the VM and configure it (hopefully using puppet!)
4. I'll drain, shutdown, and delete staging-master
5. Amy will remove staging-master from DHCP, DNS, Inventory, and Nagios.

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

From IRC, there's no need to re-create /rpmbuild on the new master.  We should be able to do that sort of thing on a slave (and much more quickly).

Whether we need to back up /rpmbuild remains to be seen.

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

There's at least some NDK stuff bear needs in /rpmbuild, and we should keep the server alive at least until ben's back, in case he need anything.  After that, we can back up that partition and kill the VM.

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

Amy has set up the new VM on the KVM/ganetti cluster in scl1.  It's only got 1G of RAM for now, as the cluster is now full, but we can bump that up later.  We're now on to step 3, and hopefully bug 607179 will be finished soon and we can use that to set up this system and to add any future masters.

Comment 4

[Amy Rich [:arr] [:arich]]

After a kvm lesson with Ben, I've installed a small memory print centos 5.5
image on the kvm cluster in scl1.  It's in DNS and DHCP, conforming to the new
DNS scheme we've all discussed and agreed upon.  I've also added the ping check
to nagios.

I would highly recommend running the same nagios checks that get run in
production, though.  We still want to know if the disk is filling up, we're
running out of swap, etc.

Off to you for configuration, Dustin.

Comment 5

[Dustin J. Mitchell [:dustin] (he/him)]

This is configured and currently being used by catlee to test buildbot performance.  Catlee, re-assigning to you to finish up testing, re-image and puppetize (might as well..), and then figure out how to migrate development masters here.  The last part is a bit over my head as a design question, but I'll be happy to help with any legwork.

Comment 6

[Chris AtLee [:catlee]]

Over to dustin to enable pam_ldap

Comment 7

[Dustin J. Mitchell [:dustin] (he/him)]

The user accounts will be set up in bug 660020, actually.

Back to catlee for "...and then figure out how to migrate development masters here.  The last part is a bit over my head as a design question, but I'll be happy to help with any legwork."

Comment 8

[Chris AtLee [:catlee]]

On the backburner until we get user accounts set up so folks can move their masters over/set up new masters.

Comment 9

[Amy Rich [:arr] [:arich]]

LDAP accounts have been set up for a while, so I'll send this over to release for further work if necessary.