704329 - put new releng-dedicated nameserver in mtv1 into production

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: General, task)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

Bug 610304 set up an HA nameserver (really two nameservers on a virtual IP), at 10.250.48.19.

I'd like to start by pointing staging hosts at this IP, and checking that they still get the forward and reverse resolution that they expect.  As with the deletion of the build view today, I fully expect this to work, but I want to be as careful as possible.

Once that's done and we're happy, I'll turn this on for the build network in mtv1.

So, to start, please list some dev/pp hosts that I can adjust.  I'll change their configured resolvers in DHCP, so at their next reboot they'll start using the new nameservers.

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

Ping?

Comment 2

[Chris Cooper [:coop] (he/him)]

I've marked the following dev/pp slaves as using the new NS in slavealloc:

linux64-ix-slave02	
moz2-darwin10-slave01	
moz2-darwin9-slave08	
mv-moz2-linux-ix-slave01	
talos-r3-fed-001	
talos-r3-fed64-001	
talos-r3-leopard-002
talos-r3-w7-003	win7
talos-r4-lion-002	
talos-r4-snow-010	
w32-ix-slave01

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

Only two of those are in mountain view:

moz2-darwin10-slave01
mv-moz2-linux-ix-slave01

but I've changed the DHCP information for them.  The rest should have those notes removed, probably, and if there are others in mtv1 you'd like to test, let me know.  I rebooted the first, which was not doing a job, to verify that the DHCP info was correct.

Comment 4

[Chris Cooper [:coop] (he/him)]

If we can also try it with mw32-ix-slave01 (now marked in slavealloc), I think that will be sufficient for me.

Comment 5

[Dustin J. Mitchell [:dustin] (he/him)]

OK, DHCP config is updated, so we now have three hosts in this test.  Shall we give this a week, and then turn it on for everything in mtv1?

Comment 6

[Chris Cooper [:coop] (he/him)]

(In reply to Dustin J. Mitchell [:dustin] from comment #5)
> OK, DHCP config is updated, so we now have three hosts in this test.  Shall
> we give this a week, and then turn it on for everything in mtv1?

Yes, please.

Comment 7

[Nick Thomas [:nthomas] (UTC+12)]

If I could insert a comment 3.5 here, it would say that I removed the slavealloc comments for all the hosts in comment #2 but not in comment #3.

Comment 8

[Dustin J. Mitchell [:dustin] (he/him)]

Coop, were there any issues with these hosts?  If not, I'd like to enable this globally build.mtv1-wide today.

Comment 9

[Chris Cooper [:coop] (he/him)]

(In reply to Dustin J. Mitchell [:dustin] from comment #8)
> Coop, were there any issues with these hosts?  If not, I'd like to enable
> this globally build.mtv1-wide today.

All three slaves are behaving normally AFAICT. The windows slave had the wrong ssh keys on it for a while, but I've fixed that now.

Comment 10

[Dustin J. Mitchell [:dustin] (he/him)]

OK, turned on for all mtv1 hosts.  The dhcp commit (r25933) is easy to revert if this goes badly.  Bug 712449 tracks taking out mv-buildproxy01.