985482 - Move machines from scl1->scl3 to test platforms on new vlans

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: General, task)

Description

[Amy Rich [:arr] [:arich]]

We want to take at least one machine of each type that needs to move out of scl1 that does not already exist in scl3 and make sure that dns, dhcp, and the network flows are all functional for the platform.

b-linux64-hp-001.build.scl1.mozilla.com -> b-linux64-hp-0001.try.releng.scl3.mozilla.com (scl3-releng-vlan264)
bld-linux64-ix-027.build.scl1.mozilla.com -> b-linux64-ix-0001.build.releng.scl3.mozilla.com (scl3-releng-vlan252)
w64-ix-slave03.winbuild.scl1.mozilla.com -> b-2008-ix-0018.wintry.releng.scl3.mozilla.com (scl3-releng-vlan244)
talos-r4-snow-001.build.scl1.mozilla.com -> t-snow-r4-0001.test.releng.scl3.mozilla.com (scl3-releng-vlan256)

callek will handle removing these from the pool and updating buildbot/slavealloc/puppet/etc.

dcops will move the machines and update inventory.

relops will reimage each machine and hand them back to releng for validation.

Comment 1

[Amy Rich [:arr] [:arich]]

Since the w64-ix machine was loaned out, this list is now:

b-linux64-hp-001.build.scl1.mozilla.com -> b-linux64-hp-0001.try.releng.scl3.mozilla.com (scl3-releng-vlan264)
bld-linux64-ix-027.build.scl1.mozilla.com -> b-linux64-ix-0001.build.releng.scl3.mozilla.com (scl3-releng-vlan252)
w64-ix-slave31.winbuild.scl1.mozilla.com -> b-2008-ix-0019.wintry.releng.scl3.mozilla.com (scl3-releng-vlan244)
talos-r4-snow-001.build.scl1.mozilla.com -> t-snow-r4-0001.test.releng.scl3.mozilla.com (scl3-releng-vlan256)

Comment 2

[Ashish Vijayaram [:ashish]]

I've commented out the old hosts in puppet-nagios since they're yanked from DNS which stalls puppet runs.

Comment 3

[Amy Rich [:arr] [:arich]]

all machines have been moved, all but b-2008-ix-0019.wintry.releng.scl3.mozilla.com are ready to test (we're still working out some kinks with the imaging for the wintry vlan).

I'll submit a separate bug to update nagios once we are confident these are working.

Comment 4

[Justin Wood (:Callek)]

STATUS CHECKPOINT:

b-2008-ix-0019.wintry.releng.scl3.mozilla.com - needs:
    https://wiki.mozilla.org/ReleaseEngineering/How_To/Adjust_SSH_keys_on_a_slave

b-linux64-hp-0001.try.releng.scl3.mozilla.com
b-linux64-ix-0001.build.releng.scl3.mozilla.com
t-snow-r4-0001.test.releng.scl3.mozilla.com
   all imaged fine, rebooted into production

Comment 5

[Justin Wood (:Callek)]

(In reply to Justin Wood (:Callek) from comment #4)
> STATUS CHECKPOINT:

> b-linux64-hp-0001.try.releng.scl3.mozilla.com

Looking good: https://secure.pub.build.mozilla.org/builddata/reports/slave_health/slave.html?name=b-linux64-hp-0001

> b-linux64-ix-0001.build.releng.scl3.mozilla.com

Looking good: https://secure.pub.build.mozilla.org/builddata/reports/slave_health/slave.html?name=b-linux64-ix-0001

> t-snow-r4-0001.test.releng.scl3.mozilla.com

Looking good: https://secure.pub.build.mozilla.org/builddata/reports/slave_health/slave.html?name=t-snow-r4-0001

Comment 6

[Justin Wood (:Callek)]

(In reply to Justin Wood (:Callek) from comment #4)
> STATUS CHECKPOINT:
> 
> b-2008-ix-0019.wintry.releng.scl3.mozilla.com - needs:
>    
> https://wiki.mozilla.org/ReleaseEngineering/How_To/Adjust_SSH_keys_on_a_slave

Just enabled in production

Comment 7

[Justin Wood (:Callek)]

And to be clear, the production run on windows has had 2 green (successful) jobs since its deployment today