1011488 - Create 4 more linux64 tests masters

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Chris AtLee [:catlee]]

Adding more masters will help spread the load around a bit.

Let's add 2 more in each of use1 and usw2.

Comment 1

[Rail Aliiev [:rail]]

Attachment: tools

disabled for now

Comment 2

[Rail Aliiev [:rail]]

Attachment: puppet

Comment 3

[Rail Aliiev [:rail]]

Comment on attachment 8423858 [details] [diff] [review]
tools

https://hg.mozilla.org/build/tools/rev/63b5e2229f1c

Comment 4

[Rail Aliiev [:rail]]

Comment on attachment 8423862 [details] [diff] [review]
puppet

remote:   https://hg.mozilla.org/build/puppet/rev/de3ee7eb29d9
remote:   https://hg.mozilla.org/build/puppet/rev/d1a97727a4fd

Comment 5

[Rail Aliiev [:rail]]

Added to DNS:

invtool A create --ip 10.134.49.144 --fqdn buildbot-master113.srv.releng.use1.mozilla.com --private  --description "Releng convention"
invtool PTR create --ip 10.134.49.144 --target buildbot-master113.srv.releng.use1.mozilla.com --private --description "Releng convention"
invtool CNAME create --fqdn buildbot-master113.build.mozilla.org --target buildbot-master113.srv.releng.use1.mozilla.com --private --description "Releng convention"

invtool A create --ip 10.134.48.122 --fqdn buildbot-master114.srv.releng.use1.mozilla.com --private  --description "Releng convention"
invtool PTR create --ip 10.134.48.122 --target buildbot-master114.srv.releng.use1.mozilla.com --private --description "Releng convention"
invtool CNAME create --fqdn buildbot-master114.build.mozilla.org --target buildbot-master114.srv.releng.use1.mozilla.com --private --description "Releng convention"

invtool A create --ip 10.132.49.29 --fqdn buildbot-master115.srv.releng.usw2.mozilla.com --private  --description "Releng convention"
invtool PTR create --ip 10.132.49.29 --target buildbot-master115.srv.releng.usw2.mozilla.com --private --description "Releng convention"
invtool CNAME create --fqdn buildbot-master115.build.mozilla.org --target buildbot-master115.srv.releng.usw2.mozilla.com --private --description "Releng convention"

invtool A create --ip 10.132.50.189 --fqdn buildbot-master116.srv.releng.usw2.mozilla.com --private  --description "Releng convention"
invtool PTR create --ip 10.132.50.189 --target buildbot-master116.srv.releng.usw2.mozilla.com --private --description "Releng convention"
invtool CNAME create --fqdn buildbot-master116.build.mozilla.org --target buildbot-master116.srv.releng.usw2.mozilla.com --private --description "Releng convention"

Comment 6

[Rail Aliiev [:rail]]

Added the masters to the "Systems" section of Inventory

Comment 7

[Rail Aliiev [:rail]]

imported into slavealloc (disabled by default):

nickname,fqdn,http_port,pb_port,datacenter,pool
bm113-tests1-linux64,buildbot-master113.srv.releng.use1.mozilla.com,8201,9201,aws-us-east-1,tests-use1-linux64
bm114-tests1-linux64,buildbot-master114.srv.releng.use1.mozilla.com,8201,9201,aws-us-east-1,tests-use1-linux64
bm115-tests1-linux64,buildbot-master115.srv.releng.usw2.mozilla.com,8201,9201,aws-us-west-2,tests-usw2-linux64
bm116-tests1-linux64,buildbot-master116.srv.releng.usw2.mozilla.com,8201,9201,aws-us-west-2,tests-usw2-linux64

Comment 8

[Rail Aliiev [:rail]]

ran aws_create_instance.py. masters are up now, waiting for netflows.

make checkconfig passes

Once the netflows are in place, I'm going to enable the masters in production-masters.json file, start them and attach some slaves.

Comment 9

[Armen [:armenzg]]

I filed bug 1011523 and bug 1011537 as per https://wiki.mozilla.org/ReleaseEngineering/AWS_Master_Setup#IT

Comment 10

[Rail Aliiev [:rail]]

Attachment: puppet fix

oops

Comment 11

[Rail Aliiev [:rail]]

Comment on attachment 8425439 [details] [diff] [review]
puppet fix

r+ on irc
remote:   https://hg.mozilla.org/build/puppet/rev/7d8e46046d3f
remote:   https://hg.mozilla.org/build/puppet/rev/e26a615e09e6

Comment 12

[Rail Aliiev [:rail]]

Deleted the following files:
 /etc/default/buildbot.d/bm51-tests1-linux64
 /etc/cron.d/bm51-tests1-linux64

Comment 13

[Rail Aliiev [:rail]]

Attachment: [tools] enable masters

Once the masters pass some minimal testing I'm going to enable them in tools and slavealloc

Comment 14

[Rail Aliiev [:rail]]

Comment on attachment 8425470 [details] [diff] [review]
[tools] enable masters

Seen ~10 green jobs, enabling in slavealloc and tools
https://hg.mozilla.org/build/tools/rev/beb5b6ac5099