672844 - Bring buildbot-master2 back to action

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Armen [:armenzg]]

Because talking with ~45 test slaves is always easier than talking to ~57 test slaves.

Having less slaves per master has been better for wait times and I have seen bm{04,06,11} with WAIT CPU recently.

This will be mid-term until we can have more VM masters rather than physical hardware.

Comment 1

[Armen [:armenzg]]

I have locked down several slaves to bm02-tests1 and bm02-tests2 and see how it fairs.

If we can let's enable bm02-tests1 tomorrow morning and measure load for other masters on bug 672851.

Comment 2

[Armen [:armenzg]]

REMINDER: We have to add the master on the json file once we are about to enable it.

Comment 3

[Armen [:armenzg]]

Attachment: enable bm2-tests1

Comment 4

[Armen [:armenzg]]

Attachment: enable bm2-tests2

bm2 is not even sweating and the cpu_wio does not seem to be as bad.

I have looked at the load of bm{04,06,11} and it looks much better today.

I will re-evaluate again in the morning to determine if I want to enable bm2-tests2.

Meanwhile I will look into enabling puppet on bm2 if it was supposed to.

Comment 5

[Armen [:armenzg]]

Comment on attachment 547458 [details] [diff] [review]
enable bm2-tests2

http://hg.mozilla.org/build/tools/rev/506d8a4c87d3
http://hg.mozilla.org/build/tools/rev/d3bbaa6a81fb

Comment 6

[Armen [:armenzg]]

I don't see buildbot-master2 on the list of masters managed by puppet so there is no pupetry adjustment to be done. [1]

This is all done.
We now have bm2-tests1 and bm2-tests2 enabled.

zandr is looking to get drives to improve the wait IO on the kvm cluster.
At that point we could disable this master again.

[1] http://hg.mozilla.org/build/puppet-manifests/file/b165672e096c/buildmaster-production.pp