653539 - move non-test slaves away from buildbot-master5 (and even shut off the build and try masters on it)

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P1)

Description

[Armen [:armenzg]]

Currently buildbot-master5 is overloaded since it has more than 250 slaves connected to it.

A simple pwd step can take more than 40 seconds on the slave.
This means that test jobs are taking forever.

For now we will temporarily move slaves from bm05 and try_master5 to other masters.

Comment 1

[Armen [:armenzg]]

I have disabled bm6 & bm05-try on slavealloc (the previous names were from the Masters wiki page).

Comment 2

[Armen [:armenzg]]

bm05-try was gracefully shutdown.

bm6 has been marked to be gracefully shutdown but still has jobs running.

bm04-tests1 is now being enabled in bug 607179 to help distribute the test slaves.

Comment 3

[Armen [:armenzg]]

The load on buildbot-master5 is now sane since bm04-tests1 and bm06-tests1 have been enabled.

We have now running on it:
- bm6
- bm05-try
- tm3

On slavealloc we have the following masters enabled:
- bm6
- tm3

I will leave bm05-try disabled on slavealloc for now as I don't see the need to enable it (I am still watching migration to bm6).

Comment 4

[Armen [:armenzg]]

buildbot-master5 looks good.
Nothing left to be done.