867695 - Re-purpose 90 Ubuntu32/64 ix machines as Windows XP/7/8 ix machines

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task, P3)

Description

[Armen [:armenzg]]

We seem to run 4-5x more jobs on win8 hw that on Ubuntu hw.
We run 3-4 times more jobs on Ubuntu VMs than Ubuntu HW.

I am going to disable 45 Ubuntu HW machines from each pool to match this plan:
Ubuntu32: 65 (-45%)
Ubuntu64: 65 (-45%)
WinXp: 130 (+30%)
Win7:  130 (+30%)
Win8:  130 (+30%)

This will make the ratio of our machines to be:
130/65=2:1 rather than 1:1 which is still less than 4:1 or 5:1.

Sheriffs, would you like me to run a thread on dev.platform to determine if we will be doubling or tripling jobs in the foreseeable future?

Does this plan make sense to you?

We have bad win8 wait times with 100 nodes and I expect that we will also have bad wait times with win7 and winxp.

From 04-18:
ubuntu32_hw/ubuntu32_vm: 714/2910 (100%/100%)
ubuntu64_hw/ubuntu64_vm: 696/2494 (100%/100%)
win8: 3633 (86.02%)

From 05-01:
ubuntu32_hw: 924 - 100.00%
ubuntu64_hw: 900 - 100.00%
win8: 4217 - 83.21%

Comment 1

[Armen [:armenzg]]

Attachment: Idle Ubuntu machines

This attachment shows that 81 Ubuntu hardware machines have been idle for the last 3 hours to 22 hours.

Comment 2

[Armen [:armenzg]]

I have disabled these slaves:
* talos-linux32-ix-[055-100]
* talos-linux64-ix-[055-100]

Comment 3

[Armen [:armenzg]]

I have filed 3 bugs to re-purpose those 90 Ubuntu machines as Windows iX machines as we have bad wait times.

Comment 4

[Armen [:armenzg]]

Current rev3 wait times are better than iX machines.
I will not be allowed to switch over until they're on parity.

Comment 5

[Armen [:armenzg]]

Attachment: [configs] add winxp, win7 & win8 ix machines

Comment 6

[Armen [:armenzg]]

Attachment: [slavealloc] add winxp, win7 and win8 ix machines

Comment 7

[Armen [:armenzg]]

Attachment: [graphs] add more winxp, win7 and win8 ix machines

Comment 8

[Armen [:armenzg]]

Live in production.

Comment 9

[Armen [:armenzg]]

Comment on attachment 751914 [details] [diff] [review]
[configs] add winxp, win7 & win8 ix machines

r=Callek

Comment 10

[Armen [:armenzg]]

Comment on attachment 751915 [details]
[slavealloc] add winxp, win7 and win8 ix machines

r=Callek

Comment 11

[Armen [:armenzg]]

Comment on attachment 751916 [details]
[graphs] add more winxp, win7 and win8 ix machines

r=Callek

Comment 12

[Armen [:armenzg]]

I've rebooted all of those machines into production.

Comment 13

[Armen [:armenzg]]

I will check tomorrow to see our wait times but I assume that it will be close or past 95%.