534329 - Mothball FF3.0 staging machines

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Please take backups of the following machines, so we can mothball them.

 fx-linux-1.9-slave1
 fx-linux-1.9-slave03
 fx-mac-1.9-slave1
 fx-win32-1.9-slave1
 fx-win32-1.9-slave03
 staging-1.9-master

fx-linux-1.9-slave08
fx-linux-1.9-slave09
fx-win32-1.9-slave08
fx-win32-1.9-slave09

Comment 1

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

...and one more:

 bm-stage-osx-01

Comment 2

[matthew zeier [:mrz]]

I bet this will clear up enough space to get 532709 going.

Are these VMs powered off?

Comment 3

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #2)
> I bet this will clear up enough space to get 532709 going.
Yep! :-)

> Are these VMs powered off?
Not yet. Can you first remove these machines from nagios, to prevent it squawking when I power off??

Comment 4

[Phong Tran [:phong]]

aravind: where can I copy these VM files over for backup and archive?

Comment 5

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #3)
> (In reply to comment #2)
> > Are these VMs powered off?
> Not yet. Can you first remove these machines from nagios, to prevent it
> squawking when I power off??

Phong: can you remove these from nagios, so I can do clean poweroff of each VM?

Comment 6

[Aravind Gottipati [:aravind]]

@phong:  Please copy them to nfs dracula:/data/backup-drop.

Comment 7

[Phong Tran [:phong]]

joduinn: removed from nagios.  let me know when those are off.

Comment 8

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #7)
> joduinn: removed from nagios.  let me know when those are off.

Phong: ok, all VMs now off, and afaict, the mac physical machines are also. Back to you for backups.

Comment 9

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #7)
> joduinn: removed from nagios.  let me know when those are off.

staging-1.9-master is listed as staging-trunk-automation in nagios. Could you remove that one too please.

Comment 10

[Phong Tran [:phong]]

staging-trunk-automation has been removed from nagios as well.

Comment 11

[Phong Tran [:phong]]

Deleted VMs from ESX, DNS, DHCP, and nagios.  Still need to image the MAC machines for backups.

fx-mac-1.9-slave1
bm-stage-osx-01

Comment 12

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Phong: can you also remove these from inventory?

Comment 13

[Phong Tran [:phong]]

VMs removed from inventory

Comment 14

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #11)
> Deleted VMs from ESX, DNS, DHCP, and nagios.  
Do we now have room for creating new VMs?


> Still need to image the MAC machines for backups.
> fx-mac-1.9-slave1
> bm-stage-osx-01

Any ETA?

Comment 15

[matthew zeier [:mrz]]

John, won't be until next week - will that work?

Comment 16

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #15)
> John, won't be until next week - will that work?

Backing up those two macs can wait until next week. I'm actually more interested in when the new VMs will be available; they were blocked on freeing up space by deleting the VMs in this bug.

Comment 17

[matthew zeier [:mrz]]

There's about 400GB of storage space available which is just about the same amount needed to move those VMs off netapp c/d.  Depends what you want to do - I can move them and you'll get your new VMs after the upgrade or keep them and get the VMs now-ish.

Thoughts?

Comment 18

[Phong Tran [:phong]]

Each window VMs takes about 110GB.  That's not nearly enough for 10.  Most of the equallogic datastore have between 50-100 GB of free space.

/vmfs/volumes/483f106b-1607ea73-98f6-0015600e49f9
                      499G  329G  170G  65% /vmfs/volumes/eql01-bm02
/vmfs/volumes/48482656-3f7d9d79-de58-000423c43ac0
                      660G  415G  244G  62% /vmfs/volumes/d-sata-build-001
/vmfs/volumes/4849fcd9-e93b2580-3cdd-0019bb271854
                      660G  489G  170G  74% /vmfs/volumes/d-sata-build-002
/vmfs/volumes/484c9c5e-5607d8df-1d2c-000423c43ac0
                      660G  387G  272G  58% /vmfs/volumes/d-sata-build-003
/vmfs/volumes/484caf60-4a35e079-a3e7-000423c43ac0
                      552G  452G   99G  82% /vmfs/volumes/d-fcal-build-002
/vmfs/volumes/484d3731-4410c8fa-d4ec-000423c43ac0
                      660G  585G   74G  88% /vmfs/volumes/d-fcal-build-001
/vmfs/volumes/48518a98-b32c9d56-e30a-0015600e49f9
                      495G  404G   90G  81% /vmfs/volumes/c-sata-build-002
/vmfs/volumes/48518adf-1ffec9b8-fc2c-0015600e49f9
                      595G  301G  293G  50% /vmfs/volumes/c-fcal-build-001
/vmfs/volumes/48518afa-a611ff1c-5813-0015600e49f9
                      615G  482G  132G  78% /vmfs/volumes/c-fcal-build-002
/vmfs/volumes/4897876b-dcbce799-f7ed-000423c43ac0
                      549G  388G  161G  70% /vmfs/volumes/eql01-bm03
/vmfs/volumes/48e53acd-178287a9-4b55-000423c43ac0
                      699G  596G  102G  85% /vmfs/volumes/eql01-bm04
/vmfs/volumes/48e6a1b3-40fc22e3-d20e-000423c43ac0
                      699G  599G  100G  85% /vmfs/volumes/eql01-bm05
/vmfs/volumes/49148f39-68196c3f-7c19-000423c43ac0
                      699G  609G   89G  87% /vmfs/volumes/eql01-bm06
/vmfs/volumes/4982377f-6d1148cc-4e19-001e0bd9220c
                      699G  623G   76G  89% /vmfs/volumes/eql01-bm07
/vmfs/volumes/49da717e-a6a96c05-ff3b-000423c43ac0
                      699G  601G   97G  86% /vmfs/volumes/eql01-bm08
/vmfs/volumes/49e4c75a-0cf3453c-d468-00215ace8f9e
                      699G  589G  109G  84% /vmfs/volumes/eql01-bm09
/vmfs/volumes/4a1c5bff-079dcdc4-42bd-001e0bd9220c
                      499G  419G   79G  84% /vmfs/volumes/eql01-bm10
/vmfs/volumes/4a1f1246-ac717759-71d3-00215ace8f9e
                      499G  419G   79G  84% /vmfs/volumes/eql01-bm11
/vmfs/volumes/4a234766-480925d8-11bc-001e0bd9220c
                      499G  457G   42G  91% /vmfs/volumes/eql01-bm12
/vmfs/volumes/4a383168-d5181034-7521-001e0bd9220c
                      499G  385G  114G  77% /vmfs/volumes/eql01-bm13
/vmfs/volumes/4a649d35-89da286d-8246-00215ace8f9e
                      499G  419G   79G  84% /vmfs/volumes/eql02-bm01
/vmfs/volumes/4a64a07c-db8e1390-1cf0-00215af384b0
                      499G  420G   78G  84% /vmfs/volumes/eql02-bm02
/vmfs/volumes/4a6522bb-ee5d0216-147d-00215ace8f9e
                      499G  417G   81G  83% /vmfs/volumes/eql02-bm03
/vmfs/volumes/4a8c86c8-7a1d398a-2db9-00215ace8f9e
                      499G  419G   79G  84% /vmfs/volumes/eql02-bm04
/vmfs/volumes/4a8c871b-5b9fd92b-4403-00215ace8f9e
                      499G  419G   79G  84% /vmfs/volumes/eql02-bm05
/vmfs/volumes/4a8c8784-6339770c-d1b2-00215ace8f9e
                      499G  436G   62G  87% /vmfs/volumes/eql02-bm06
/vmfs/volumes/4a8c8840-b676f3ae-d944-00215ace8f9e
                      499G  403G   96G  80% /vmfs/volumes/eql02-bm07
/vmfs/volumes/4ac17924-fd8c6144-42b7-001e0bd9220c
                      499G  419G   79G  84% /vmfs/volumes/eql02-bm08
/vmfs/volumes/4ac179c4-32655ebe-4ad1-001e0bd9220c
                      499G  419G   79G  84% /vmfs/volumes/eql02-bm09
/vmfs/volumes/4ac3ddba-a79dde30-0b3c-001e0bd9220c
                      499G  419G   79G  84% /vmfs/volumes/eql02-bm10
/vmfs/volumes/4ac3debc-03d07de0-1cbe-001e0bd9220c
                      499G  388G  111G  77% /vmfs/volumes/eql02-bm11
/vmfs/volumes/4ac3dfa3-5e58cc40-0e61-001e0bd9220c
                      499G  397G  102G  79% /vmfs/volumes/eql02-bm12
/vmfs/volumes/4b1c5e62-e34a08d6-8016-001e0bd9220c
                      499G  447G   52G  89% /vmfs/volumes/eql02-bm13
/vmfs/volumes/4b1c5f0d-9fd3ed22-e984-001e0bd9220c
                      499G  447G   52G  89% /vmfs/volumes/eql02-bm14
[root@bm-vmware01 root]#

Comment 19

[Phong Tran [:phong]]

aravind: the last 2 mac images are also at dracula:/data/backup-drop

fx-mac-1.9-slave1
bm-stage-osx-01

Comment 20

[Phong Tran [:phong]]

removed the last 2 from dns/dhcp and updated inventory.