487654 - reset ESX host bm-vmware02

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task, P1)

Description

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

The new bm-vmware02 just died, taking down the VMs that were on it. 

From irc, mrz thinkgs this needs physical reset.

Comment 1

[Aravind Gottipati [:aravind]]

The physical reset did not help, we are now bringing up the VMs on the other esx servers.

Comment 2

[Aravind Gottipati [:aravind]]

Here is the list of VMs impacted.

bm-l10n-win2k3-01
fx-win32-1.9-slave2
fxdbug-win32-tbox
tb-linux-tbox
moz2-linux-slave22
moz2-linux-slave24
production-prometheus-vm
try-master
fx-linux-1.9-slave1
fx-linux-1.9-slave2
moz2-linux-slave15
moz2-win32-slave24
moz2-win32-slave27

Comment 3

[Aravind Gottipati [:aravind]]

Working with vmware on re-attaching the VMs.

Comment 4

[Phong Tran [:phong]]

The following are back online:

fxdbug-win32-tbox		eql01-bm01
tb-linux-tbox			eql01-bm02, netapp-c-001
moz2-linux-slave22		eql01-bm08
moz2-linux-slave24		eql01-bm08
try-master			eql01-bm01
fx-linux-1.9-slave1		d-sata-build-001
fx-linux-1.9-slave2		eql01-bm02
moz2-linux-slave15		eql01-bm05
moz2-win32-slave24*		eql01-bm07
moz2-win32-slave27		eql01-bm08
fx-win32-1.9-slave2		eql01-bm01

Comment 5

[Phong Tran [:phong]]

still working on the confusion over production-prometheus-vm.

Comment 6

[Phong Tran [:phong]]

production-prometheus-vm is online.  Please reopen if there are any issues.