Closed Bug 473112 Opened 16 years ago Closed 16 years ago

Build machines have lost network connectivity

Categories

(mozilla.org Graveyard :: Server Operations, task)

Product:
mozilla.org Graveyard
Component:
Server Operations
Type:
task
Priority:
Not set
Severity:
blocker

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: nthomas, Assigned: mrz)

References

Details

Approximately 70 reports from nagios of the form
 [30] moz2-linux-slave04.build is DOWN: CRITICAL - Host Unreachable (10.2.71.18)

Linux & Windows VMs, xserves, talos slaves, and buildbot masters.
Related to switch upgrade which has failed, possibly due to a faulty memory card.  I'm not sure why ESX is failing to connect to storage (spanning-tree has already failed over but the service console is unresponsive).
Assignee: mrz → dmoore
netapp-d is single homed or for some reason hasn't failed over.  I was on the wrong track about ESX - it's fine but any VM on netapp-d is likely offline.
See bug 473113.
I rebooted fx-win32-tbox.  Can't seem to find anyone from build online for guidance on what's up or down.
I'll be around in 30 minutes or so to bring machines back up.
IT fixed this up.
Assignee: dmoore → mrz
Status: ASSIGNED → RESOLVED
Closed: 16 years ago
Resolution: --- → FIXED
... and we're tracking bringing the machines back up in bug 473126.
Blocks: 473126
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.