616407 - Can't access cb-xserve01

Status: VERIFIED FIXED

(Infrastructure & Operations :: RelOps: General, task)

Description

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

This evening cb-xserve01 went perma-yellow on our tinderbox page, and I can't SSH to it (I get a "ssh_exchange_identification: Connection closed by remote host" response).  I also can't ping it from the jumphost.

Can someone please investigate (and restart, if necessary) cb-xserve01?  Thanks!

Comment 1

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Any ETA on the next colo trip (it's been almost a week now since this bug was filed)?

Comment 2

[Samuel Sidler (:ss)]

Phong, Matthew? Don't make me come upstairs tomorrow!

Comment 3

[Phong Tran [:phong]]

rebooted and back online.

Comment 4

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

I started Tinderbox on the machine and it seems to have immediately frozen up (I can't type anything over SSH, and I'm not getting any UI reaction via VNC).

Comment 5

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

I suspect, based on the symptoms, that we've probably experienced bug 513718 comment 4 again and the box will need re-imaging again :( (using the 10.4 Xserve image).

Comment 6

[Phong Tran [:phong]]

Do you want to upgrade to 10.5.2 like the rest of the releng machines?  If it needs to be 10.4, then I can find the older image.

Comment 7

[Samuel Sidler (:ss)]

It needs to be 10.4 for our purposes.

Comment 8

[Samuel Sidler (:ss)]

I know the all-hands is this week, but what's the ETA on this? Early next week?

Comment 9

[Phong Tran [:phong]]

Can you send me the logon credentials for this server?  I'll try to get to it today.

Comment 10

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Any ETA on this yet?  I know Sam sent the credentials via email the week before Christmas.

Comment 11

[Phong Tran [:phong]]

heading there this afternoon.

Comment 12

[Phong Tran [:phong]]

re-imaged with 10.4.8 image.

Comment 13

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Thanks, Phong!  We're building again :)

There's a firmware update that wants to be applied; I opened bug 623798 about it, so that when someone is in the colo again for whatever reason, the update can be done as a ride-along.