1120783 - sea-mini-osx64-2 doesn't want to talk to me.

Status: RESOLVED DUPLICATE of bug 1126162

(Infrastructure & Operations :: DCOps, task)

Description

[Edmund Wong (:ewong)]

sea-mini-osx64-2 is not responding to ssh nor VNC connections; but it
is responding to pings, so it's up but hung somewhere along the line I'm
guessing.

Comment 1

[Vinh Hua [:vinh]]

Host was stuck at beachball.  I've rebooted the mini.  Can you try now?

Comment 2

[Edmund Wong (:ewong)]

(In reply to Vinh Hua [:vinh] from comment #1)
> Host was stuck at beachball.  I've rebooted the mini.  Can you try now?

Hm.. looking at the buildmaster, it says it's building something.  But
it's still not responding to vnc or ssh connections.  

Can you give it a kick again? 

Apparently, it's headed to the beach again. ;/

Comment 3

[Edmund Wong (:ewong)]

(In reply to Edmund Wong (:ewong) from comment #2)
> (In reply to Vinh Hua [:vinh] from comment #1)
> > Host was stuck at beachball.  I've rebooted the mini.  Can you try now?
> 
> Hm.. looking at the buildmaster, it says it's building something.  But
> it's still not responding to vnc or ssh connections.  
> 
> Can you give it a kick again? 
> 
> Apparently, it's headed to the beach again. ;/

Actually, ignore the above.  It's building something.  It just doesn't
let me connect via vnc or ssh.  The important thing that it's building
something and I'll take it that it doesn't want to be disturbed.

I'll update this bug when you can give it a kick.

Thanks!

Comment 4

[Edmund Wong (:ewong)]

In the end, it wasn't building anything.  I'm guessing it's playing
with a beachball right now. ;/

Anyway, it's good for a swift kick.

Since this has been a regular occurrence with this, I suspect a
hardware issue (will file a bug when I've conferred with Callek
in what we should be doing next).

Thanks!

Comment 5

[Van Le [:van]]

[[e-mail]3 ~]$ fping !$
fping sea-mini-osx64-2.community.scl3.mozilla.com
sea-mini-osx64-2.community.scl3.mozilla.com is alive
[[e-mail]3 ~]$ ssh !$
ssh sea-mini-osx64-2.community.scl3.mozilla.com
The authenticity of host 'sea-mini-osx64-2.community.scl3.mozilla.com (63.245.223.81)' can't be established.
RSA key fingerprint is e7:19:ba:03:b7:5b:02:8a:7a:0d:e5:7d:a6:8b:2a:35.
Are you sure you want to continue connecting (yes/no)?

Comment 6

[Van Le [:van]]

[[e-mail]3 ~]$ fping !$
fping sea-mini-osx64-2.community.scl3.mozilla.com
sea-mini-osx64-2.community.scl3.mozilla.com is alive
[[e-mail]3 ~]$ ssh !$
ssh sea-mini-osx64-2.community.scl3.mozilla.com
The authenticity of host 'sea-mini-osx64-2.community.scl3.mozilla.com (63.245.223.81)' can't be established.
RSA key fingerprint is e7:19:ba:03:b7:5b:02:8a:7a:0d:e5:7d:a6:8b:2a:35.
Are you sure you want to continue connecting (yes/no)?

Comment 7

[Edmund Wong (:ewong)]

It's not responding again.

I'm guessing it's beachballing. ;/

pings work,  ssh and vnc don't.

Comment 8

[Van Le [:van]]

running extended diagnostics.

Comment 9

[Van Le [:van]]

host passed extended diags so we can't pin point a hardware issue.  we can try to reimage if this host keeps crashing.

[[e-mail]3 ~]$fping sea-mini-osx64-2.community.scl3.mozilla.com
sea-mini-osx64-2.community.scl3.mozilla.com is alive
[[e-mail]3 ~]$ ssh !$
ssh sea-mini-osx64-2.community.scl3.mozilla.com
The authenticity of host 'sea-mini-osx64-2.community.scl3.mozilla.com (63.245.223.81)' can't be established.
RSA key fingerprint is e7:19:ba:03:b7:5b:02:8a:7a:0d:e5:7d:a6:8b:2a:35.
Are you sure you want to continue connecting (yes/no)? no
Host key verification failed.
[[e-mail]3 ~]$

Comment 10

[Edmund Wong (:ewong)]

it's back at the beach. ;/

Callek, what should we do about this?  Diagnostics have shown nothing.

re-image?

Comment 11

[Van Le [:van]]

let's track this under 1126162 and we can tackle both nodes since they're showing the same symptoms. i'll need called in that bug instead.