Closed Bug 961076 Opened 10 years ago Closed 10 years ago

Please diagnose/recover sea-mini-osx64-{3,4}

Categories

(Infrastructure & Operations :: DCOps, task)

task
Not set
normal

Tracking

(Not tracked)

VERIFIED FIXED

People

(Reporter: Callek, Unassigned)

Details

So sea-mini-osx64-3 has been down for a while (Since Jan 3) - Unable to ping
And sea-mini-osx64-4 has been burning jobs since Jan 11.

(Timeouts in relatively simple commands relating to cleanup), it also is unable to let me ssh in, despite actually allowing the tcp connection.

[Callek@jump1.community.scl3 ~]$ ssh -vv seabld@sea-mini-osx64-4
OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010
debug1: Reading configuration data /home/Callek/.ssh/config
debug1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to sea-mini-osx64-4 [63.245.223.83] port 22.
debug1: Connection established.
debug2: key_type_from_name: unknown key type '-----BEGIN'
debug2: key_type_from_name: unknown key type '-----END'
debug1: identity file /home/Callek/.ssh/seabld_dsa type -1
♥

If the diagnostics run forces a need for reimage we need to clone sea-mini-osx64-N rather than using any "standard" image, then of course fix the static IP address configured, as well as do a `rm -f /builds/slave/buildbot.tac` on the reimaged host.


For priority setting, this is one half of or OSX pool
colo-trip: --- → scl3
:callek, these 2 hosts were hung but came back after a reboot. i didnt see any errors during boot up. let me know if you want me to take these hosts back down to run any diagnostics.

[vle@admin1a.private.scl3 ~]$ fping !$
fping sea-mini-osx64-3.community.scl3.mozilla.com
sea-mini-osx64-3.community.scl3.mozilla.com is alive
[vle@admin1a.private.scl3 ~]$ ssh !$
ssh sea-mini-osx64-3.community.scl3.mozilla.com
The authenticity of host 'sea-mini-osx64-3.community.scl3.mozilla.com (63.245.223.82)' can't be established.
RSA key fingerprint is e7:19:ba:03:b7:5b:02:8a:7a:0d:e5:7d:a6:8b:2a:35.
Are you sure you want to continue connecting (yes/no)? 

[vle@admin1a.private.scl3 ~]$ fping sea-mini-osx64-4.community.scl3.mozilla.com
sea-mini-osx64-4.community.scl3.mozilla.com is alive
[vle@admin1a.private.scl3 ~]$ ssh !$
ssh sea-mini-osx64-4.community.scl3.mozilla.com
The authenticity of host 'sea-mini-osx64-4.community.scl3.mozilla.com (63.245.223.83)' can't be established.
RSA key fingerprint is e7:19:ba:03:b7:5b:02:8a:7a:0d:e5:7d:a6:8b:2a:35.
Are you sure you want to continue connecting (yes/no)?
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Both systems are up and running.
Status: RESOLVED → VERIFIED
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.