889266 - reboots-scl1

Status: RESOLVED FIXED

(Infrastructure & Operations :: DCOps, task)

Description

[Justin Wood (:Callek)]

+++ This bug was initially created as a clone of Bug #886991 +++

Comment 1

[acoh]

w64-ix-slave08 doesn't seem to be turning on. I double checked the power cables, but it still does not appear to be working.

The following has been rebooted:

PING tegra-231.build.mtv1.mozilla.com (10.250.51.71): 56 data bytes
64 bytes from 10.250.51.71: icmp_seq=0 ttl=63 time=12.387 ms
64 bytes from 10.250.51.71: icmp_seq=1 ttl=63 time=5.292 ms
64 bytes from 10.250.51.71: icmp_seq=2 ttl=63 time=5.419 ms

PING talos-r3-fed64-056.build.scl1.mozilla.com (10.12.50.229): 56 data bytes
64 bytes from 10.12.50.229: icmp_seq=0 ttl=58 time=6.522 ms
64 bytes from 10.12.50.229: icmp_seq=1 ttl=58 time=15.821 ms
64 bytes from 10.12.50.229: icmp_seq=2 ttl=58 time=9.313 ms

PING talos-r3-w7-122.build.scl1.mozilla.com (10.12.49.188): 56 data bytes
64 bytes from 10.12.49.188: icmp_seq=0 ttl=122 time=7.198 ms
64 bytes from 10.12.49.188: icmp_seq=1 ttl=122 time=16.054 ms
64 bytes from 10.12.49.188: icmp_seq=2 ttl=122 time=9.983 ms

Comment 2

[Ashlee Nguyen [:ashlee]]

I took a look at this machine. Even trying different outlets, this machine will not power on. 
I went ahead and filed a ticket with iX Systems. I will keep the bug updated.
Ticket ID: 	FJP-297884

Comment 3

[Ashlee Nguyen [:ashlee]]

iX Systems picked up machine to diagnose the problem. Will update when I hear back from them.

Comment 4

[Ashlee Nguyen [:ashlee]]

w64-ix-slave08 was returned today. They replaced the power supply. It powers on now. It needs to be reimaged.

Comment 5

[Ashlee Nguyen [:ashlee]]

w64-ix-slave08 is back online there was no need to reimage:

--- w64-ix-slave08.winbuild.scl1.mozilla.com ping statistics ---
11 packets transmitted, 0 packets received, 100.0% packet loss
LIONOSXs-MacBook-Pro:~ defaultuser$ ping w64-ix-slave08.winbuild.scl1.mozilla.com
PING w64-ix-slave08.winbuild.scl1.mozilla.com (10.12.40.28): 56 data bytes
64 bytes from 10.12.40.28: icmp_seq=0 ttl=127 time=4.086 ms
64 bytes from 10.12.40.28: icmp_seq=1 ttl=127 time=3.991 ms
64 bytes from 10.12.40.28: icmp_seq=2 ttl=127 time=3.404 ms
64 bytes from 10.12.40.28: icmp_seq=3 ttl=127 time=3.788 ms
64 bytes from 10.12.40.28: icmp_seq=4 ttl=127 time=1.710 ms
^C
--- w64-ix-slave08.winbuild.scl1.mozilla.com ping statistics ---
5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 1.710/3.396/4.086/0.875 ms
LIONOSXs-MacBook-Pro:~ defaultuser$ PING 10.12.41.28
PING 10.12.41.28 (10.12.41.28): 56 data bytes
64 bytes from 10.12.41.28: icmp_seq=0 ttl=63 time=24.755 ms
64 bytes from 10.12.41.28: icmp_seq=1 ttl=63 time=4.152 ms
64 bytes from 10.12.41.28: icmp_seq=2 ttl=63 time=4.311 ms
64 bytes from 10.12.41.28: icmp_seq=3 ttl=63 time=1.581 ms
64 bytes from 10.12.41.28: icmp_seq=4 ttl=63 time=4.179 ms
64 bytes from 10.12.41.28: icmp_seq=5 ttl=63 time=5.477 ms
64 bytes from 10.12.41.28: icmp_seq=6 ttl=63 time=3.363 ms
64 bytes from 10.12.41.28: icmp_seq=7 ttl=63 time=3.410 ms
64 bytes from 10.12.41.28: icmp_seq=8 ttl=63 time=3.675 ms
64 bytes from 10.12.41.28: icmp_seq=9 ttl=63 time=1.369 ms
64 bytes from 10.12.41.28: icmp_seq=10 ttl=63 time=2.872 ms
64 bytes from 10.12.41.28: icmp_seq=11 ttl=63 time=3.463 ms
64 bytes from 10.12.41.28: icmp_seq=12 ttl=63 time=8.778 ms
64 bytes from 10.12.41.28: icmp_seq=13 ttl=63 time=3.354 ms
^C64 bytes from 10.12.41.28: icmp_seq=14 ttl=63 time=1.448 ms
64 bytes from 10.12.41.28: icmp_seq=15 ttl=63 time=4.168 ms
64 bytes from 10.12.41.28: icmp_seq=16 ttl=63 time=7.133 ms
^C
--- 10.12.41.28 ping statistics ---
17 packets transmitted, 17 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 1.369/5.146/24.755/5.233 ms
LIONOSXs-MacBook-Pro:~ defaultuser$ ssh w64-ix-slave08.winbuild.scl1.mozilla.com
The authenticity of host 'w64-ix-slave08.winbuild.scl1.mozilla.com (10.12.40.28)' can't be established.
RSA key fingerprint is 3e:a3:15:bf:b5:38:31:b2:0c:a3:29:bf:40:a6:6d:e5.
Are you sure you want to continue connecting (yes/no)?

Comment 6

[Justin Wood (:Callek)]

Looks like a handful of these weren't rebooted (per comments anyway) --> new bug