889567 - tegra-recovery

Status: RESOLVED FIXED

(Infrastructure & Operations :: DCOps, task)

Description

[Justin Wood (:Callek)]

Recovery bug for tegras, default-action is to reimage.
-->Automatically filed by script

Comment 1

[Justin Wood (:Callek)]

Please swap sdcard for tegra-345 and tegra-368 as well.

Thanks!

Comment 2

[Ashlee Nguyen [:ashlee]]

The following tegras have been recovered:

Swapped sd cards for tegra-345 and tegra-368

--- tegra-328.build.mtv1.mozilla.com ping statistics ---
7 packets transmitted, 7 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 2.161/3.915/5.224/1.186 ms

--- tegra-304.build.mtv1.mozilla.com ping statistics ---
7 packets transmitted, 7 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 4.371/6.571/10.877/2.065 ms

--- tegra-348.build.mtv1.mozilla.com ping statistics ---
5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 5.195/6.803/9.250/1.505 ms

--- tegra-307.build.mtv1.mozilla.com ping statistics ---
6 packets transmitted, 6 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 5.814/39.579/63.639/20.413 ms

--- tegra-311.build.mtv1.mozilla.com ping statistics ---
5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 3.565/20.212/51.360/16.682 ms

--- tegra-295.build.mtv1.mozilla.com ping statistics ---
6 packets transmitted, 6 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 4.896/14.829/44.226/13.906 ms

--- tegra-363.build.mtv1.mozilla.com ping statistics ---
9 packets transmitted, 9 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 2.540/4.566/5.364/0.855 ms

--- tegra-344.build.mtv1.mozilla.com ping statistics ---
6 packets transmitted, 6 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 2.585/13.085/43.137/13.883 ms

--- tegra-360.build.mtv1.mozilla.com ping statistics ---
6 packets transmitted, 6 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 6.383/24.888/94.817/31.483 ms

--- tegra-346.build.mtv1.mozilla.com ping statistics ---
6 packets transmitted, 6 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 2.486/4.306/5.301/1.167 ms

--- tegra-339.build.mtv1.mozilla.com ping statistics ---
7 packets transmitted, 7 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 5.232/16.194/32.749/10.278 ms

--- tegra-368.build.mtv1.mozilla.com ping statistics ---
6 packets transmitted, 6 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 4.993/10.031/28.615/8.492 ms

--- tegra-315.build.mtv1.mozilla.com ping statistics ---
6 packets transmitted, 6 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 1.757/3.045/5.553/1.534 ms

--- tegra-345.build.mtv1.mozilla.com ping statistics ---
6 packets transmitted, 6 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 6.932/30.473/91.191/28.250 ms

--- tegra-356.build.mtv1.mozilla.com ping statistics ---
6 packets transmitted, 6 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 2.164/3.703/5.337/1.252 ms

--- tegra-364.build.mtv1.mozilla.com ping statistics ---
5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 2.384/3.986/5.409/1.301 ms

Comment 3

[Trey Franco [:tfranco]]

PING tegra-100.build.mtv1.mozilla.com (10.250.49.88): 56 data bytes
64 bytes from 10.250.49.88: icmp_seq=0 ttl=63 time=6.606 ms
64 bytes from 10.250.49.88: icmp_seq=1 ttl=63 time=5.279 ms
64 bytes from 10.250.49.88: icmp_seq=2 ttl=63 time=5.177 ms
 
PING tegra-103.build.mtv1.mozilla.com (10.250.49.91): 56 data bytes
64 bytes from 10.250.49.91: icmp_seq=0 ttl=63 time=3.889 ms
64 bytes from 10.250.49.91: icmp_seq=1 ttl=63 time=2.850 ms
64 bytes from 10.250.49.91: icmp_seq=2 ttl=63 time=2.195 ms

PING tegra-142.build.mtv1.mozilla.com (10.250.50.52): 56 data bytes
64 bytes from 10.250.50.52: icmp_seq=0 ttl=63 time=9.082 ms
64 bytes from 10.250.50.52: icmp_seq=1 ttl=63 time=4.969 ms
64 bytes from 10.250.50.52: icmp_seq=2 ttl=63 time=4.796 ms
 
PING tegra-110.build.mtv1.mozilla.com (10.250.49.98): 56 data bytes
64 bytes from 10.250.49.98: icmp_seq=0 ttl=63 time=7.571 ms
64 bytes from 10.250.49.98: icmp_seq=1 ttl=63 time=5.379 ms
64 bytes from 10.250.49.98: icmp_seq=2 ttl=63 time=10.191 ms

PING tegra-228.build.mtv1.mozilla.com (10.250.51.68): 56 data bytes
64 bytes from 10.250.51.68: icmp_seq=0 ttl=63 time=9.467 ms
64 bytes from 10.250.51.68: icmp_seq=1 ttl=63 time=2.051 ms
64 bytes from 10.250.51.68: icmp_seq=2 ttl=63 time=5.313 ms
  
PING tegra-219.build.mtv1.mozilla.com (10.250.51.59): 56 data bytes
64 bytes from 10.250.51.59: icmp_seq=0 ttl=63 time=4.093 ms
64 bytes from 10.250.51.59: icmp_seq=1 ttl=63 time=3.163 ms
64 bytes from 10.250.51.59: icmp_seq=2 ttl=63 time=5.007 ms

PING tegra-218.build.mtv1.mozilla.com (10.250.51.58): 56 data bytes
64 bytes from 10.250.51.58: icmp_seq=0 ttl=63 time=5.042 ms
64 bytes from 10.250.51.58: icmp_seq=1 ttl=63 time=4.143 ms
64 bytes from 10.250.51.58: icmp_seq=2 ttl=63 time=3.716 ms
 
PING tegra-149.build.mtv1.mozilla.com (10.250.50.59): 56 data bytes
64 bytes from 10.250.50.59: icmp_seq=0 ttl=63 time=5.257 ms
64 bytes from 10.250.50.59: icmp_seq=1 ttl=63 time=5.232 ms
64 bytes from 10.250.50.59: icmp_seq=2 ttl=63 time=6.395 ms

PING tegra-157.build.mtv1.mozilla.com (10.250.50.67): 56 data bytes
64 bytes from 10.250.50.67: icmp_seq=0 ttl=63 time=10.250 ms
64 bytes from 10.250.50.67: icmp_seq=1 ttl=63 time=2.057 ms
64 bytes from 10.250.50.67: icmp_seq=2 ttl=63 time=2.048 ms
 
PING tegra-141.build.mtv1.mozilla.com (10.250.50.51): 56 data bytes
64 bytes from 10.250.50.51: icmp_seq=0 ttl=63 time=5.700 ms
64 bytes from 10.250.50.51: icmp_seq=1 ttl=63 time=7.512 ms
64 bytes from 10.250.50.51: icmp_seq=2 ttl=63 time=4.996 ms
 
PING tegra-121.build.mtv1.mozilla.com (10.250.50.31): 56 data bytes
64 bytes from 10.250.50.31: icmp_seq=0 ttl=63 time=5.013 ms
64 bytes from 10.250.50.31: icmp_seq=1 ttl=63 time=5.122 ms
64 bytes from 10.250.50.31: icmp_seq=2 ttl=63 time=5.092 ms

PING tegra-120.build.mtv1.mozilla.com (10.250.50.30): 56 data bytes
64 bytes from 10.250.50.30: icmp_seq=0 ttl=63 time=4.947 ms
64 bytes from 10.250.50.30: icmp_seq=1 ttl=63 time=2.544 ms
64 bytes from 10.250.50.30: icmp_seq=2 ttl=63 time=2.534 ms

PING tegra-140.build.mtv1.mozilla.com (10.250.50.50): 56 data bytes
64 bytes from 10.250.50.50: icmp_seq=0 ttl=63 time=29.557 ms
64 bytes from 10.250.50.50: icmp_seq=1 ttl=63 time=2.183 ms
64 bytes from 10.250.50.50: icmp_seq=2 ttl=63 time=3.091 ms
 
PING tegra-247.build.mtv1.mozilla.com (10.250.51.87): 56 data bytes
64 bytes from 10.250.51.87: icmp_seq=0 ttl=63 time=5.281 ms
64 bytes from 10.250.51.87: icmp_seq=1 ttl=63 time=5.300 ms
64 bytes from 10.250.51.87: icmp_seq=2 ttl=63 time=5.275 ms
 
PING tegra-111.build.mtv1.mozilla.com (10.250.50.21): 56 data bytes
64 bytes from 10.250.50.21: icmp_seq=0 ttl=63 time=15.833 ms
64 bytes from 10.250.50.21: icmp_seq=1 ttl=63 time=5.306 ms
64 bytes from 10.250.50.21: icmp_seq=2 ttl=63 time=7.701 ms

Comment 4

[Trey Franco [:tfranco]]

PING tegra-236.build.mtv1.mozilla.com (10.250.51.76): 56 data bytes
64 bytes from 10.250.51.76: icmp_seq=0 ttl=63 time=7.742 ms
64 bytes from 10.250.51.76: icmp_seq=1 ttl=63 time=4.660 ms
64 bytes from 10.250.51.76: icmp_seq=2 ttl=63 time=4.877 ms

PING tegra-262.build.mtv1.mozilla.com (10.250.51.102): 56 data bytes
64 bytes from 10.250.51.102: icmp_seq=0 ttl=63 time=2.767 ms
64 bytes from 10.250.51.102: icmp_seq=1 ttl=63 time=2.374 ms
64 bytes from 10.250.51.102: icmp_seq=2 ttl=63 time=5.052 ms

Comment 5

[Trey Franco [:tfranco]]

   
PING tegra-056.build.mtv1.mozilla.com (10.250.49.43): 56 data bytes
64 bytes from 10.250.49.43: icmp_seq=0 ttl=63 time=42.698 ms
64 bytes from 10.250.49.43: icmp_seq=1 ttl=63 time=7.447 ms
64 bytes from 10.250.49.43: icmp_seq=2 ttl=63 time=5.143 ms

PING tegra-078.build.mtv1.mozilla.com (10.250.49.66): 56 data bytes
64 bytes from 10.250.49.66: icmp_seq=0 ttl=63 time=5.100 ms
64 bytes from 10.250.49.66: icmp_seq=1 ttl=63 time=4.892 ms
64 bytes from 10.250.49.66: icmp_seq=2 ttl=63 time=2.820 ms

tegra-088.build.mtv1.mozilla.com (10.250.49.76): 56 data bytes
64 bytes from 10.250.49.76: icmp_seq=0 ttl=63 time=5.074 ms
64 bytes from 10.250.49.76: icmp_seq=1 ttl=63 time=5.221 ms
64 bytes from 10.250.49.76: icmp_seq=2 ttl=63 time=5.935 ms

tegra-072.build.mtv1.mozilla.com (10.250.49.59): 56 data bytes
64 bytes from 10.250.49.59: icmp_seq=0 ttl=63 time=5.121 ms
64 bytes from 10.250.49.59: icmp_seq=1 ttl=63 time=8.175 ms
64 bytes from 10.250.49.59: icmp_seq=2 ttl=63 time=20.454 ms

tegra-093.build.mtv1.mozilla.com (10.250.49.81): 56 data bytes
64 bytes from 10.250.49.81: icmp_seq=0 ttl=63 time=26.527 ms
64 bytes from 10.250.49.81: icmp_seq=1 ttl=63 time=6.334 ms
64 bytes from 10.250.49.81: icmp_seq=2 ttl=63 time=2.242 ms

tegra-038.build.mtv1.mozilla.com (10.250.49.25): 56 data bytes
64 bytes from 10.250.49.25: icmp_seq=0 ttl=63 time=10.674 ms
64 bytes from 10.250.49.25: icmp_seq=1 ttl=63 time=5.123 ms
64 bytes from 10.250.49.25: icmp_seq=2 ttl=63 time=2.959 ms

PING tegra-247.build.mtv1.mozilla.com (10.250.51.87): 56 data bytes
64 bytes from 10.250.51.87: icmp_seq=0 ttl=63 time=16.129 ms
64 bytes from 10.250.51.87: icmp_seq=1 ttl=63 time=29.851 ms
64 bytes from 10.250.51.87: icmp_seq=2 ttl=63 time=27.273 ms

PING tegra-195.build.mtv1.mozilla.com (10.250.50.105): 56 data bytes
64 bytes from 10.250.50.105: icmp_seq=0 ttl=63 time=5.197 ms
64 bytes from 10.250.50.105: icmp_seq=1 ttl=63 time=5.311 ms
64 bytes from 10.250.50.105: icmp_seq=2 ttl=63 time=5.136 ms

PING tegra-096.build.mtv1.mozilla.com (10.250.49.84): 56 data bytes
64 bytes from 10.250.49.84: icmp_seq=0 ttl=63 time=15.703 ms
64 bytes from 10.250.49.84: icmp_seq=1 ttl=63 time=2.565 ms

Comment 6

[Trey Franco [:tfranco]]

Unable to locate Tegra-051

Comment 7

[Justin Wood (:Callek)]

(In reply to Trey Franco [:tfranco] from comment #6)
> Unable to locate Tegra-051

Huh!?

        "pdu": "pdu2.df201-4.build.mtv1.mozilla.com",

Which matches recent pdu audit....

Comment 8

[Justin Wood (:Callek)]

(In reply to Justin Wood (:Callek) from comment #7)
> (In reply to Trey Franco [:tfranco] from comment #6)
> > Unable to locate Tegra-051
> 
> Huh!?
> 
>         "pdu": "pdu2.df201-4.build.mtv1.mozilla.com",
> 
> Which matches recent pdu audit....

I should also note that 051 is taking jobs now..... so it LOOKS like it was reimaged anyway.

Comment 9

[Justin Wood (:Callek)]

(In reply to Justin Wood (:Callek) from comment #8)
> I should also note that 051 is taking jobs now..... so it LOOKS like it was
> reimaged anyway.

.... and I just saw signs of it still being broken which means it probably didn't in fact. (disabled and tossed on the newest recovery bug)