Status

Infrastructure & Operations
DCOps
RESOLVED FIXED
4 years ago
3 years ago

People

(Reporter: dividehex, Unassigned)

Tracking

Details

(Reporter)

Description

4 years ago
This was removed and demoed at the summit but is now back at SCL1.  It is sitting on the floor near its rack space and just needs to be put back on its rails and re-cabled.
(Reporter)

Updated

4 years ago
Blocks: 892691

Comment 1

4 years ago
HD panda chassis racked and cabled.  903 & 909 is very unstable.  I've rebooted them multiple times.  It would ping but right after I close the lid and have every thing in place, it dies on me.


sudo fping < pingtest 
panda-0887.p10.releng.scl1.mozilla.com is alive
panda-0888.p10.releng.scl1.mozilla.com is alive
panda-0889.p10.releng.scl1.mozilla.com is alive
panda-0890.p10.releng.scl1.mozilla.com is alive
panda-0891.p10.releng.scl1.mozilla.com is alive
panda-0892.p10.releng.scl1.mozilla.com is alive
panda-0893.p10.releng.scl1.mozilla.com is alive
panda-0894.p10.releng.scl1.mozilla.com is alive
panda-0895.p10.releng.scl1.mozilla.com is alive
panda-0896.p10.releng.scl1.mozilla.com is alive
panda-0897.p10.releng.scl1.mozilla.com is alive
panda-0898.p10.releng.scl1.mozilla.com is alive
panda-0899.p10.releng.scl1.mozilla.com is alive
panda-0900.p10.releng.scl1.mozilla.com is alive
panda-0901.p10.releng.scl1.mozilla.com is alive
panda-0902.p10.releng.scl1.mozilla.com is alive
panda-0904.p10.releng.scl1.mozilla.com is alive
panda-0905.p10.releng.scl1.mozilla.com is alive
panda-0906.p10.releng.scl1.mozilla.com is alive
panda-0907.p10.releng.scl1.mozilla.com is alive
panda-0908.p10.releng.scl1.mozilla.com is alive
panda-0903.p10.releng.scl1.mozilla.com is unreachable
panda-0909.p10.releng.scl1.mozilla.com is unreachable

Updated

4 years ago
colo-trip: --- → scl1

Comment 2

4 years ago
The inconsistency of panda-0903 before it completely goes unpingable.

64 bytes from 10.12.137.147: icmp_seq=279 ttl=53 time=11.197 ms
64 bytes from 10.12.137.147: icmp_seq=280 ttl=53 time=14.775 ms
Request timeout for icmp_seq 281
Request timeout for icmp_seq 282
Request timeout for icmp_seq 283
Request timeout for icmp_seq 284
Request timeout for icmp_seq 285
64 bytes from 10.12.137.147: icmp_seq=286 ttl=53 time=14.859 ms
64 bytes from 10.12.137.147: icmp_seq=287 ttl=53 time=24.711 ms
Request timeout for icmp_seq 288
Request timeout for icmp_seq 289
Request timeout for icmp_seq 290
Request timeout for icmp_seq 291
64 bytes from 10.12.137.147: icmp_seq=292 ttl=53 time=11.130 ms
64 bytes from 10.12.137.147: icmp_seq=293 ttl=53 time=12.512 ms
64 bytes from 10.12.137.147: icmp_seq=294 ttl=53 time=11.012 ms
Request timeout for icmp_seq 295
Request timeout for icmp_seq 296
Request timeout for icmp_seq 297
Request timeout for icmp_seq 298
Request timeout for icmp_seq 299
Request timeout for icmp_seq 300
Request timeout for icmp_seq 301
Request timeout for icmp_seq 302
Request timeout for icmp_seq 303
Request timeout for icmp_seq 304
Request timeout for icmp_seq 305
Request timeout for icmp_seq 306
Request timeout for icmp_seq 307
Request timeout for icmp_seq 308
Request timeout for icmp_seq 309
Request timeout for icmp_seq 310

Comment 3

4 years ago
And here's panda-0909

Request timeout for icmp_seq 105
Request timeout for icmp_seq 106
64 bytes from 10.12.137.153: icmp_seq=107 ttl=53 time=8.679 ms
64 bytes from 10.12.137.153: icmp_seq=108 ttl=53 time=8.538 ms
64 bytes from 10.12.137.153: icmp_seq=109 ttl=53 time=16.581 ms
Request timeout for icmp_seq 110
Request timeout for icmp_seq 111
Request timeout for icmp_seq 112
Request timeout for icmp_seq 113
64 bytes from 10.12.137.153: icmp_seq=114 ttl=53 time=14.756 ms
64 bytes from 10.12.137.153: icmp_seq=115 ttl=53 time=8.437 ms
64 bytes from 10.12.137.153: icmp_seq=116 ttl=53 time=12.542 ms
Request timeout for icmp_seq 117
Request timeout for icmp_seq 118
Request timeout for icmp_seq 119
Request timeout for icmp_seq 120
Request timeout for icmp_seq 121
64 bytes from 10.12.137.153: icmp_seq=122 ttl=53 time=11.137 ms
64 bytes from 10.12.137.153: icmp_seq=123 ttl=53 time=11.543 ms
64 bytes from 10.12.137.153: icmp_seq=124 ttl=53 time=8.333 ms
Request timeout for icmp_seq 125
Request timeout for icmp_seq 126
Request timeout for icmp_seq 127
Request timeout for icmp_seq 128
Request timeout for icmp_seq 129
Request timeout for icmp_seq 130
Request timeout for icmp_seq 131
Request timeout for icmp_seq 132
Request timeout for icmp_seq 133
Request timeout for icmp_seq 134
Request timeout for icmp_seq 135
Request timeout for icmp_seq 136
Request timeout for icmp_seq 137
Request timeout for icmp_seq 138
Request timeout for icmp_seq 139
Request timeout for icmp_seq 140
(Reporter)

Comment 4

4 years ago
That is typical of a pandaboard since the bootloader initializes the network hardware and loads a tcp/ip stack in order to pxe boot.  When I took them out of the locked_out in mozpool, those 2 were detected and mozpool started the self-test on them right away.  I started the selftests manually on the other pandas in the chassis.   And ... checking back on them, they have all passed their self-test.

Thanks for racking it!
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.