754413 - (tegra-172) tegra-172 problem tracking

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task)

Description

[Justin Wood (:Callek)]

+++ This bug was initially created as a clone of Bug #753257 +++

So tegra-172 needs some TLC.

It is not in prod or staging atm, and is sitting idle meant as a replacement when others go bad. But it is alerting in nagios as unpingable, and a manual powercycle didn't help it.

Please reflash and check.

Comment 1

[Jake Watkins [:dividehex]]

NVRAM reflashed and SDcard re-initialized.

Comment 2

[[e-mail] (:bhearsum)]

Didn't respond to a PDU reboot, needs recovery.

Comment 3

[Van Le [:van]]

Looks like the NIC on this tegra is dead. I've formated, flashed and reimaged the tegra but I am not getting a link light on the NIC. I've tried testing by using a known good cable/switch port without success.

van

Comment 4

[[e-mail] (:bhearsum)]

Callek, can you remove this slave from our configs?

Comment 5

[Justin Wood (:Callek)]

Removed from configs in Bug 786966, marked as decom in slavalloc just now.

Comment 6

[Justin Wood (:Callek)]

Absolutely no rush van, since we didn't miss this devices absense... however....

c#3 says it was dead, like a doornail, we pulled it from our configs (last job recorded was Friday, August 31, 2012 10:44:50 PM

this weeks audit of the pdu structure (in Bug 878347) which verified all pdu outlet ID's were accurate, still has this listed in the pdu web interface for pdu6.df202-1.build.mtv1.mozilla.com.

The device is also still in DNS and is still pingable, and just a bit ago I could also telnet (I marked it to reboot via software and no I can't telnet into the SUT port, but I'm not trying too hard to recover anyway).

---- So my Q is, is it actually dead, is this IP handling a different device, or should we "properly" decom it now?

Comment 7

[Derek Moore [:dmoore]]

It is, in fact the same device (confirmed by MAC address). It's just been sitting in place as the bug was closed and fell out of our queue before any request to remove the hardware.

Comment 8

[Van Le [:van]]

No, it's the same board. I wasn't able to keep the tegra stable (it was losing connectivity randomly) and I didn't remove it because I wasn't sure if anyone on the A-team wanted to poke at it or if we were just going to decomm it. I might have lost track of this bug as it was never in our queue. 

Let me know how you want to proceed.

Comment 9

[Justin Wood (:Callek)]

(In reply to Van Le [:van] from comment #8)
> Let me know how you want to proceed.

Ok, great - that info is helpful... I'll chat with a few people over the next week(s) and figure out our course of action... self-pointed needinfo so I don't lose this.

Comment 10

[[e-mail] (:bhearsum)]

(In reply to Justin Wood (:Callek) from comment #9)
> (In reply to Van Le [:van] from comment #8)
> > Let me know how you want to proceed.
> 
> Ok, great - that info is helpful... I'll chat with a few people over the
> next week(s) and figure out our course of action... self-pointed needinfo so
> I don't lose this.

ping?

Comment 11

[Justin Wood (:Callek)]

(In reply to Van Le [:van] from comment #8)
> No, it's the same board. I wasn't able to keep the tegra stable (it was
> losing connectivity randomly) and I didn't remove it because I wasn't sure
> if anyone on the A-team wanted to poke at it or if we were just going to
> decomm it. I might have lost track of this bug as it was never in our queue. 
> 
> Let me know how you want to proceed.

Given this, and given that we are evac'ing mtv1 soon we can leave it in-place, just not worth bringing to the new home.