819698 - (tegra-192) tegra-192 problem tracking

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task, P3)

Description

[Justin Wood (:Callek)]

Needs a reimage, not giving a working SUT Port after multiple powercycles.

Comment 1

[Carsten Book [:Tomcat]]

reboot after hung and its back

bash-3.2$ ps auxw | grep "tegra-192"
cltbld         96093   0.0  0.0  2475240   3328   ??  S     6:44am   0:00.03 /opt/local/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python clientproxy.py -b --device=tegra-192
cltbld         96255   0.0  0.0  2434892    544 s003  S+    6:49am   0:00.00 grep tegra-192
cltbld         96198   0.0  0.2  2482972  12864   ??  S     6:46am   0:01.38 /opt/local/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python /opt/local/bin/twistd --no_save --rundir=/builds/tegra-192 --pidfile=/builds/tegra-192/twistd.pid --python=/builds/tegra-192/buildbot.tac
cltbld         96094   0.0  0.0  2447068   2796   ??  S     6:44am   0:00.03 /opt/local/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python clientproxy.py -b --device=tegra-192
bash-3.2$ /builds/check.sh -t tegra-192 
02/01/2013 06:53:45: INFO:  Tegra ID M    Tegra       CP    Slave :: Msg
2013-02-01 06:53:45,400  Tegra ID M    Tegra       CP    Slave :: Msg
02/01/2013 06:53:51: INFO: tegra-192 p    online   active   active :: 
2013-02-01 06:53:51,439 tegra-192 p    online   active   active ::

Comment 2

[Justin Wood (:Callek)]

Disabled due to issue with its foopy (foopy24) tracked in bug 839375

Comment 3

[[e-mail] (:bhearsum)]

Back in production.

Comment 4

[Justin Wood (:Callek)]

Saturday, October 19, 2013 2:24:15 AM

Comment 5

[[e-mail] (:bhearsum)]

This slave took one job and then went down again. Callek, I recall you talking about this symptom recently. How do we fix it?

Comment 6

[[e-mail] (:bhearsum)]

Down again already :(

Comment 7

[Justin Wood (:Callek)]

(In reply to Ben Hearsum [:bhearsum] from comment #5)
> This slave took one job and then went down again. Callek, I recall you
> talking about this symptom recently. How do we fix it?

The symptom I chatted about was for pandas, tegras shouldn't hit this normally, lets see how it behaves after this next round of recovery

Comment 8

[Eric Ramirez [:Eric]]

sdcard replaced and reimaged/flashed.

Comment 9

[Justin Wood (:Callek)]

handled in last recovery on Dec 16.

Comment 10

[[e-mail] (:bhearsum)]

2014-01-16 10:20:49 tegra-192 p  INACTIVE   active  OFFLINE :: SUTAgent not present;error.flg [Automation Error: Unable to connect to device after 5 attempts] 

pdu reboot didn't help

Comment 11

[John Pech [:jpech-lbc]]

SD card have been wiped and tegra have been flashed/re-imaged.

Comment 12

[Justin Wood (:Callek)]

taking production jobs

Comment 13

[Phil Ringnalda (:philor)]

Hasn't taken a job for 6 days.

Comment 14

[Phil Ringnalda (:philor)]

Disabled in slavealloc to stop the pointless stream of reboots.

Comment 15

[Van Le [:van]]

SD card formatted, tegra flashed and reimaged.

vle@vle-10516 ~ $ telnet tegra-192.tegra.releng.scl3.mozilla.com 20701
Trying 10.26.85.147...
Connected to tegra-192.tegra.releng.scl3.mozilla.com.
Escape character is '^]'.
$>^]
telnet> q

Comment 16

[Phil Ringnalda (:philor)]

Reenabled.