787622 - (tegra-072) tegra-072 problem tracking

Status: RESOLVED WONTFIX

(Infrastructure & Operations Graveyard :: CIDuty, task, P3)

Description

[Justin Wood (:Callek)]

Needs recovery.

:joel, is there anything beyond simple reimage that we (or DCOps) could/should do for this one.

Its failing verifying the watcher.ini for similar reasons to what was up with your panda probs:

$>exec ls /data/data/com.mozilla.watcher
lib
files
return code [0]
$>exec ls -l /data/data/com.mozilla.watcher
drwxr-xr-x system   system                      1970-01-05 10:55 lib
drwxrwx--x app_32   app_32                      1970-01-01 00:03 files
return code [0]
$>exec ls -l /data/data/com.mozilla.watcher/files
---x-w---T root     app_31                 68 1970-01-05 06:03 watcher.ini
return code [0]
$>hash /data/data/com.mozilla.watcher/files/watcher.ini
##AGENT-WARNING## Couldn't calculate hash for file /data/data/com.mozilla.watcher/files/watcher.ini file not found
$>

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

For the tegras this works on all the other ones, lets just reimage this.  I assume there are some file permission problems with the way the watcher or sutagent are installed.  Not sure if that is something that happened during imaging of the OS or of something happened in the field.

Comment 2

[Kim Moir [:kmoir] ET]

This tegra has been running builds okay so I'm going to close this bug.

Comment 3

[Justin Wood (:Callek)]

Last job 20 days, 1:03:01 ago

error.flg [Remote Device Error: Unable to properly remove /mnt/sdcard/tests]

remotely formatted sdcard

Comment 4

[Armen [:armenzg]]

This seems to have been taking jobs.

Comment 5

[[e-mail] (:bhearsum)]

Had to manually reformat its sdcard today, pdu reboot, and clear the error flag. It's back in production though.

Comment 6

[Justin Wood (:Callek)]

We manually reformatted the sdcard a few too many times, please swap it with this reimage.

Comment 7

[[e-mail] (:bhearsum)]

Hasn't run a job in 40 days, 20:09:06

Comment 8

[[e-mail] (:bhearsum)]

pdu reboot didn't work

Comment 9

[[e-mail] (:bhearsum)]

back in production

Comment 10

[Phil Ringnalda (:philor)]

Broken phone.

Comment 11

[Ed Morley [:emorley]]

Tegra kindly disabled by Callek.

Comment 12

[Justin Wood (:Callek)]

Sending this slave to recovery
-->Automated message.

Comment 13

[Justin Wood (:Callek)]

Successfully recovered.

Comment 14

[Justin Wood (:Callek)]

Friday, November 08, 2013 2:55:49 AM

Comment 15

[Eric Ramirez [:Eric]]

SD card has been reimaged/flashed.

Comment 16

[Justin Wood (:Callek)]

handled in last recovery on Dec 16.

Comment 17

[[e-mail] (:bhearsum)]

2014-01-16 10:24:31 tegra-072 p    online   active  OFFLINE :: error.flg [Remote Device Error: unable to write to sdcard] 

pdu reboot didn't help

Comment 18

[Eric Ramirez [:Eric]]

SD card reformatted & flashed.

Comment 19

[[e-mail] (:bhearsum)]

(In reply to Eric Ramirez [:Eric] from comment #18)
> SD card reformatted & flashed.

can we try again?

Comment 20

[Van Le [:van]]

SD card formatted, tegra reimaged and flashed.

[[e-mail]3 ~]$ fping tegra-072.tegra.releng.scl3.mozilla.com
tegra-072.tegra.releng.scl3.mozilla.com is alive

Comment 21

[John Hopkins (:jhopkins)]

Power cycled, waited a day.

error.flg: Automation Error: Unable to connect to device after 5 attempts

Tried reformatting:

$>exec newfs_msdos -F 32 /dev/block/vold/179:9
newfs_msdos: /dev/block/vold/179:9: No such file or directory
return code [1]

Comment 22

[John Hopkins (:jhopkins)]

Given that no builds were triggered after the last SD card replacement (according to slave health build history), I suggest we decomm this tegra.

Comment 23

[Phil Ringnalda (:philor)]

Disabled in slavealloc to stop the pointless stream of reboots.

Comment 24

[Van Le [:van]]

formatted SD card, flashed and reimaged tegra.

vle@vle-10516 ~ $ telnet tegra-072.tegra.releng.scl3.mozilla.com 20701
Trying 10.26.85.52...
Connected to tegra-072.tegra.releng.scl3.mozilla.com.
Escape character is '^]'.
$>^]
telnet> q

Comment 25

[Phil Ringnalda (:philor)]

Oh, comment 22 (though in fact it did take exactly one job three weeks after that, so total of one between February and July).

Comment 26

[Chris Cooper [:coop] (he/him)]

All tegras are being decommissioned.