Status

Infrastructure & Operations
RelOps
RESOLVED FIXED
7 years ago
5 years ago

People

(Reporter: bear, Assigned: MaRu)

Tracking

Details

(Whiteboard: [android][tegra])

(Reporter)

Description

7 years ago
please pull tegra-102, tegra-106, tegra-107 to find out why they are not allowing network connections.

all refuse connections from the network.
all have had multiple pdu resets.
ping responses:

PING tegra-102.build.mtv1.mozilla.com (10.250.49.90): 56 data bytes
64 bytes from 10.250.49.90: icmp_seq=0 ttl=64 time=1.147 ms

PING tegra-106.build.mtv1.mozilla.com (10.250.49.94): 56 data bytes
Request timeout for icmp_seq 0

PING tegra-107.build.mtv1.mozilla.com (10.250.49.95): 56 data bytes
Request timeout for icmp_seq 0
(Reporter)

Updated

7 years ago
Whiteboard: [android][tegra]
Hi, bear, is this critical, or can this wait till after zandr gets back on the 10th?  I'm not sure we have anyone on mtv who can debug the tegra boards right now.
(Reporter)

Comment 2

7 years ago
(In reply to Amy Rich [:arich] from comment #1)
> Hi, bear, is this critical, or can this wait till after zandr gets back on
> the 10th?  I'm not sure we have anyone on mtv who can debug the tegra boards
> right now.

Not critical at all.  If its possible if someone could confirm the mac/dhcp/dns setup that would be good, but if not then it can wait.
(Reporter)

Comment 3

7 years ago
add the following to the list to look at:

tegra-012
tegra-028
tegra-033
tegra-034

I'm taking these offline completely now
Summary: 3 new tegras are not responding to network connects → tegra recovery
(Reporter)

Comment 4

7 years ago
(sigh)

remove tegra-033 from the list, silly bear can't read
(Reporter)

Comment 5

7 years ago
tegra-115
(Reporter)

Comment 6

7 years ago
(In reply to Mike Taylor [:bear] from comment #5)
> tegra-115

bah! 10 minutes after I add this one to the bug it comes back online.  keeping it in the bug so we check for any issues with the screen attached
(Reporter)

Comment 7

7 years ago
tegra-013 - possible bad sdcard
(Reporter)

Comment 8

7 years ago
tegra-122
tegra-123
(Reporter)

Comment 9

7 years ago
tegra-039
To be clear, do you just want us to reimage?  That can be done by following aki's instructions, so we could have someone else look at them.  If you want actual debugging, though, that's either someone from releng or zandr, and he's going to be a constrained resource when he gets back from vacation.
I heard from bear while on vacation, and he says reimaging is fine.
(Assignee)

Comment 12

7 years ago
Reformatted all the SD cards
Ran ./nvflash_crt.sh and ./initialize_tegra.sh <tegra-name> then renetworked and powered devices.

Imaged:
tegra-102          
tegra-106          
tegra-107          
tegra-013          
tegra-115   
tegra-012   
tegra-028   

Cannot find these in Haxxor, can someone enlighten me as to there location so I can finish this bug? Thanks :)
tegra-034
tegra-039
tegra-122
tegra-123

Comment 13

7 years ago
tegra-033 appears to not have an sdcard mounted :(
https://bugzilla.mozilla.org/show_bug.cgi?id=692346
I don't think 122 and 123 were put into service.  Zandr can chime in on whether they went to developers (I thought 123 did at the very least and inventory seems to back that up) or where they wound up.
(In reply to Matthew Larrain[:digipengi] from comment #12)

> Cannot find these in Haxxor, can someone enlighten me as to there location
> so I can finish this bug? Thanks :)
> tegra-034
> tegra-039

2/IDF. Be *very* careful pulling these out. The ACOK switch is on the *bottom* as these are resting on the shelves, so if you bump an adjacent one, they won't come back from their next power cycle (or maybe even reboot). Also, you will be sad if the cables slip down the back of the rack, so pull the board out of the front of the rack before you disconnect the cables, and leave the cables hanging off the front of the shelf.

> tegra-122
> tegra-123

These were given away to developers.
(Assignee)

Comment 16

7 years ago
Fixed these and put them back into production:

tegra-034
tegra-039
tegra-033 Had a bad SDcard replaced it and working fine now
Assignee: server-ops-releng → mlarrain
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED
Component: Server Operations: RelEng → RelOps
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.