Closed
Bug 843744
(panda-0081)
Opened 11 years ago
Closed 9 years ago
Decommission panda-0081
Categories
(Infrastructure & Operations :: DCOps, task, P3)
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: kmoir, Unassigned)
References
()
Details
(Whiteboard: [buildduty][buildslaves][capacity])
DC ops is looking at this panda, as it's very flaky. See bug 817103 for more details
Reporter | ||
Comment 1•11 years ago
|
||
Also forgot to mention that I chmoded og-r the panda-0081 directory on foopy85 so it would stay out of commission
Reporter | ||
Comment 2•11 years ago
|
||
echo "bug 843744" > /builds/panda-0081/disabled.flg is a better way to disable a panda
Comment 3•11 years ago
|
||
The issue that first caused this panda to fail (5 months ago) might have been an issue with the automation process and not really a problem with the panda board itself. Since I really didn't find anything wrong with this panda board, I issued a reimage via mozpool.
Comment 4•11 years ago
|
||
Resolving all panda bugs linked from Bug 817103 that are not in troubleshoot or failed_pxe* state in lifeguard.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Updated•11 years ago
|
Product: mozilla.org → Release Engineering
Comment 5•11 years ago
|
||
pdu reboot didn't help, needs recovery
Comment 6•11 years ago
|
||
Sending this slave to recovery -->Automated message.
Comment 7•11 years ago
|
||
panda-081 - selftest.py[INFO]: test_preseed_file_integrity[FAILED] boot.scr : 5a5c34aa07d2d8f23e1b69347d49bacf205041dd != 6261fdd19a45db13e6503c5010e3917dbb13eeed fix - replaced SD card with correct preseed.
Comment 8•11 years ago
|
||
Back in production
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED
Comment 9•10 years ago
|
||
Attempting SSH reboot...Failed. Attempting reboot via Mozpool...Failed. Filed IT bug for reboot (bug 1067672)
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Comment 10•10 years ago
|
||
Didn't recover, hasn't taken a job for 26 days.
QA Contact: armenzg → bugspam.Callek
Comment 11•10 years ago
|
||
Panda was added to the proper foopy and is now taking jobs.
Status: REOPENED → RESOLVED
Closed: 11 years ago → 10 years ago
Resolution: --- → FIXED
Comment 12•10 years ago
|
||
Failing every other job, disabled in slavealloc.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Comment 13•10 years ago
|
||
Tools repo updated on foopy so that this panda can be properly rebooted again.
Status: REOPENED → RESOLVED
Closed: 10 years ago → 10 years ago
Resolution: --- → FIXED
Comment 14•10 years ago
|
||
Since then, 2 green, 1 orange, 17 retry, 0 jobs for the last 4 days. Not sure this was a good choice for backfill. Disabled.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Comment 15•10 years ago
|
||
replaced SD card, panda passed self test
Comment 16•10 years ago
|
||
Reenabled and rebooted.
Status: REOPENED → RESOLVED
Closed: 10 years ago → 10 years ago
Resolution: --- → FIXED
Comment 17•10 years ago
|
||
Attempting SSH reboot...Failed. Attempting reboot via Mozpool...Failed. Filed IT bug for reboot (bug 1082738)
Updated•10 years ago
|
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Comment 18•10 years ago
|
||
Attempting SSH reboot...Failed. Attempting reboot via Mozpool...Failed. Filed IT bug for reboot (bug 1083006)
Comment 19•10 years ago
|
||
replaced SD card, panda passed self test. panda-0081 ready (request 2313518)
Comment 20•10 years ago
|
||
Score for the 2014-10-10 SD card: 11 retries, 2 failures, 5 passes. Score for the 2014-10-20 SD card: 3 retries, 2 failures, 0 passes (and I'm already calling that a complete score since it hasn't taken a job for three days now). It ain't the SD card. Disabled in slavealloc so we can skip another pointless card replacement the next time it fails to reboot.
Comment 21•10 years ago
|
||
Let's decomm this one.
Assignee: nobody → server-ops-dcops
Component: Buildduty → Server Operations: DCOps
Product: Release Engineering → mozilla.org
QA Contact: bugspam.Callek → dmoore
Summary: panda-0081 problem tracking → Decommission panda-0081
Updated•10 years ago
|
colo-trip: --- → scl3
Updated•10 years ago
|
Assignee: server-ops-dcops → nobody
Component: Server Operations: DCOps → Server Operations: MOC
Comment 22•10 years ago
|
||
Hey MOC team can you remove this host from nagios before I physically decomm it? Thanks.
Comment 23•9 years ago
|
||
(In reply to Vinh Hua [:vinh] from comment #22) > Hey MOC team can you remove this host from nagios before I physically decomm > it? Thanks. Panda's are commented in nagios. Can you update the status of the bug ?
Flags: needinfo?(vhua)
Updated•9 years ago
|
Assignee: nobody → server-ops
Component: Server Operations: MOC → Server Operations
Updated•9 years ago
|
Assignee: server-ops → server-ops-dcops
Component: Server Operations → DCOps
Product: mozilla.org → Infrastructure & Operations
Comment 24•9 years ago
|
||
Panda-0081 has been physically decomm'd.
Status: REOPENED → RESOLVED
Closed: 10 years ago → 9 years ago
Flags: needinfo?(vhua)
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•