Closed Bug 853605 (b-2008-ix-0018) Opened 10 years ago Closed 7 years ago

b-2008-ix-0018 problem tracking

Categories

(Infrastructure & Operations Graveyard :: CIDuty, task, P3)

x86_64
Windows Server 2008

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: armenzg, Unassigned)

References

Details

(Whiteboard: [buildduty][buildslaves][capacity])

I can't get it to reboot even through IPMI.
IPMI was responding fine and the machine was rebooting.  The problem was a missing interface entry in inventory for the public interface (so it was getting a randomly assigned IP from the dynamic pool).  I added the interface info and rebooted it again and it's up now.
No longer blocks: 853153
:jhopkins fixed the password; I adjusted the keys for safety, reenabled in slavealloc (staging) and rebooted.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
Depends on: 906245
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
loaner is over, I just unlocked it from coop's environ, marked it as "core" trust level build-scl3 pool and left it disabled until the (to file) reimage is done.
Depends on: 929235
(In reply to Justin Wood (:Callek) from comment #3)
> loaner is over, I just unlocked it from coop's environ, marked it as "core"
> trust level build-scl3 pool

Err the reimage is done, its rev2; we just need to bringup keys/etc now, then enable in slavealloc
I'm going to take this machine and use it as a rev2 try slave.
Assignee: nobody → jhopkins
Status: REOPENED → RESOLVED
Closed: 10 years ago9 years ago
Resolution: --- → FIXED
So if it's a rev2 slave now, it doesn't have an E drive, right? It's been alerting in #buildduty about not being able to check the free space on E: for... well, probably since October 30th, or October 21st.
Status: RESOLVED → REOPENED
Flags: needinfo?(jhopkins)
Resolution: FIXED → ---
That's correct - there is no drive E: on rev2 slaves.  Requested move to w64r2-ix-slaves nagios host group in https://bugzilla.mozilla.org/show_bug.cgi?id=920667#c26
Flags: needinfo?(jhopkins)
Moved to try-aws-us-west-2-rev2 pool in slavealloc and rebooted.
Status: REOPENED → RESOLVED
Closed: 9 years ago9 years ago
Resolution: --- → FIXED
Loaning.
Status: RESOLVED → REOPENED
Depends on: 970509
Resolution: FIXED → ---
Assignee: jhopkins → nobody
I added the staging tbirdbld key to this so Fallen can upload to dev-stage01.
Depends on: 997339
back in production
Status: REOPENED → RESOLVED
Closed: 9 years ago9 years ago
Resolution: --- → FIXED
And since then, it has failed every job. Don't have a good way to find any of the logs right now, but my bet would be that it's failing by not having keys, that being the usual way.

Disabled in slavealloc.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
QA Contact: armenzg → bugspam.Callek
Command ['ssh', '-o', 'IdentityFile=~/.ssh/trybld_dsa', 'trybld@stage.mozilla.org', 'mktemp -d'] returned non-zero exit code: -1
Depends on: 1011499
Alias: w64-ix-slave03 → b-2008-ix-0018
Summary: w64-ix-slave03 problem tracking → b-2008-ix-0018 problem tracking
formatted sd card, reimaged and flashed tegra

[vle@admin1a.private.scl3 ~]$  telnet tegra-073.tegra.releng.scl3.mozilla.com 20701
Trying 10.26.85.53...
Connected to tegra-073.tegra.releng.scl3.mozilla.com.
Escape character is '^]'.
$>^]q

telnet> q
err, ignore comment 14... updated wrong bug
Attempting SSH reboot...Failed.
Attempting IPMI reboot...Failed.
Filed IT bug for reboot (bug 1018019)
Ignore comment 16, too - the things to focus on are "needs keys" and "needs keys" and "needs the try key" and "needs keys" :)
Just tried to ssh in and fix said keys, but host is down....
keys added, rebooted to "try"

[note for posterity I was having trouble adding production keys to this host, because it is not a "PRODUCTION" host. I'm now really really happy we have the VLAN split.]
Status: REOPENED → RESOLVED
Closed: 9 years ago9 years ago
Resolution: --- → FIXED
Depends on: 1180843
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Re-imaged and returned to production (try).
Status: REOPENED → RESOLVED
Closed: 9 years ago7 years ago
Resolution: --- → FIXED
allocated to bug 1198317
Status: RESOLVED → REOPENED
Depends on: 1198317
Resolution: FIXED → ---
deallocated from bug 1198317
Status: REOPENED → RESOLVED
Closed: 7 years ago7 years ago
Resolution: --- → FIXED
Product: Release Engineering → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.