Closed
Bug 1095059
(b-2008-ix-0078)
Opened 10 years ago
Closed 9 years ago
b-2008-ix-0078 problem tracking
Categories
(Infrastructure & Operations Graveyard :: CIDuty, task, P3)
Infrastructure & Operations Graveyard
CIDuty
x86_64
Windows Server 2008
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: RyanVM, Unassigned)
References
Details
(Whiteboard: [buildduty][buildslaves][capacity])
Corrupted builds and an awful-looking recent health. Disabled for reimaging. https://treeherder.mozilla.org/ui/logviewer.html#?job_id=379008&repo=mozilla-aurora
Comment 1•10 years ago
|
||
Diagnostics saw nothing, maybe the reimage helped. Reenabled.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Reporter | ||
Comment 2•10 years ago
|
||
Still seems pretty burntastic to me. I've seen a couple like the one below since it was put back into production: https://treeherder.mozilla.org/ui/logviewer.html#?job_id=31332&repo=mozilla-b2g34_v2_1
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Comment 3•10 years ago
|
||
Re-imaged. If we still see failures, we'll try another pass of diagnostics.
Status: REOPENED → RESOLVED
Closed: 10 years ago → 10 years ago
Resolution: --- → FIXED
Reporter | ||
Comment 4•10 years ago
|
||
https://treeherder.mozilla.org/ui/logviewer.html#?job_id=59297&repo=mozilla-b2g34_v2_1 LINK : fatal error LNK1123: failure during conversion to COFF: file invalid or corrupt
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Comment 5•9 years ago
|
||
Re-imaged and returned to production.
Status: REOPENED → RESOLVED
Closed: 10 years ago → 9 years ago
Resolution: --- → FIXED
Updated•9 years ago
|
Comment 6•9 years ago
|
||
Re-imaged and re-enabled.
Status: REOPENED → RESOLVED
Closed: 9 years ago → 9 years ago
Resolution: --- → FIXED
Comment 7•9 years ago
|
||
This host has been showing as down for 33 days. I'm not sure why slaveapi hasn't filed a bug for it?
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Comment 8•9 years ago
|
||
It's now a staging slave, in the dev/pp environment and attached to dev-bhearsum-builds, so slaveapi has no reason to touch it.
Comment 9•9 years ago
|
||
Attempting SSH reboot...Failed. Attempting IPMI reboot...Failed. Filed IT bug for reboot (bug 1139927)
Comment 10•9 years ago
|
||
Re-imaged and returned to the production environment.
Status: REOPENED → RESOLVED
Closed: 9 years ago → 9 years ago
Resolution: --- → FIXED
Reporter | ||
Updated•9 years ago
|
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Comment hidden (Legacy TBPL/Treeherder Robot) |
Reporter | ||
Comment 12•9 years ago
|
||
This is burning Windows nightly jobs: https://treeherder.mozilla.org/logviewer.html#?job_id=877823&repo=mozilla-aurora I've disabled it in slavealloc.
Reporter | ||
Updated•9 years ago
|
Flags: needinfo?(mcornmesser)
Comment 13•9 years ago
|
||
I am looking into this. I am not seeing a clear cut reason for the failure. There is the "03:48:52 INFO - IOError: [Errno 13] Permission denied: 'c:/builds/crash-stats-api.token'" error, but the permissions are consistent with other machines. RyanVM: any thoughts on why it burned?
Flags: needinfo?(mcornmesser) → needinfo?(ryanvm)
Reporter | ||
Comment 14•9 years ago
|
||
Nope. I asked Ted about it and he said it was a slave issue.
Flags: needinfo?(ryanvm)
Comment 15•9 years ago
|
||
Are the running processes consistent with other machines? "I downloaded a file, and when I went to use it I got a permission denied error" on a Windows machine, my first thought is AV or search indexing grabbing onto the newly-created file, including maybe something that is running on other machines too, but on other machines has been told to keep its hands off c:/builds/.
Comment 16•9 years ago
|
||
This machine, due to troubleshooting, had puppet ran by 2 different local accounts, system and cltbld. I suspect that may have caused an issue with permissions. I am going to reimage the machine and reenable it.
Comment 17•9 years ago
|
||
Re-imaged slave, did not enable it in slavealloc.
Status: REOPENED → RESOLVED
Closed: 9 years ago → 9 years ago
Resolution: --- → FIXED
Updated•6 years ago
|
Product: Release Engineering → Infrastructure & Operations
Updated•4 years ago
|
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in
before you can comment on or make changes to this bug.
Description
•