Corrupted builds and an awful-looking recent health. Disabled for reimaging. https://treeherder.mozilla.org/ui/logviewer.html#?job_id=379008&repo=mozilla-aurora
Diagnostics saw nothing, maybe the reimage helped. Reenabled.
Still seems pretty burntastic to me. I've seen a couple like the one below since it was put back into production: https://treeherder.mozilla.org/ui/logviewer.html#?job_id=31332&repo=mozilla-b2g34_v2_1
Re-imaged. If we still see failures, we'll try another pass of diagnostics.
https://treeherder.mozilla.org/ui/logviewer.html#?job_id=59297&repo=mozilla-b2g34_v2_1 LINK : fatal error LNK1123: failure during conversion to COFF: file invalid or corrupt
Re-imaged and returned to production.
Re-imaged and re-enabled.
This host has been showing as down for 33 days. I'm not sure why slaveapi hasn't filed a bug for it?
It's now a staging slave, in the dev/pp environment and attached to dev-bhearsum-builds, so slaveapi has no reason to touch it.
Attempting SSH reboot...Failed. Attempting IPMI reboot...Failed. Filed IT bug for reboot (bug 1139927)
Re-imaged and returned to the production environment.
This is burning Windows nightly jobs: https://treeherder.mozilla.org/logviewer.html#?job_id=877823&repo=mozilla-aurora I've disabled it in slavealloc.
I am looking into this. I am not seeing a clear cut reason for the failure. There is the "03:48:52 INFO - IOError: [Errno 13] Permission denied: 'c:/builds/crash-stats-api.token'" error, but the permissions are consistent with other machines. RyanVM: any thoughts on why it burned?
Nope. I asked Ted about it and he said it was a slave issue.
Are the running processes consistent with other machines? "I downloaded a file, and when I went to use it I got a permission denied error" on a Windows machine, my first thought is AV or search indexing grabbing onto the newly-created file, including maybe something that is running on other machines too, but on other machines has been told to keep its hands off c:/builds/.
This machine, due to troubleshooting, had puppet ran by 2 different local accounts, system and cltbld. I suspect that may have caused an issue with permissions. I am going to reimage the machine and reenable it.
Re-imaged slave, did not enable it in slavealloc.