Bug 1095059 (b-2008-ix-0078)

b-2008-ix-0078 problem tracking

RESOLVED FIXED

Status

Release Engineering
Buildduty
P3
normal
RESOLVED FIXED
3 years ago
a month ago

People

(Reporter: RyanVM, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [buildduty][buildslaves][capacity])

(Reporter)

Description

3 years ago
Corrupted builds and an awful-looking recent health. Disabled for reimaging.
https://treeherder.mozilla.org/ui/logviewer.html#?job_id=379008&repo=mozilla-aurora

Updated

3 years ago
Depends on: 1095137
Diagnostics saw nothing, maybe the reimage helped. Reenabled.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
(Reporter)

Comment 2

3 years ago
Still seems pretty burntastic to me. I've seen a couple like the one below since it was put back into production:
https://treeherder.mozilla.org/ui/logviewer.html#?job_id=31332&repo=mozilla-b2g34_v2_1
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Comment 3

3 years ago
Re-imaged. If we still see failures, we'll try another pass of diagnostics.
Status: REOPENED → RESOLVED
Last Resolved: 3 years ago3 years ago
Resolution: --- → FIXED
(Reporter)

Comment 4

3 years ago
https://treeherder.mozilla.org/ui/logviewer.html#?job_id=59297&repo=mozilla-b2g34_v2_1

 LINK : fatal error LNK1123: failure during conversion to COFF: file invalid or corrupt
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Comment 5

2 years ago
Re-imaged and returned to production.
Status: REOPENED → RESOLVED
Last Resolved: 3 years ago2 years ago
Resolution: --- → FIXED
Status: RESOLVED → REOPENED
Depends on: 1026870
Resolution: FIXED → ---

Comment 6

2 years ago
Re-imaged and re-enabled.
Status: REOPENED → RESOLVED
Last Resolved: 2 years ago2 years ago
Resolution: --- → FIXED
This host has been showing as down for 33 days. I'm not sure why slaveapi hasn't filed a bug for it?
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
It's now a staging slave, in the dev/pp environment and attached to dev-bhearsum-builds, so slaveapi has no reason to touch it.
Depends on: 1139927
Attempting SSH reboot...Failed.
Attempting IPMI reboot...Failed.
Filed IT bug for reboot (bug 1139927)
Re-imaged and returned to the production environment.
Status: REOPENED → RESOLVED
Last Resolved: 2 years ago2 years ago
Resolution: --- → FIXED
(Reporter)

Updated

2 years ago
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Comment hidden (Treeherder Robot)
Depends on: 1165457
(Reporter)

Comment 12

2 years ago
This is burning Windows nightly jobs:
https://treeherder.mozilla.org/logviewer.html#?job_id=877823&repo=mozilla-aurora

I've disabled it in slavealloc.
(Reporter)

Updated

2 years ago
Flags: needinfo?(mcornmesser)
I am looking into this. I am not seeing a clear cut reason for the failure. There is the "03:48:52 INFO - IOError: [Errno 13] Permission denied: 'c:/builds/crash-stats-api.token'" error, but the permissions are consistent with other machines. 

RyanVM: any thoughts on why it burned?
Flags: needinfo?(mcornmesser) → needinfo?(ryanvm)
(Reporter)

Comment 14

2 years ago
Nope. I asked Ted about it and he said it was a slave issue.
Flags: needinfo?(ryanvm)
Are the running processes consistent with other machines? "I downloaded a file, and when I went to use it I got a permission denied error" on a Windows machine, my first thought is AV or search indexing grabbing onto the newly-created file, including maybe something that is running on other machines too, but on other machines has been told to keep its hands off c:/builds/.
This machine, due to troubleshooting, had puppet ran by 2 different local accounts, system and cltbld. I suspect that may have caused an issue with permissions.  I am going to reimage the machine and reenable it.
No longer depends on: 1165457
Depends on: 1165771
Re-imaged slave, did not enable it in slavealloc.
Status: REOPENED → RESOLVED
Last Resolved: 2 years ago2 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.