Closed Bug 1435635 Opened 7 years ago Closed 7 years ago

Run worker-killing script as a hook

Categories

(Taskcluster :: Workers, enhancement)

Product:
Taskcluster
Component:
Workers
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: dustin, Unassigned)

References

Details

There's a "kill script" running as a cron task, mentioned in bug 1435503. This or its friends have caused a few outages, and would probably be easier to diagnose if it was more visible. Can this be moved into a hook, so that we can see it execute and debug if necessary?
Flags: needinfo?(rthijssen)
probably (i'm not sure how hooks work). the only problem i foresee is that the instance which does the rebooting needs aws credentials or an iam role (enough access to view instance state and reboot instances). the script which currently does the rebooting is here: https://bugzilla.mozilla.org/attachment.cgi?id=8919820
Flags: needinfo?(rthijssen)
We should be able to set up an IAM user with those permissions and put it in the secrets store. Disclosure of those permissions would really only enable denial of service, so very low risk.
I ended up doing this in a fit of pique a few months ago - it's now "winstance-slayer" (https://tools.taskcluster.net/hooks/project-releng/winstance-slayer)
Status: NEW → RESOLVED
Closed: 7 years ago
QA Contact: pmoore
Resolution: --- → FIXED
just a note that the original cause for a requirement for this hook script (impaired windows 10 gpu instances, bug 1372172) has now been resolved. it may be that the winstance slayer is no longer required.
Component: Worker → Workers
You need to log in before you can comment on or make changes to this bug.