It's looking like we're terminating the [worker scanner on too many timeouts](https://github.com/taskcluster/taskcluster/blob/66941613e4242d69dd2aff3fb560359eb633d59c/services/worker-manager/src/worker-scanner.js#L22-L37); same with the [provisioner](https://github.com/taskcluster/taskcluster/blob/66941613e4242d69dd2aff3fb560359eb633d59c/services/worker-manager/src/provisioner.js#L24-L39). (Same is true of the [queue claim resolver](https://github.com/taskcluster/taskcluster/blob/66941613e4242d69dd2aff3fb560359eb633d59c/services/queue/src/claimresolver.js#L55-L73) and [deadline resolver](https://github.com/taskcluster/taskcluster/blob/66941613e4242d69dd2aff3fb560359eb633d59c/services/queue/src/deadlineresolver.js#L59-L77), so it isn't specific to the worker manager) I'm wondering if the `process.exit(1)` combined with the GCE API timeouts or errors from the gcloud status page may have resulted in the worker-manager process(es) exiting and not restarting. Is this our desired behavior?
Bug 1734959 Comment 2 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
It's looking like we're terminating the [worker scanner on too many timeouts](https://github.com/taskcluster/taskcluster/blob/66941613e4242d69dd2aff3fb560359eb633d59c/services/worker-manager/src/worker-scanner.js#L22-L37); same with the [provisioner](https://github.com/taskcluster/taskcluster/blob/66941613e4242d69dd2aff3fb560359eb633d59c/services/worker-manager/src/provisioner.js#L24-L39). (Same is true of the [queue claim resolver](https://github.com/taskcluster/taskcluster/blob/66941613e4242d69dd2aff3fb560359eb633d59c/services/queue/src/claimresolver.js#L55-L73) and [deadline resolver](https://github.com/taskcluster/taskcluster/blob/66941613e4242d69dd2aff3fb560359eb633d59c/services/queue/src/deadlineresolver.js#L59-L77), so it isn't specific to the worker manager) I'm wondering if the `process.exit(1)` combined with the GCE API timeouts or errors from the gcloud status page may have resulted in the worker-manager process(es) exiting and not restarting. Is this our desired behavior? (if so, I wonder if we should separate AWS from Azure from GCP in terms of total number of errors?)