1398084 - Generic worker exception kills all workers

Status: RESOLVED FIXED

(Taskcluster :: Workers, defect)

Description

[Franziskus Kiefer [:franziskus]]

Our NSS Mac generic worker ran into an error with one job

2017/09/08 06:12:08 ERROR encountered: Task not successful due to following exception(s):
Exception 1)
(Permanent) HTTP response code 400
HTTP/1.1 400 Bad Request^M
Connection: close^M
Transfer-Encoding: chunked^M
Content-Type: application/xml^M
Date: Fri, 08 Sep 17 10:10:15 GMT^M

Following this error all other jobs were marked as failed.

Comment 1

[Pete Moore [:pmoore][:pete]]

Hi Franziskus,

Sorry to hear about this. Do you have more logs, and the taskId where this occurred?

Thanks,
Pete

Comment 2

[Pete Moore [:pmoore][:pete]]

It looks like this was due to "idleTimeoutSecs" generic-worker configuration property being set to 345600 instead of 0. I've updated the 10 worker generic-worker.config files for worker[0..9] to remove this property, so that the default value of 0 is used (which means worker runs forever).

I believe this should solve it.

It might not be a bad thing to reboot once a week or so though.