1544983 - TASKCLUSTER_WORKER_GROUP is not set on Windows workers

Status: NEW

(Taskcluster :: Workers, defect)

Description

[Mike Hommey [:glandium]]

I was trying to set up run-task on Windows... and that failed because TASKCLUSTER_WORKER_GROUP is not set.

Comment 1

[Pete Moore [:pmoore][:pete]]

Looking at docker-worker codebase, it looks like docker-worker sets four environment variables that generic-worker doesn't set:

1. TASKCLUSTER_WORKER_TYPE

The task provisioner ID is not set in an environment variable by docker-worker so I'm not sure of the value of setting TASKCLUSTER_WORKER_TYPE in isolation, since worker type is not unique across provisioner IDs. Therefore I don't think it makes sense to port this to generic-worker at the moment, unless we also set an environment variable for the provisioner ID too. Note, the worker type and provisioner ID are available from the task definition, if the task really wants to know, although it is a network call to fetch the task definition from inside the task.

2. TASKCLUSTER_INSTANCE_TYPE

The naming here is pretty specific to AWS, for example, what would we set this to on a mac mini? I suspect we don't need this either in generic-worker.

3. TASKCLUSTER_WORKER_GROUP

The worker group has less significance under worker manager, where for at least aws and google it is just the provider ID. The worker group provides a namespace for the worker ID, in much the same way that provisioner ID is a namespace for worker type - the unique identifier for a worker is <workerGroup>/<workerId>, so providing worker group as an environment variable, but not providing an environment variable for workerId seems strange. But it looks like, since run-task broke (comment 0) that some code is relying on it.

4. TASKCLUSTER_PUBLIC_IP

Not all workers have a public IP, so it seems strange to provide it. I'm not sure what value it is to a task, when a public IP cannot be guaranteed.

---

In summary, I'm tempted not to add any of these environment variables to task environments, since they either provide incomplete information, irrelevant information, and in all cases, information that is available via queue.status and queue.task endpoints.

Comment 2

[Pete Moore [:pmoore][:pete]]

Mike, the only place I see TASKCLUSTER_WORKER_GROUP referenced in gecko is in mozconfig.cache. Let me know if I missed something!

Can you confirm if TASKCLUSTER_WORKER_GROUP is still needed, or if TASKCLUSTER_WORKER_LOCATION (from RFC 148) now provides all the information needed?

Thanks!

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

In summary, I'm tempted not to add any of these environment variables to task environments, since they either provide incomplete information, irrelevant information, and in all cases, information that is available via queue.status and queue.task endpoints.

I agree -- TASKCLUSTER_WORKER_LOCATION is the preferred variable to use here.

Comment 4

[Mike Hommey [:glandium]]

TASKCLUSTER_WORKER_GROUP was used in run-task when I filed this bug. That changed in bug 1596479.

Comment 5

[Pete Moore [:pmoore][:pete]]

Hey Mike, does that mean we can close this bug now, since TASKCLUSTER_WORKER_LOCATION can now be used instead in mozconfig.cache?

Comment 6

[Mike Hommey [:glandium]]

I guess.

Comment 7

[Pete Moore [:pmoore][:pete]]

Not actively working on this right now.