1047082 - Queue: Add schedulerId and taskGroupId (remove taskGraphId)

Status: RESOLVED FIXED

(Taskcluster :: Services, defect)

Description

[Jonas Finnemann Jensen (:jonasfj)]

Currently the queue requires the scope (A) to define a task and scope (B) to schedule the task:
 A) queue:post:define-task:<provisionerId>/<workerType>
 B) queue:post:schedule-task:<provisionerId>/<workerType>

Scope (A) certainly makes sense. But I'm thinking that, maybe a defined task should only be scheduled by the entity that defined it.

The idea is to replace scope (B) with:
  queue:assume:scheduler-id:<schedulerId>/<taskGroupId>

Then when calling `defineTask` a scheduler would provide `schedulerId` and `taskGroupId` in the `meta-data` section of the task definition.
With this approach task defined by the task-graph scheduler can't be scheduled by anybody else.

In addition to better scoping of permissions... The queue can also include schedulerId and taskGroupId in the well-defined section of routing keys for AMQP messages.
Right now it's <taskId>...<workerType>.<taskRoutingKey>, where if the task was submitted by
the task-graph scheduler <taskRoutingKey> will have been prefixed with <schedulerId><taskGraphId>.

With this approach the task-graph scheduler wouldn't have to prefix the `task.routing` key.
And the queue would only be aware of the concepts <schedulerId> and <taskGroupId>, but not
really have to deal with them other than including them in messages.

Note, I've deliberately called it <taskGraphId> on the task-graph scheduler, and <taskGroupId> on the queue. As the queue really doesn't care if it's a graph, set, list, tree, or cyclic graph of crazyness :)

Another benefit is that <schedulerId> and <taskGroupId> would be declared in the routing key definition and hence automatically parsed by taskcluster-client. So it won't be necessary to parse the task-specific routing key to get taskGraphId and schedulerId.

Comment 1

[Jonas Finnemann Jensen (:jonasfj)]

@lightsofapollo,
Let me know if this is totally crazy... I'm currently a bit concerned with not accidentally giving tasks more authority (scopes) than they need.
And I think this would allow us to separate permissions for defining tasks and scheduling tasks.

IMO, the task-graph scheduler should provide the authority to schedule tasks. But the caller for createTaskGraph should provide the authority/scopes to define tasks for a given provisionerId/workerType.

Anyways, let me know is this crazy?

Comment 2

[Jonas Finnemann Jensen (:jonasfj)]

From IRC, jlal gave an r+ on this.

In general it'll make me quite happy.

Comment 3

[Jonas Finnemann Jensen (:jonasfj)]

I've revisited this a bit. Realizing that the permission scope for `rerunTask` is equally stupid.

So a scope `queue:define-task:<provisionerId>/<workerType>` is necessary to create/define a task.
And then all tasks should define a schedulerId and taskGroupId. which forms the scope:
`queue:assume:scheduler-id:<schedulerId>/<taskGroupId>` which covers scheduleTask and rerunTask.

Hence, in order to schedule or rerun a task, you must have a scope for the given schedulerId and taskGroupId.
This is much better than using <provisionerId>/<workerType> for rerunTask and scheduleTask, because tasks
carries scopes, so only someone authorized to work on the task should be allowed to schedule/rerun it.

Comment 4

[Jonas Finnemann Jensen (:jonasfj)]

Merged in https://github.com/taskcluster/taskcluster-queue/pull/20