1045246 - Queue, scheduler: version number in tasks and task-graphs ?

Status: RESOLVED FIXED

(Taskcluster :: General, defect)

Description

[Jonas Finnemann Jensen (:jonasfj)]

With the latest iteration of taskcluster-queue, I moved from "x.y.z" version numbers in the task definition to integers on the form: `version: 1`.

But starting on the task-graph scheduler I realize that we probably don't want version numbers of tasks and task-graphs as these are no longer formats you'll to store on disk..
They are more like exchange formats, where you submit something that has already been initialization from a disk format containing labels and what not.

I still think it makes sense to maintain version numbers on the AMQP messages, even though we have "v1/" in the exchange names "queue/v1/task-..." as a version number can be used to indicate which properties can be assumed to be available.

If an old queue still lives with the same exchange prefix, then old message might still appear, or they could (theoretically) be stored in a queue somewhere. So consumers of message should be prepared to handle old message versions (when we add new properties).
If we break backwards compatibility (ie. remove properties) we'll move to a new exchange prefix).

Anyways, since tasks and task-graphs are always submitted through a url with v1/ encoded in it. It seems we don't need to have a version number in the JSON format.

Any thoughts, disagree?

Comment 1

[Jonas Finnemann Jensen (:jonasfj)]

This should be trivial to fix, I'll probably do in the scheduler now... Please let me know if you any objections.
If not I'll do the same for the queue.

Comment 2

[James Lal [:lightsofapollo]]

sounds good to me

Comment 3

[Jonas Finnemann Jensen (:jonasfj)]

Fixed in revision f3f6fca, now pushed to master to be auto-deployed as tests are completed.

Note, the task definition stored in azure blob storage still features a version number hardcoded to 1.
But this is a purely internal version number, that will not be returned from the queue or visible to consumers.
Basically, it just ensures that if we, in the future, decide to modify the task definition then we can upgrade old task definitions on-the-fly (ie. in the GET /v1/task/<taskId> request handler on the queue)...

Comment 4

[Pete Moore [:pmoore][:pete]]

Resetting Version and Target Milestone that accidentally got changed...