1428417 - [tracker] taskcluster clients must be able to interact with different clusters

Status: RESOLVED FIXED

(Taskcluster :: Services, enhancement)

Description

[Brian Stack [:bstack]]

At the moment, we expect this will work by setting some sort of environment variable or passing in a configuration option. This would probably be called TASKCLUSTER_BASE_URL or something along those lines and would contain a string that is the "root" of a cluster. At this root we will be running a reverse-proxy that sends requests to services based on the path. Examples:


> export TASKCLUSTER_BASE_URL="taskcluster.imbstack.com"

> const taskcluster = require('taskcluster-client');
> const auth = new taskcluster.Auth({});
> await auth.ping(); // would make a request to taskcluster.imbstack.com/auth/v1/ping

In addition to this, the client should be able to interact with the cluster even if it has new endpoints added or current endpoints have api changes. We have yet to 100% settle on a solution to this, but for language clients like Python/JS/etc it could involve being able to tell the client that all sub-clients created should be dynamically generated at runtime. Example:

> export TASKCLUSTER_BASE_URL="taskcluster.imbstack.com"

> const taskcluster = require('taskcluster-client');
> taskcluster.config({generateDynamicClients: true});
> const auth = new taskcluster.Auth({});
> await auth.newCrazyThing(); // would make a request to taskcluster.imbstack.com/auth/v1/newCrazyThing

The above example is not set in stone, just a guiding principle to follow. The clients that are for static languages would be rebuilt during the cluster build process. The dynamic language clients can/will also be rebuilt during that process but we allow them to be runtime-configurable to avoid needing to publish new clients for every development change and to avoid issues with circular dependencies during cluster bootstrapping.

Please add more of your thoughts here! I expect what I've written is incomplete/wrong!

Comment 1

[[github robot]]

Commit pushed to master at https://github.com/taskcluster/taskcluster-client

https://github.com/taskcluster/taskcluster-client/commit/0698805944e8adf81108cd61e0df741561a416c4
Merge pull request #87 from taskcluster/bug-1428417

Bug 1428417

Comment 2

[Brian Stack [:bstack]]

The first half of this work is now done. The work to allow easy dynamic client building will come later.

Comment 3

[Brian Stack [:bstack]]

We're starting to lose faith in our current plan for this work. We have scheduled a meeting on Monday to discuss the path forward.

Comment 4

[Brian Stack [:bstack]]

We've decided this actually works after all. Updates:


The env var is now `TASKCLUSTER_ROOT_URL`. We are going to use a library called taskcluster-lib-urls to generate paths based on that.

Comment 5

[Dustin J. Mitchell [:dustin] (he/him)]

Pete is currently working on this..

Comment 6

[Pete Moore [:pmoore][:pete]]

(In reply to Dustin J. Mitchell [:dustin] pronoun: he from comment #5)
> Pete is currently working on this..

RFC 128 has now been approved, and covers the design in detail:

https://github.com/taskcluster/taskcluster-rfcs/blob/rfc0128/rfcs/0128-redeployable-clients.md

Comment 7

[Pete Moore [:pmoore][:pete]]

Better link:

https://github.com/taskcluster/taskcluster-rfcs/blob/master/rfcs/0128-redeployable-clients.md

Comment 8

[Dustin J. Mitchell [:dustin] (he/him)]

Done! TC-CLI will get done when it moves to the monorepo.