1476602 - [tc-references] ensure all references match their schema

Status: RESOLVED FIXED

(Taskcluster :: Services, enhancement, P5)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

api.json and exchanges.json should match the schema.  It'd be great to check that in individual services, but the schema isn't available at that time (it's in tc-references).  So let's check it at tc-references build time.

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

I think this will be easier when the as-yet-ill-defined "taskcluster-lib-references" is finished. as it will do most of the work of the tc-references build process (incorporating all of these schemas, references, and so on).

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

The idea with the library is to define an on-disk data format that contains a whole set of API references including a manifest.  Then tc-references can produce that as an artifact (even allow downloading it), and other things can consume it using the library.  It would probably be a multi-language library, or at least have some Go support for consuming the data format.

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

Pete, Brian, could you peek at the README in
  https://github.com/taskcluster/taskcluster-lib-references
and see if it makes sense?

My thinking is that one of these formats would form the input to the client-generation machinery.  Probably the as-yet-unwritten JSON format so it would be easy to download.

But mostly, this library will be used to help build and deploy taskcluster-references, including the abstract -> absolute transformation at deploy time (currently done with some sed scripts!).

Comment 4

[Brian Stack [:bstack]]

Looks good to me. This is also probably a nice step in the openapi/swagger direction I think we should go in eventually!

Comment 5

[Pete Moore [:pmoore][:pete]]

I've had an initial look, but I need to spend some more time on it, which I will do in the next days.

John, can you also take a look? Thanks!

Comment 6

[Pete Moore [:pmoore][:pete]]

Note, I've created https://github.com/taskcluster/taskcluster-rfcs/pull/128 as an overarching design of all the redeployability concerns.

Comment 7

[John Ford [:jhford] CET/CEST Berlin Time]

I've scheduled a meeting to discuss this system with Dustin tomorrow.  I'll comment afterwards, but I'll clear the needinfo for now.

Comment 8

[Dustin J. Mitchell [:dustin] (he/him)]

John, did you want to add some comments?  I'm sort of unable to proceed until I have written evidence we agree :)

Comment 9

[Dustin J. Mitchell [:dustin] (he/him)]

https://github.com/taskcluster/taskcluster-lib-references now handles this, as part of implementing RFC 128.