1246643 - Auto generate global config go struct using go-composite-schema

Status: RESOLVED FIXED

(Taskcluster :: Workers, defect)

Description

[Pete Moore [:pmoore][:pete]]

The global config will be built up something like this:

struct Config {
    struct Global {
        ....
    }
    struct Engines {
        struct Docker {
            ....
        }
        ....
    }
    struct Plugins {
        struct LiveLog {
            ....
        }
        ....
    }
}

We would build the go types with a buildtime discovery of plugins and engines.

Each engine and plugin will define its own config json schema which can be used to generate the static model for its configuration, ensuring static type compile-time compliance for code that references config settings.

Comment 1

[Pete Moore [:pmoore][:pete]]

(In reply to Pete Moore [:pmoore][:pete] from comment #0)
> The global config will be built up something like this:
> 
> struct Config {
>     struct Global {
>         ....
>     }
>     struct Engines {
>         struct Docker {
>             ....
>         }
>         ....
>     }
>     struct Plugins {
>         struct LiveLog {
>             ....
>         }
>         ....
>     }
> }
> 
> We would build the go types with a buildtime discovery of plugins and
> engines.
> 
> Each engine and plugin will define its own config json schema which can be
> used to generate the static model for its configuration, ensuring static
> type compile-time compliance for code that references config settings.

We've decided against this now, since this would create a dependency cycle. For example, plugins would depend on the package the config structure is defined in, but likewise the config package would depend on the plugin packages. Instead of doing this, we will use CompositeSchema (https://godoc.org/github.com/taskcluster/taskcluster-worker/runtime#CompositeSchema) to take care of Parsing json config, and provide this config back to the plugins/engines to manage themselves (e.g. the Payload member of https://godoc.org/github.com/taskcluster/taskcluster-worker/plugins#TaskPluginOptions).

Comment 2

[Pete Moore [:pmoore][:pete]]

Attachment: Github Pull Request for taskcluster-worker (#28)

There seem to be some race conditions showing up in the tests - let me take a look at this before final review.

Thanks,
Pete

Comment 3

[[github robot]]

Commits pushed to master at https://github.com/taskcluster/taskcluster-worker

https://github.com/taskcluster/taskcluster-worker/commit/e5eb4ac200bf3f7f2e36e05b464e7530280c4e8b
Bug 1246643 - auto generates global config struct from json schema yaml file

https://github.com/taskcluster/taskcluster-worker/commit/4ff4ee7a8950e7d3fb694825db8ed7a1a25ff2f3
Merge pull request #28 from taskcluster/bug1246643

Bug 1246643 - auto generate global config struct from json schema yaml file