1057882 - docker-worker: Support "Services"

Status: RESOLVED WONTFIX

(Taskcluster :: Workers, defect)

Description

[James Lal [:lightsofapollo]]

As part of the refactoring for TC v1.0 the worker has a generic concept of "links" as part of making task cluster a generally useful CI arbitrary links should be available.

The logic is something like this:

 - ensure "links" are pulled (using same pull logic added for private repos)
 - allow "aliases" in the docker worker schema
 - follow same teardown logic for the "proxy" link which needs its own logic.

Example schema:

payload: {
  ...,
  services: {
    image: "user/rabbitmq:tag",
    alias: "amqp",
    command: ["/bin/bash", "-c", "startamqpthing"]
  }
}

The schema should mirror the primary payload (it could even be recursive to allow services in services though that may be meta =/)

Comment 1

[James Lal [:lightsofapollo]]

Basically this is needed to run taskcluster tests in taskcluster (or related services)... It is generically useful however for many tasks. In particular this could allow "readonly" links to private images where we link the private images to public ones but do not allow scripts to be run on the private instances (which could be very useful for build caching)

Comment 2

[Jonas Finnemann Jensen (:jonasfj)]

> Basically this is needed to run taskcluster tests in taskcluster
I've said it before, bootstrapping shouldn't necessarily be a goal, it's just a footgun.
But it might be nice to have this for a lot of other reasons :)

Comment 3

[James Lal [:lightsofapollo]]

We should up the priority of this I think we can use this to deploy sscache sanely (note that we should allow private service images to open up a number of workflows)

Comment 4

[James Lal [:lightsofapollo]]

Got blocked on other things for the moment so getting at least the basics up.

Comment 5

[James Lal [:lightsofapollo]]

Attachment: https://github.com/taskcluster/docker-worker/pull/61

Comment 6

[Greg Arndt [:garndt]]

Comment on attachment 8530658 [details] [review]
https://github.com/taskcluster/docker-worker/pull/61

Looks good.

Comment 7

[Greg Arndt [:garndt]]

One issue that occurred to me is if there is a chance to use this to do bad things on a docker-worker host.  Could multiple services be spawned that are now taking up resources that aren't accounted for in the capacity of an instance?

Comment 8

[James Lal [:lightsofapollo]]

Not critical to gecko work

Comment 9

[Brian Stack [:bstack]]

This is old enough that I'm not entirely sure what is being discussed. please reopen if this is still valid.