1088243 - Provisioner: Port aws-provisioner to taskcluster-base and refactor...

Status: RESOLVED FIXED

(Taskcluster :: Services, defect)

Description

[Jonas Finnemann Jensen (:jonasfj)]

We should refactor the aws-provisioner, it's really old...

  * Remove UI
  * Add API
  * Add exchanges for all exchanges to workerTypes
  * Setup queues and publish messages to IronMQ for each workerType
  * Re-architect into web-frontend and worker backend
  * Support min/max spot nodes
  * Ability to fallback to on-demand nodes when spot proves unavailable
  * Support min/max on-demand nodes
  * Consider (or at least prepare for) multi-region support
    (So we can get the cheapest spot nodes)
  * Track state:
      - Number of instances
      - Amount of capacity
      - Number of pending tasks
      - Number of running tasks
      - Number of pending spot requests
      - Number of ordered on-demand nodes
    Without using anything but the exhanges offered by the queue, and state that we can
    get from IronMQ.
  * Consider (or at least prepare for) health check support.

Note, publishing messages to IronMQ could be done from the queue. But might in fact be
better to do from the provisioner. We'll see what makes sense.

Comment 1

[Jonas Finnemann Jensen (:jonasfj)]

I think we can declare this as fixed... we've rolled out new provisioner that is much better.