1428068 - Move pricing for spot requests from internal aws-provisioner infrastructure over to the ec2-manager's pricing system

Status: RESOLVED WONTFIX

(Taskcluster :: Services, enhancement)

Description

[John Ford [:jhford] CET/CEST Berlin Time]

Currently, the AWS Provisioner uses internal code (aws-manager.js) to determine the prices of specified resources.  We also have this logic implemented in the ec2-manager.  Let's use the ec2-manager's pricing system.

We'll need to do the following:

1. Remove pricing updates from aws-manager.js's update() function -- the availableAZ stuff needs to stay for the time being

2. Verify that the pricing functionality in ec2-manager is still functional after the changes to how spot prices work.  This might involve supporting pagination of data in the ec2-manager as well as caching the prices in the Postgres database

3. Calling out to the ec2-manager's getPrices() api to retrieve prices for each iteration of the provisioner and passing that information into the worker type

Some useful touchpoints:

https://github.com/taskcluster/ec2-manager/blob/a116aba/lib/pricing-poller.js
https://github.com/taskcluster/ec2-manager/blob/a116aba/lib/api.js#L560
https://github.com/taskcluster/aws-provisioner/blob/1a094bd/src/aws-manager.js#L214
https://github.com/taskcluster/aws-provisioner/blob/1a094bd/src/worker-type.js#L807

Comment 1

[Wander Lairson Costa]

https://github.com/taskcluster/aws-provisioner/pull/174

Comment 2

[John Ford [:jhford] CET/CEST Berlin Time]

Given the worker manager RFC, we're probably better off using the time we need to finish this bug into implementing the worker manager.  We have something working now and we'd like to deprecate the aws-provisioner codebase in favour of worker-manager anyway.