1466186 - Ability to declare EBS volumes conditional on instance type

Status: RESOLVED WORKSFORME

(Taskcluster :: Services, enhancement)

Description

[Gregory Szorc [:gps]]

Amazon recently announced the c5d instance types. These have NVMe instance storage and don't require EBS volumes. https://aws.amazon.com/blogs/aws/ec2-instance-update-c5-instances-with-local-nvme-storage-c5d/

My testing in bug 1462528 shows that I/O on c5d's is *very* good. I dare say we should use c5d's in as many places as possible (within reason) because of it.

Unfortunately, the way we currently define worker types (at least for docker-worker in AWS provisioner) doesn't give us enough flexibility to mix the c5d instances with non-c5d instances.

We like to define worker types so they are composed of multiple EC2 instance types. e.g. c5.4xlarge and m5.4xlarge. We provision a mix of instance types, often getting types that are available / cheaper at any given time.

The way things currently work, we define a "BlockDeviceMappings" key that enables EBS volumes on a per-region basis. Unfortunately, this doesn't play well with mixing c5d instances and non-c5d instances. We don't need EBS volumes for c5d instances. So we have to choose between:

a) mixing multiple EC2 instance types in the worker type and using an extraneous EBS volume on c5d instances
b) only using c5d instances

"a" is actually a bit worse, since I believe docker-worker will bind the instance and EBS storage to the same volume and I/O will span both devices, not yielding the benefits of instance NVMe storage.

As Amazon said in the blog post, they will be releasing more instance types with instance storage in the future. So this problem will only get worse for us over time.

I believe we want some kind of mechanism in the worker type definition to declare that an EBS volume should be conditional based on e.g. the instance type. e.g. if we're using a c5 instance, we should request an EBS volume but if we're using a c5d instance, we should not. I /think/ this is an AWS Provisioner feature request since it seems like a limitation in the worker type definition.

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

The BlockDeviceMappings field can go in the launchSpec property of the items under instanceTypes.

The whole thing gets somewhat repetitive, but with tools we can automatically generate that.

Comment 2

[Gregory Szorc [:gps]]

!!!

So perhaps this should be WORKSFORME?

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

If, indeed, it WORKS FOR (you) :)