1265390 - Introduce AMI Sets

Status: RESOLVED WONTFIX

(Taskcluster :: Services, defect)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

Background:
  bug 1249933
  https://public.etherpad-mozilla.org/p/tc-aws-api-stuff

An AMI Set is a collection of AMIs with a single name (its AMI Set ID).  Each AMI in the set is keyed by its virtualization type (PV or HVM) and by its AWS region.

A worker type definition refers to an AMI set by name, in the amiSetId field.  When the provisioner goes to launch an instance of that worker, it looks up the appropiate AMI ID based on the AMI Set ID and the region and virtualization type.

The ability to create AMI Sets is limited by a scope, say `aws-provisioner:manage-ami-set:<amiSetId>`.

The ability to assign an AMI Set ID to a workerType is also limited by a scope, say `aws-provisioner:deploy-ami-set:<amiSetId>`, and this scope is checked when creating a workerType and when updating it to change its amiSetId.

We need to validate AMI sets against EC2 frequently: when creating or updating the AMI set, when deploying to a workerType, and when launching a workerType.  See bug 1264956.

A rough TO-DO list for the project (subject to revision!):

    create AmiSet Entity

    add create, remove, update, delete endpoints for Ami Sets

    tools UI for AMI sets

    write schema for Ami sets

    add function to check that an AMI set points to all valid and existing AMIs

    translate ami set to AMI ID when generating launch specification (must validate ami set as stated above)

    alter workerType schema to allow ami set IDs *or* imageIds, so we can migrate.  Once everything is migrated, disallow imageIds.

    verify scope for deploying an ami set is satisfied when creating/updating a workerType