1580572 - [image-generation] support feeding monopacker output into ci-admin

Status: NEW

(Taskcluster :: Workers, enhancement)

Description

[Miles Crabill [:miles]]

When we generate new images we will need to update ci-admin with their AMI and compute engine image IDs.

This bug covers creating a script that takes output from monopacker (JSON format, specified here: https://www.packer.io/docs/post-processors/manifest.html), aggregating it if necessary (from multiple tasks), and feeding that into ci-admin.

For end-to-end automation, this script should push a change that can be by releng reviewed in Phabricator (?)

Comment 1

[Miles Crabill [:miles]]

Multiple teams at Mozilla run taskcluster workers. We should make it easy for these teams to will automate their worker image generation processes, though running monopacker locally will still be easiest.

Already, these teams include:

• taskcluster
• relops
• releng
• deepspeech

End-to-end, my rough vision of this process:

• someone makes a change to a teams' worker infra repo, which consists of:
  • builders / scripts / templates / vars for monopacker
  • taskcluster.yml
• this triggers a meta task that runs subtasks for all changed builders
  • subtasks runs on workers specifically used for image generation
  • these workers have permissions to start instances, snapshot, etc.
  • these workers also have access to staging and production secrets
• the meta task aggregates output image IDs, AMI IDs, etc. and creates push with updated worker-types
• the meta task flags relevant parties for review in phabricator

This fully automates the process of updating worker-types. The whole process should also be able to be run locally.

Comment 2

[Miles Crabill [:miles]]

The output format of packer builders varies in cases where the builder produces multiple artifacts, so we would need to consume output differently per builder.

We're currently stalled on productionizing end-to-end image baking automation because of chain of trust, so this bug is stalled as well.

Example multi-region AMI bake using the amazon-ebs builder:

{
  "builds": [
    {
      "name": "docker_worker_aws",
      "builder_type": "amazon-ebs",
      "build_time": 1570817165,
      "files": null,
      "artifact_id": "us-east-1:ami-06788257191c11c0e,us-west-1:ami-097ac09313fabbab4,us-west-2:ami-08b57489faf2ad880",
      "packer_run_uuid": "34fe3119-c1e1-b75f-8248-6977d4beb454",
      "custom_data": null
    }
  ],
  "last_run_uuid": "34fe3119-c1e1-b75f-8248-6977d4beb454"
}

Example of two google compute images baked using the googlecompute builder:

{
  "builds": [
    {
      "name": "docker_worker_gcp",
      "builder_type": "googlecompute",
      "build_time": 1570557864,
      "files": null,
      "artifact_id": "docker-worker-gcp-googlecompute-2019-10-08t17-57-05z",
      "packer_run_uuid": "2d1ac54c-8aa2-ac99-5ec3-a861b929d8c7",
      "custom_data": null
    },
    {
      "name": "docker_worker_gcp_l3",
      "builder_type": "googlecompute",
      "build_time": 1570557879,
      "files": null,
      "artifact_id": "docker-worker-gcp-l3-googlecompute-2019-10-08t17-57-06z",
      "packer_run_uuid": "2d1ac54c-8aa2-ac99-5ec3-a861b929d8c7",
      "custom_data": null
    }
  ],
  "last_run_uuid": "2d1ac54c-8aa2-ac99-5ec3-a861b929d8c7"
}

The format here is pretty clear, we can map builders by name in the builds array, get their artifact_ids, splitting if necessary, and we have the image to substitute into tc-admin.

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

If monopacker can convert those into something like

- type: aws
  name: docker_worker_aws
  version: 12345099658  # or whatever makes sense to distinguish different runs of monpacker
  images:
    - {region: "us-east-1", image: "ami-06788257191c11c0e"}
    - {region: "us-west-1", image: "ami-097ac09313fabbab4"}
    - {region: "us-west-2", image: "ami-08b57489faf2ad880"}

and

- type: google
  name: docker_worker_gcp
  version: 320987943875
  images:
    - {project_id: "taskcluster-imaging", image: "docker-worker-gcp-googlecompute-2019-10-08t17-57-05z"}
- type: google
  name: docker_worker_gcp_l3
  version: 320987943875
  images:
    - {project_id: "taskcluster-imaging", image: "docker-worker-gcp-l3-googlecompute-2019-10-08t17-57-06z"}

sort of spitballing here, but the idea is a pretty easily parsable format that includes both the metadata by which one might want to search for the image, and the details on how to configure a worker to use the image.