1580434 - support zone & region token/placeholders in worker pool definition

Status: RESOLVED FIXED

(Taskcluster :: Services, enhancement)

Description

[Rob Thijssen [:grenade (EET/UTC+0300)]]

at: https://cloud.google.com/compute/docs/reference/rest/v1/instanceTemplates, the disk definition parameters are described and include: properties.disks[].initializeParams.diskType which is used to define the disk type. if this property is omitted, google defaults to a disk type of pd-standard.

since we prefer to use a disk type of pd-ssd to keep build times down, it is necessary for us to define this property.

unfortunately all of the syntax formats accepted by gcloud for defining this property expect to find a zone identifier preceding pd-ssd.

when we are defining worker pools, the definition may span multiple zones and regions, so we don't have a way to specify that we want to use ssd's while at the same time specifying a worker pool that can take advantage of multiple zones.

to work around this, it would be useful if the worker pool definition could include a zone placeholder. something like what is conveyed below:

  "disks": [
    {
      "type": "PERSISTENT",
      "boot": true,
      "autoDelete": true,
      "initializeParams": {
        "sourceImage": "projects/windows-cloud/global/images/family/windows-2012-r2",
        "diskSizeGb": 50,
        "diskType": "zones/{{zone-placeholder}}/diskTypes/diskType/pd-ssd"
      }
    },
    {
      "type": "PERSISTENT",
      "autoDelete": true,
      "initializeParams": {
        "diskSizeGb": 120,
        "diskType": "zones/{{zone-placeholder}}/diskTypes/diskType/pd-ssd"
      }
    }
  ],

of course, the worker manager would then have to fill in the zone information when spawning the instance later.

Comment 1

[Rob Thijssen [:grenade (EET/UTC+0300)]]

windows workers now also need to run on a dedicated vpc (bug 1531371) which has region specific identifiers in the network config. eg:

  "networkInterfaces": [
    {
      "kind": "compute#networkInterface",
      "subnetwork": "projects/moz-fx-taskcl-winvpchost-3f31/regions/us-west1/subnetworks/kms-us-west1",
      "accessConfigs": [
        {
          "type": "ONE_TO_ONE_NAT"
        }
      ]
    }
  ],

we'll need a way to define these in the worker manager without specifying the cloud region. eg:

  "networkInterfaces": [
    {
      "kind": "compute#networkInterface",
      "subnetwork": "projects/moz-fx-taskcl-winvpchost-3f31/regions/{{region-placeholder}}/subnetworks/kms-{{region-placeholder}}",
      "accessConfigs": [
        {
          "type": "ONE_TO_ONE_NAT"
        }
      ]
    }
  ],

Comment 2

[Brian Stack [:bstack]]

We're going to support this by allowing an array of valid definitions basically. I'm working on this today since we need it for disks as well.

Comment 3

[Brian Stack [:bstack]]

https://github.com/taskcluster/taskcluster/pull/1540

Will start on a ci-config patch that updates everything to this format with the hope that this patch lands. If all goes smoothly, we could be ready to support zone-specific things by tomorrow evening.

Comment 4

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Attachment: gcp win: enable ssd, explicit subnet, custom images, startup script

• https://bugzilla.mozilla.org/show_bug.cgi?id=1531371
• https://bugzilla.mozilla.org/show_bug.cgi?id=1580434
• https://bugzilla.mozilla.org/show_bug.cgi?id=1580430
• https://bugzilla.mozilla.org/show_bug.cgi?id=1585613

Comment 5

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Attachment: gcp win: enable ssd, explicit subnet, custom images, startup script

correction to disk type uri

Comment 6

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Attachment: correction to disktype uris