1448407 - Move *-decision workers to [cm]5.xlarge instances

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Gregory Szorc [:gps]]

We currently run decision tasks on [cmr]3.xlarge instances. This is an older EC2 instance type. We should switch the worker type to c5.xlarge / m5.xlarge so we're executing on modern CPUs. This will hopefully make the tasks faster.

In theory this is only a change to the AWS Provisioner worker type config. But it might have implications for CoT.

Comment 1

[Aki Sasaki (not active)]

I believe the CoT parameters here are:

- Level 3 decision workers have valid docker-worker gpg keypairs. If we keep the same AMI, this should be true.
- We expect the workerType to stay the same. If we're changing the existing workerType rather than introducing a new one, this should be true.
- We expect the docker image shas to be allowlisted, until we turn off cotv1. If we're not changing the docker image, this should be true.

I think this is doable without affecting CoT.

Comment 2

[Gregory Szorc [:gps]]

Thanks, aki!

The only issues I can think of with changing the instance type is things will run from an EBS volume and we'll need AMI support for nvme devices. The AMI is the same as what we're running for the build workers and it supports nvme. So unless there's something in the Docker image (I can't imagine there is) or the task configuration that needs to react to the presence of an EBS volume, then bumping the worker type should "just work."

Comment 3

[Gregory Szorc [:gps]]

I almost filed this bug again today.

We'll want to switch to c5.xlarge and m5.xlarge. Let's make the EBS volume size 80 GB. All other parameters can likely stay the same.

wcosta: is this something you can roll out for us?

Comment 4

[Wander Lairson Costa]

(In reply to Gregory Szorc [:gps] from comment #3)
> I almost filed this bug again today.
> 
> We'll want to switch to c5.xlarge and m5.xlarge. Let's make the EBS volume
> size 80 GB. All other parameters can likely stay the same.
> 
> wcosta: is this something you can roll out for us?

Which worker-type(s) do you need I change to new instance types??

Comment 5

[Gregory Szorc [:gps]]

This would be the gecko-{1,2,3}-decision worker types.

There is a gecko-decision and gaia-decision worker type as well. But those may be both unused (and can be deleted)? I don't care about these types for purposes of this bug.

Comment 6

[Wander Lairson Costa]

(In reply to Gregory Szorc [:gps] from comment #5)
> This would be the gecko-{1,2,3}-decision worker types.
> 
> There is a gecko-decision and gaia-decision worker type as well. But those
> may be both unused (and can be deleted)? I don't care about these types for
> purposes of this bug.

What about r3, should I delete it or leave at it is?

Comment 7

[Gregory Szorc [:gps]]

The r3 should be deleted.

Comment 8

[Wander Lairson Costa]

(In reply to Gregory Szorc [:gps] from comment #7)
> The r3 should be deleted.

Ok, done!