Closed Bug 1640046 Opened 4 years ago Closed 8 months ago

Experiment with larger instances running more tasks for builds

Categories

(Release Engineering :: Firefox-CI Administration, task)

task

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: glandium, Assigned: glandium)

References

Details

Attachments

(2 files)

We're currently building on 4xlarge instances, which have 16 cores. A large part of the build time is actually spent not using those 16 cores at all. It is very likely that getting the next level of instance size (9xlarge), but running more tasks in parallel, would yield better build times, or worst case, similar build times, but with less money spending, since we'd be doing more tasks with less (but larger) instances.

Start with 9xlarge instead of 4xlarge (36 vs 16 cores), with 3 tasks per
worker. I suspect we could go with more, but let's start small.

The change landed, and a push to try using the new pool led to only one worker (despite maxCapacity: 2), and no more than one task running at the same time (despite capacityPerInstance: 3)? Any idea what's up?

Looking around, though, it seems weird that gecko-1/b-linux-aws only has maxCapacity: 50, that seems ridiculously low... and impossible to be true.

Also, looking at a bunch of recent decision workers, which are supposed to be running multiple tasks (capacityPerInstance: 2), I actually can't find a worker that has run two tasks at the same time...

Flags: needinfo?(mozilla)
Flags: needinfo?(dustin)
Flags: needinfo?(aki)
Depends on: 1640088

See Bug 1640088 for why the capacity is not effective.

Flags: needinfo?(mozilla)
Flags: needinfo?(dustin)
Flags: needinfo?(aki)

This should run 2 workers as initially intended.

Pushed by mozilla@hocat.ca:
https://hg.mozilla.org/ci/ci-configuration/rev/d0162ed8de64
Bump maxCapacity to 6 for the b-linux-experiment pool. r=tomprince
QA Contact: mgoossens
Severity: -- → N/A

IIRC this didn't yield interesting results, and the pool was removed in bug 1818761

Status: NEW → RESOLVED
Closed: 8 months ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: