971303 - Use lower-spec builders(m3.medium?) when possible(jsshell jobs like warnaserr, warnaserr-debug, maybe others?)

Status: RESOLVED INCOMPLETE

(Release Engineering :: General, defect)

Description

[(dormant account)]

We did m3.xlarge -> c3.xlarge because unified mc builds allowed a smaller linker footprint. We should continue applying this approach to our build jobs.

At the moment our c3.xlarge builder efficiency(in terms of time cpu is not idle or waiting on IO) is < 50%. We should run low-mem-friendly workloads like bug 965447 on m3.medium or even m1.small since our inability to keep the cpu busy on higher-spec machines means most of our cost is spent paying amazon for idle cycles.

In general node performance does scale linearly as specs/costs would have one believe. It's tends to be better to run things on more cheaper nodes than a bigger node for an equivalent price.

Comment 1

[Mike Hommey [:glandium]]

> since our inability to keep the cpu busy on higher-spec machines means most of our
> cost is spent paying amazon for idle cycles.

It's not that simple though. We're paying a tradeoff between CPU, RAM, and SSD sizes. You can't choose arbitrary combinations of those.

Comment 2

[(dormant account)]

(In reply to Mike Hommey [:glandium] from comment #1)
> > since our inability to keep the cpu busy on higher-spec machines means most of our
> > cost is spent paying amazon for idle cycles.
> 
> It's not that simple though. We're paying a tradeoff between CPU, RAM, and
> SSD sizes. You can't choose arbitrary combinations of those.

until we spend less time idling it is that simple.

Comment 3

[Chris Cooper [:coop] (he/him)]

Has anything changed here in the intervening 4 months? Do we need a mapping of test type to allowable builder spec? It sounds like we might need this anyway for some of the emulator jobs.

Comment 4

[Rail Aliiev [:rail]]

This will be much easier to do in Taskcluster.