1626376 - Stop running spidermonkey-sm-arm-sim on linux32

Status: NEW

(Testing :: General, enhancement)

Description

[Marco Castelluccio [:marco]]

Similarly to bug 1608390, could we just keep spidermonkey-sm-arm64-sim-linux64 on autoland and run spidermonkey-sm-arm-sim-linux32 on autoland only?

Comment 1

[Marco Castelluccio [:marco]]

Attachment: Bug 1626376 - Don't run Spidermonkey ARM sim on autoland. r=jmaher,sfink

Comment 2

[Steve Fink [:sfink] [:s:]]

(In reply to Marco Castelluccio [:marco] from comment #0)

Similarly to bug 1608390, could we just keep spidermonkey-sm-arm64-sim-linux64 on autoland and run spidermonkey-sm-arm-sim-linux32 on autoland only?

This one feels much different to me. arm64-sim does not stand in for arm-sim at all, despite the similarity in names: arm64 is an entirely different architecture from arm32, and uses an entirely different simulator. So the question of whether to reduce the frequency of running arm32 should not be related to arm64.

Second, the arm-sim run is a proxy for very expensive tests on arm32 hardware. I would much prefer to run more arm-sim jobs if it lessens the need to run arm32 hardware jobs.

I haven't looked at the spreadsheet to look at the cost of these, but before I would reduce the frequency of these jobs, I would probably look to whittling down the duration by reducing the number of configurations tested. But only if this is expensive in the first place -- this is a different configuration from everything else.

If the cost warrants further investigation, please needinfo :jandem to decide what is acceptable here.

Comment 3

[Marco Castelluccio [:marco]]

I've analyzed 24592 pushes (9 months) and out of 4116 pushes where spidermonkey jobs run, spidermonkey-sm-arm-sim-linux32 failed by itself only four times.
Let's check what the cost of this job is and then we can make a decision. :bc, could you run your script to check the cost of spidermonkey-sm-arm-sim-linux32 on autoland?

Second, the arm-sim run is a proxy for very expensive tests on arm32 hardware. I would much prefer to run more arm-sim jobs if it lessens the need to run arm32 hardware jobs.

What are the jobs that run on hardware?

Comment 4

[Bob Clary [:bc] (inactive)]

It doesn't seem that I need the full tasks_costs.py with just the one task. For the Firefox 74 release

select w.provisionerId, w.workerType, s.project, s.tier,
       s.suite, s.groupSymbol, s.symbol, s.collection,
       s.os, s.platform,
       sum(s.execution*1000.0*w.cost_per_ms) as cost
from taskclusteretl.derived_task_summary as s,
     taskclusteretl.derived_daily_cost_per_workertype as w
where s.workerType = w.workerType
and s.provisionerid = w.provisionerid
and s.date = w.date
and s.date between '2020-03-10' and '2020-04-07'
and s.project = 'autoland'
and s.platform = 'linux32'
and s.groupSymbol = 'SM'
group by provisionerId, workerType, project, tier, suite, groupSymbol, symbol, collection, os, platform
order by provisionerId, workerType, project, tier, suite, groupSymbol, symbol, collection, os, platform;

Row	provisionerId	workerType	project	tier	suite	groupSymbol	symbol	collection	os	platform	cost
1	gecko-3	b-linux	autoland	1	null	SM	arm	debug	linux	linux32	154.89455576828937

13 releases/year gives $2013/year. Does that seem right?

We run arm32 jobs on actual phones at bitbar. SM does not run there but jit-test and jsreftest run though not on autoland. Let me know if this isn't what you need.

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

I had calculated the sm(arm) jobs were ~$800/year