run custom-car toolchain tasks less often
Categories
(Firefox Build System :: Task Configuration, task)
Tracking
(Not tracked)
People
(Reporter: aryx, Unassigned)
References
Details
The custom-car toolchain tasks (1, 2) run for each push and have a duration of ~8.5h each. With 2-4 pushes to mozilla-central per working day, they can use the machines up to 4 x 2 x 8.5 = 68 hours per day.
There are only 4 machines in that releng-hardware/gecko-3-b-osx-1015 = 96 hours/day.
In bug 1872707, the task got deduplicated per push (see also bug 1867068).
Other tasks which use the pool (profiling for pgo builds and rust tests at least, if I remember correct) can be delayed while they wait for a machine, e.g. if there are also pushes to repositories with higher priority (release, beta instead of autoland, everything else instead of Try).
Mike, could the toolchain tasks run less often, e.g. as part of a cron task schedules them as part of its dependency chain for performance testing?
|
||
Comment 1•1 year ago
|
||
I think you're looking at this from the wrong angle. The problem is not that those builds happen on every push. The problem is that the tasks that use them run on every push.
|
||
Comment 2•1 year ago
|
||
Ideally we want CaR to be built only once a day, and all browsertime-CaR tasks scheduled in a 24 hr span to just use that cached CaR build. How would we go about doing this?
|
|
||
Comment 3•1 year ago
|
||
trigger the browsertime-car tasks via cron, and don't do anything with the toolchains. They will be triggered automatically by being dependencies.
|
|
||
Comment 4•1 year ago
|
||
Wouldn't I need a cron to schedule a daily build for CaR toolchain though?
If so I believe that was pretty much the setup before Bug 1868998 and Bug 1872707 landed
(I realize now with the latter bug we no longer have a daily cron that builds CaR)
actually in Bug 1868826 I added the run on projects here https://searchfox.org/mozilla-central/rev/6f90f50b7a32cc062ab755e0653b3d3f512fe3bd/taskcluster/ci/test/browsertime-desktop.yml#426 to have it run every push...
Should I instead do that via cron? e.g. extend this list with more times https://searchfox.org/mozilla-central/rev/6f90f50b7a32cc062ab755e0653b3d3f512fe3bd/.cron.yml#120
Description
•