1653453 - Reduce mozharness fetch costs

Status: RESOLVED WONTFIX

(Release Engineering :: Applications: MozharnessCore, enhancement)

Description

[Edwin Takahashi (:egao | infrequent contributor)]

Synopsis

Each time a task that uses mozharness is started, the test environment goes through the process of fetching mozharness, unzip, etc.

However, how often does mozharness code even change?

Proposal

Is it feasible to bake in mozharness to the machine?
If not, is there something else that can be done to reduce the number of times mozharness is fetched?

Justification

If mozharness code remains largely unchanged week-over-week, it may make sense from a bandwidth cost + machine time cost perspective to bake mozharness into the machine image.

This should be fairly easy to do in docker-based systems, but I have no insight into how feasible this may be for Windows and macOS.

Comment 1

[Mike Hommey [:glandium]]

The fetching of mozharness + unzip is a tiny part of the initialization cost on tests. The major part is the initialization of the virtualenv.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Comment 3

[Aki Sasaki (not active)]

If we end up deciding we want to run tests from the tree rather than test zips, we can dup this bug to that one.