Decouple test packaging from build tasks
Categories
(Firefox Build System :: Task Configuration, enhancement)
Tracking
(Not tracked)
People
(Reporter: ahal, Unassigned)
References
(Blocks 1 open bug)
Details
As of bug 1752677 we'll have the ability to re-use tasks from a previous try push. One of the more obvious use cases of this would be to run a single build, and then iterate on a test that depends on that build quickly.
The problem is that the test tasks also depend on the builds for the tests.zip artifacts, which contain the test in question. This means if you were iterating on a test, your test tasks would still use the old version of the test that comes from the build.
Aki had the idea to split the test packaging step out of the build and into a standalone task. So we'd have a graph like:
push ------> build ------------> test
\-----> test_zip_task ---/
This way we could replace the builds, but not replace the tasks that package up tests. Imo having tasks that adhere to the single responsibility principle is a good thing in general and has further potential benefits. For example, it'll make builds faster for cases where the test package isn't needed (e.g a release pipeline).
|
||
Comment 1•2 years ago
|
||
What if we stop packaging tests and have test tasks clone the repo? I guess that's a more invasive change and might affect runtimes?
|
||
Comment 2•2 years ago
|
||
It would be a killer on Windows, where very often a clone takes > 30 minutes (especially true on try)
One aspect of this is that the build configuration has an effect of what tests are packaged. Also, some of the test files are compiled. So you'd need as many of those tasks as there are build types.
How about allowing mach try --artifact to do artifact builds using artifacts from another push?
|
||
Comment 3•1 month ago
|
||
Andrew, can you weight in about Mike's proposal in comment 2? I'm willing to help some, I'm currently debugging performance tests, and the debugging cycle is extremely long because of PGO. Or any other idea you'd have to speed my case up, or anything.
|
Reporter | |
Comment 4•1 month ago
|
||
I like that idea too, it's probably a "yes and" type of thing with this bug rather than either or (I don't see any reason not to split test packaging out of the build).
My only problem with Mike's idea is that I don't know how to implement it :)
|
||
Comment 5•1 month ago
|
||
(In reply to Andrew Halberstadt [:ahal] from comment #4)
I like that idea too, it's probably a "yes and" type of thing with this bug rather than either or (I don't see any reason not to split test packaging out of the build).
My only problem with Mike's idea is that I don't know how to implement it :)
If you set MOZ_ARTIFACT_TASK* appropriately, you'd end up with binaries from a pre-existing task and test (packages) from the current push. It's not clear to me that's better than clever use of --use-existing-tasks, which kinda-sorta should achieve the same thing.
Description
•