Open Bug 1434914 Opened 2 years ago Updated 3 months ago

Figure out how to chunk "mach try coverage"

Categories

(Testing :: Code Coverage, enhancement)

enhancement
Not set

Tracking

(Not tracked)

People

(Reporter: marco, Unassigned)

References

(Blocks 1 open bug)

Details

If the "mach try coverage" spawns a single chunk and we try to run too many directories containing tests, the chunk will likely time out.
We need to figure out how to split the tests in chunks.
this will be different per platform and per config (debug vs opt).  Possibly some heuristics based on total number of tests in the manifests/directories we are testing and assuming a fixed runtime per test + overhead, then extrapolate out.  Assume double runtime for debug/asan cases.
Joel, do you think the work you're doing in bug 1453056 can be useful here too?
yes, I am intending that it will solve both test-verify and test-coverage.  We might need some adjustments for test-coverage, but small tweaks.
(In reply to Joel Maher ( :jmaher) (UTC-5) from comment #3)
> yes, I am intending that it will solve both test-verify and test-coverage. 
> We might need some adjustments for test-coverage, but small tweaks.

Both for the tasks that collect per-test coverage (the test-coverage job) and the "mach try coverage" code that figures out the number of chunks to use given a set of tests to run? (this bug is about the latter)
my work for test-verify is to determine the number of chunks when running |mach try fuzzy| so we can select the right number of chunks, then in the decision task it uses the same code; lastly the chunking logic in verify_tools.py is used for selecting which tests are run in each chunk.
(In reply to Joel Maher ( :jmaher) (UTC-5) from comment #5)
> my work for test-verify is to determine the number of chunks when running
> |mach try fuzzy| so we can select the right number of chunks, then in the
> decision task it uses the same code; lastly the chunking logic in
> verify_tools.py is used for selecting which tests are run in each chunk.

Awesome! So your work will basically fix this bug too.
Depends on: 1453056
Blocks: 1507108
No longer blocks: 1429463
See Also: → 1583353
You need to log in before you can comment on or make changes to this bug.