Have `mach try auto` support specifying platforms and test directories
Categories
(Developer Infrastructure :: Try, enhancement)
Tracking
(Not tracked)
People
(Reporter: jwatt, Unassigned)
References
Details
Often I know which tests/test directories contain relevant tests for a change that I'm making, and which platforms i want to run the tests on. Having to specify test suites by names rather than specify tests/test directory paths means I end up wondering:
Is the test in 'opt-reftest-e10s-1' or in 'opt-reftest-e10s-32'? Who knows, I guess I just run the tests in all 32 of those test groups...
Worse, I sometimes don't even know which group it's in so I end up throwing multiple groups into the mix.
This has got to be costly in terms of CI expenditure, will contribute to Try backlogs that slow down other devs, and wastes my time trying to figure out what to send to Try and then sifting through a much larger set of results than I'd like.
Could we have a new mach try sub-command that accepts a list of platforms and test paths, and have it do the work to figure out which test groups to run?
|
Reporter | |
Comment 1•3 years ago
|
||
Apparently try fuzzy has been able to do this for some time and I was just unaware of that:
https://firefox-source-docs.mozilla.org/tools/try/selectors/fuzzy.html#test-paths
|
||
Comment 2•3 years ago
|
||
I was actually looking for this yesterday and didn't find it. I don't see a reason not to add support for mach try path.
|
||
Comment 3•3 years ago
|
||
Do you mean you want paths in the default mach try auto selector? If so yeah, a valid feature request. If not what do you mean? The selectors where paths make the most sense already support it.
|
|
||
Comment 4•3 years ago
|
||
Do you mean you want paths in the default mach try auto selector?
Yes
|
|
||
Updated•3 years ago
|
|
|
||
Comment 5•3 years ago
|
||
I think this use case would also be solved by bug 1635921.
|
||
Updated•2 years ago
|
|
||
Comment 6•1 year ago
|
||
To make sure I understand, you'd like to select a set of tests yourself, and then have "auto" add more on top of them?
|
|
||
Comment 7•1 year ago
|
||
No, I would like to mach try path/to/specific/test and have mach figure out what it needs to trigger. Maybe this works now (I don't know I haven't tried in a while), but back when the bug was filed, it definitely didn't. Being able to also select a subset of platforms to run the test on would be useful as well.
|
|
||
Comment 8•1 year ago
|
||
It is supported now, you can run "mach try fuzzy path/to/specific/test".
Description
•