(In reply to Geoff Brown [:gbrown] from comment #4) > If I re-enable this feature, I find the test selections interesting and somewhat relevant, but often perplexing. (There seems to be a strong inclination to run python tests. If I change a random layout file, most crashtests are run.) > > Note there are very few IMPACTED_TESTS annotations in tree. Looks like "very few" is a bit of an understatement, there is only a single annotation in the entire tree: https://searchfox.org/mozilla-central/search?q=IMPACTED_TESTS&path=moz.build (the rest are all in the mozbuild unittests) > :ahal - What do you think? Should I re-enable 'mach test' with no arguments, or leave it disabled and clean up the help and supporting code? I think we should just disable it. It requires manual maintenance and doesn't provide much value. If people want to just have tests chosen for them automatically, they can now run `mach try auto`. A more interesting idea might be to have `mach test` query the bugbug ML algorithm directly and run those tests. But this would be a separate system than `IMPACTED_TESTS`.
Bug 1516847 Comment 5 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
(In reply to Geoff Brown [:gbrown] from comment #4) > If I re-enable this feature, I find the test selections interesting and somewhat relevant, but often perplexing. (There seems to be a strong inclination to run python tests. If I change a random layout file, most crashtests are run.) > > Note there are very few IMPACTED_TESTS annotations in tree. Looks like "very few" is a bit of an understatement, there is only a single annotation in the entire tree: https://searchfox.org/mozilla-central/search?q=IMPACTED_TESTS&path=moz.build (the rest are all in the mozbuild unittests) > :ahal - What do you think? Should I re-enable 'mach test' with no arguments, or leave it disabled and clean up the help and supporting code? I think we should just purge it and supporting code. It requires manual maintenance and doesn't provide much value. If people want to just have tests chosen for them automatically, they can now run `mach try auto`. A more interesting idea might be to have `mach test` query the bugbug ML algorithm directly and run those tests. But this would be a separate system than `IMPACTED_TESTS`.
(In reply to Geoff Brown [:gbrown] from comment #4) > If I re-enable this feature, I find the test selections interesting and somewhat relevant, but often perplexing. (There seems to be a strong inclination to run python tests. If I change a random layout file, most crashtests are run.) > > Note there are very few IMPACTED_TESTS annotations in tree. Looks like "very few" is a bit of an understatement, there is only a single annotation in the entire tree: https://searchfox.org/mozilla-central/search?q=IMPACTED_TESTS&path=moz.build (the rest are all in the mozbuild unittests) > :ahal - What do you think? Should I re-enable 'mach test' with no arguments, or leave it disabled and clean up the help and supporting code? I think we should just purge it and supporting code. It requires manual maintenance and doesn't provide much value. If people want to have tests chosen for them automatically, they can now run `mach try auto`. A more interesting idea might be to have `mach test` query the bugbug ML algorithm directly and run those tests. But this would be a separate system than `IMPACTED_TESTS` (though even that is questionable.. why run stuff automatically locally when you can have CI do it for you).
(In reply to Geoff Brown [:gbrown] from comment #4) > If I re-enable this feature, I find the test selections interesting and somewhat relevant, but often perplexing. (There seems to be a strong inclination to run python tests. If I change a random layout file, most crashtests are run.) > > Note there are very few IMPACTED_TESTS annotations in tree. Looks like "very few" is a bit of an understatement, there is only a single annotation in the entire tree: https://searchfox.org/mozilla-central/search?q=IMPACTED_TESTS&path=moz.build (the rest are all in the mozbuild unittests) > :ahal - What do you think? Should I re-enable 'mach test' with no arguments, or leave it disabled and clean up the help and supporting code? I think we should just purge it and supporting code. It requires manual maintenance and doesn't provide much value. If people want to have tests chosen for them automatically, they can now run `mach try auto`. A more interesting idea might be to have `mach test` query the bugbug ML algorithm directly and run those tests (though even that is questionable.. why run stuff automatically locally when you can have CI do it for you). But this would be a separate system than `IMPACTED_TESTS`.