1183232 - Add a switch to enable/disable override of chunking behavior associated with mach try

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[Chris Manchester (limited bugmail, email directly)]

When selecting tests by directory with mach try we force all tests into a single chunk to prevent weird new chunks that might fail due to exposing new test-interdependence. Doing the same would make sense under some circumstance when selecting by tag via mach try.

Comment 1

[Chris Manchester (limited bugmail, email directly)]

Looking at this now it needs to be a bit different than I mentioned in comment 0. If tags are supplied to the command but paths and -u are not, we can just look at all-tests.json to get the required suites and run those in a single chunk. Otherwise --tag should filter tests selected by another means, as it does today.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=7d300fa53352
https://treeherder.mozilla.org/#/jobs?repo=try&revision=0fa07a73a223

Comment 2

[Chris Manchester (limited bugmail, email directly)]

Attachment: MozReview Request: Bug 1183232 - Allow selecting tests and suites with just a tag from mach try. r=ahal

Bug 1183232 - Allow selecting tests and suites with just a tag from mach try. r=ahal

Comment 3

[Andrew Halberstadt [:ahal]]

Comment on attachment 8643434 [details]
MozReview Request: Bug 1183232 - Allow selecting tests and suites with just a tag from mach try. r=ahal

https://reviewboard.mozilla.org/r/15099/#review13587

The mozbuild changes should get a build peer review, from gps probably.

::: testing/tools/autotry/autotry.py:53
(Diff revision 1)
> +    def resolve_tags(self, tags):
> +        tests = list(self.resolver.resolve_tests(tags=tags,
> +                                                 cwd=self.mach_context.cwd))
> +        return self._manifests_by_flavor(tests)
> +
> +    def resolve_paths(self, paths):
> +        if not paths:
> +            return {}
> +        tests = list(self.resolver.resolve_tests(paths=paths,
> +                                                 cwd=self.mach_context.cwd))
> +        return self._manifests_by_flavor(tests)

I might be missing something, but why not just forward both 'tags' and 'paths' onto mozbuild.resolve_tests() at the same time?

Comment 4

[Gregory Szorc [:gps]]

Comment on attachment 8643434 [details]
MozReview Request: Bug 1183232 - Allow selecting tests and suites with just a tag from mach try. r=ahal

https://reviewboard.mozilla.org/r/15099/#review13591

I only looked at the build bits.

::: python/mozbuild/mozbuild/testing.py:115
(Diff revision 1)
> +                if tags and not tags.intersection(set(test.get('tags', '').split())):

set.intersection(x) is almost always written as `set & x`. Please change this, using parens as necessary.

Comment 5

[Chris Manchester (limited bugmail, email directly)]

Comment on attachment 8643434 [details]
MozReview Request: Bug 1183232 - Allow selecting tests and suites with just a tag from mach try. r=ahal

Bug 1183232 - Allow selecting tests and suites with just a tag from mach try. r=ahal

Comment 6

[Andrew Halberstadt [:ahal]]

Comment on attachment 8643434 [details]
MozReview Request: Bug 1183232 - Allow selecting tests and suites with just a tag from mach try. r=ahal

https://reviewboard.mozilla.org/r/15099/#review13669

Ship It!

Comment 7

[Pulsebot]

https://hg.mozilla.org/mozilla-central/rev/cab252737a12

Comment 8

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/cab252737a12