1342909 - Using the '--try-test-paths' syntax (paths with |mach try|) doesn't seem to run e10s tasks

Status: NEW

(Developer Infrastructure :: Try, defect)

Description

[Andrew Halberstadt [:ahal]]

Blake Kaplan ran into this. I haven't done a ton of testing so not sure if it's browser-chrome specific or not, but it's easy to reproduce:

$ ./mach try -b o -p linux64 -u mochitest-browser-chrome-e10s
# successfully runs tests

$ ./mach try -b o -p linux64 -u mochitest-browser-chrome-e10s --and browser/
# no tests found matching filters

There are obviously e10s browser-chrome tests that live under the browser/ directory, but mach try errors out without finding any. I'll do a bit more digging.

Comment 1

[Andrew Halberstadt [:ahal]]

So the autotry code is a bit of a mess because when using the --try-test-paths feature, it redefines what's allowed as try syntax again [1] instead of using what the taskcluster (or buildbot) try parsers normally allow. For example, although "mochitest-browser-chrome-e10s" is a valid string in the taskcluster try parser, it is not valid with mach try + test paths.

In fact the autotry module completely disregards e10s as a syntax, but *does* schedule e10s tasks. So this would normally get the browser chrome e10s tasks scheduled:

$ ./mach try -b o -p linux64 -u mochitest-bc --and browser/

However, due to bug 1342510, that wasn't happening. Once that bug lands (should be soon), I'd expect the above syntax to schedule the e10s tasks.

There is still a big UX problem here though, because now developers need to not only know how the taskcluster/buildbot try syntax works, but also the |mach try| idiosyncrasies. I think we need to get |mach try| to import taskcluster's try_option_syntax.py and get a list of allowed aliases out of there or something like that. This is a fairly big project and likely means a complete re-write of autotry. So keeping this bug on file, though not sure when I'll have a chance to improve this.

[1] https://dxr.mozilla.org/mozilla-central/rev/7ef1e9abd296a8edc39b7efc8d637767ba2f77ed/testing/tools/autotry/autotry.py#393

Comment 2

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1342909 - Error out of |mach try| when using test paths and no build config is detected,

Review commit: https://reviewboard.mozilla.org/r/115730/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/115730/

Comment 3

[Andrew Halberstadt [:ahal]]

Comment on attachment 8841565 [details]
Bug 1342909 - Error out of |mach try| when using test paths and no build config is detected,

Sorry, accidentally attached this to the wrong bug.

Comment 4

[Chris Manchester (limited bugmail, email directly)]

The autotry code isn't a mess, it has the messy job of providing a bridge between the build system's "flavor" naming convention that comes out of the test resolver and what's accepted by try syntax. As far as I can tell it's still the only thing that attempts to do that.

Comment 5

[Andrew Halberstadt [:ahal]]

True, though I think that just means all three things are a mess ;).

Sorry, wasn't implying autotry's code organization is a mess or anything. Building tools with try syntax is fundamentally messy (see also my recent blog post). The point I was trying to make was that all three things are defined in-tree, so we could conceivably get them to share definitions of things like that.