1342510 - Running browser-chrome tests for a directory doesn't run e10s-browser-chrome

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Blake Kaplan (:mrbkap) (inactive)]

Over in bug 1335801, I was trying to do some try server debugging on a specific test directory. The failure was in e10s browser-chrome mochitests. I would make a change to the given mochitests and use `mach try` to start my try run. As an example, see [1]. Please note that I went in after the fact to trigger my e10s test runs. They weren't run automatically.

The trychooser syntax generated there was: try: -b o -p linux64 -u mochitest-browser-chrome-1,mochitest-e10s-browser-chrome-1 -t none --try-test-paths browser-chrome:browser/base/content/test/referrer 

The mach try command line was: mach try -b o -p linux64 -u mochitest-e10s-bc browser/base/content/test/referrer/

Looking at this now, I wonder if we need another --try-test-paths argument with e10s in it?

[1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=851ed9b7ca67e3bd78aa5b5c510e6e2f2428b820

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

The parameters to -u are matched against the `unittest_try_name` attribute of test tasks [1].  You can see this attribute in the full task graph JSON [2] (warning: don't load in a browser!).  There are a bunch of shorthands defined [3] because, among other reasons, people can't seem to remember what order the words come in some identifiers :)

So, from your [1] I can see that the syntax was

  try: -b o -p linux64 -u mochitest-browser-chrome-1,mochitest-e10s-bc,mochitest-e10s-browser-chrome-1 -t none --try-test-paths browser-chrome:browser/base/content/test/referrer

** mochitest-browser-chrome-1 looks for chunk 1 of mochitest-browser-chrome, aliased to alias_prefix('mochitest-browser-chrome'), which matches tasks with that prefix in their unittest_try_name.  That matches

test-linux64-pgo/opt-mochitest-browser-chrome-1
test-linux64/opt-mochitest-browser-chrome-1
test-linux64-ccov/opt-mochitest-browser-chrome-1
test-linux32/debug-mochitest-browser-chrome-e10s-1
test-linux64/debug-mochitest-browser-chrome-e10s-1
test-linux32/opt-mochitest-browser-chrome-1
test-linux64-asan/opt-mochitest-browser-chrome-e10s-1
test-linux64-asan/opt-mochitest-browser-chrome-1
test-linux64/debug-mochitest-browser-chrome-1
test-linux64-jsdcov/opt-mochitest-browser-chrome-1
test-linux32/debug-mochitest-browser-chrome-1
test-linux64/opt-mochitest-browser-chrome-e10s-1
test-linux32/opt-mochitest-browser-chrome-e10s-1
test-linux64-pgo/opt-mochitest-browser-chrome-e10s-1

** mochitest-e10s-bc is aliased to alias_prefix('mochitest-e10s-browser-chrome'), which matches nothing, which makes sense -- the unittest_try_name for e10s tests has an `-e10s` suffix (it's automatically generated that way).

** mochitest-e10s-browser-chrome-1 is aliased to alias_prefix('mochitest-e10s-browser-chrome'), the same as mochitest-e10s-bc.  So it, too, matches nothing.

`hg praise` on that alias line points to [4], so let's see what Joel says!  I suspect changing it to `alias_prefix('mochitest-browser-chrome-e10s')` would fix this.

[1] http://gecko.readthedocs.io/en/latest/taskcluster/taskcluster/attributes.html#unittest-try-name
[2] https://queue.taskcluster.net/v1/task/DeVpSCI1Sp-H2SVz7PGL_w/runs/0/artifacts/public/full-task-graph.json
[3] https://dxr.mozilla.org/mozilla-central/source/taskcluster/taskgraph/try_option_syntax.py#59
[4] https://hg.mozilla.org/mozilla-central/rev/71346fd5e1ec

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

:dustin, this has been my complaint for the last 6 months about the differences between buildbot and taskcluster- if we change this then windows/osx will not run tests, only linux will.  I assume we can support both: mochitest-e10s-browser-chrome-X and mochitest-browser-chrome-e10s-X

Comment 3

[Andrew Halberstadt [:ahal]]

So I told Blake to file here but I think I misunderstood his problem. This might actually be a bug in ./mach try. For example:

$ ./mach try -b o -p linux64 -u mochitest-browser-chrome-e10s
# successfully runs tests

$ ./mach try -b o -p linux64 -u mochitest-browser-chrome-e10s --and browser/
# no tests found matching filters

There are obviously e10s enabled browser-chrome tests under the /browser directory, but mach try errors out. So while I agree that the difference between taskcluster and buildbot is confusing, I believe that is not the issue Blake has (though please correct me if I'm wrong).

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

you should be able to support both with aliases, yes -- but not by making an alias that expands to nothing :)

Comment 5

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: Bug 1342510: fix -u mochitest-e10s-foo aliases to point to nonempty sets of tests;

Review commit: https://reviewboard.mozilla.org/r/115428/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/115428/

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8841095 [details]
Bug 1342510: fix -u mochitest-e10s-foo aliases to point to nonempty sets of tests;

https://reviewboard.mozilla.org/r/115428/#review116950

nice and simple!

Comment 7

[Dustin J. Mitchell [:dustin] (he/him)]

Andrew, if you would like to fix the mach `try issue`, feel free to take the bug and mark leave-open.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/46e57ca6f77e
fix -u mochitest-e10s-foo aliases to point to nonempty sets of tests; r=jmaher

Comment 9

[Andrew Halberstadt [:ahal]]

Probably cleaner to just move this back to Taskcluster::Task Configuration and file another bug for the `mach try` issue.

Comment 10

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/46e57ca6f77e