Running browser-chrome tests for a directory doesn't run e10s-browser-chrome

RESOLVED FIXED in mozilla54

Status

task
RESOLVED FIXED
2 years ago
Last year

People

(Reporter: mrbkap, Assigned: dustin)

Tracking

unspecified
mozilla54

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

Over in bug 1335801, I was trying to do some try server debugging on a specific test directory. The failure was in e10s browser-chrome mochitests. I would make a change to the given mochitests and use `mach try` to start my try run. As an example, see [1]. Please note that I went in after the fact to trigger my e10s test runs. They weren't run automatically.

The trychooser syntax generated there was: try: -b o -p linux64 -u mochitest-browser-chrome-1,mochitest-e10s-browser-chrome-1 -t none --try-test-paths browser-chrome:browser/base/content/test/referrer 

The mach try command line was: mach try -b o -p linux64 -u mochitest-e10s-bc browser/base/content/test/referrer/

Looking at this now, I wonder if we need another --try-test-paths argument with e10s in it?

[1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=851ed9b7ca67e3bd78aa5b5c510e6e2f2428b820
The parameters to -u are matched against the `unittest_try_name` attribute of test tasks [1].  You can see this attribute in the full task graph JSON [2] (warning: don't load in a browser!).  There are a bunch of shorthands defined [3] because, among other reasons, people can't seem to remember what order the words come in some identifiers :)

So, from your [1] I can see that the syntax was

  try: -b o -p linux64 -u mochitest-browser-chrome-1,mochitest-e10s-bc,mochitest-e10s-browser-chrome-1 -t none --try-test-paths browser-chrome:browser/base/content/test/referrer

** mochitest-browser-chrome-1 looks for chunk 1 of mochitest-browser-chrome, aliased to alias_prefix('mochitest-browser-chrome'), which matches tasks with that prefix in their unittest_try_name.  That matches

test-linux64-pgo/opt-mochitest-browser-chrome-1
test-linux64/opt-mochitest-browser-chrome-1
test-linux64-ccov/opt-mochitest-browser-chrome-1
test-linux32/debug-mochitest-browser-chrome-e10s-1
test-linux64/debug-mochitest-browser-chrome-e10s-1
test-linux32/opt-mochitest-browser-chrome-1
test-linux64-asan/opt-mochitest-browser-chrome-e10s-1
test-linux64-asan/opt-mochitest-browser-chrome-1
test-linux64/debug-mochitest-browser-chrome-1
test-linux64-jsdcov/opt-mochitest-browser-chrome-1
test-linux32/debug-mochitest-browser-chrome-1
test-linux64/opt-mochitest-browser-chrome-e10s-1
test-linux32/opt-mochitest-browser-chrome-e10s-1
test-linux64-pgo/opt-mochitest-browser-chrome-e10s-1

** mochitest-e10s-bc is aliased to alias_prefix('mochitest-e10s-browser-chrome'), which matches nothing, which makes sense -- the unittest_try_name for e10s tests has an `-e10s` suffix (it's automatically generated that way).

** mochitest-e10s-browser-chrome-1 is aliased to alias_prefix('mochitest-e10s-browser-chrome'), the same as mochitest-e10s-bc.  So it, too, matches nothing.

`hg praise` on that alias line points to [4], so let's see what Joel says!  I suspect changing it to `alias_prefix('mochitest-browser-chrome-e10s')` would fix this.

[1] http://gecko.readthedocs.io/en/latest/taskcluster/taskcluster/attributes.html#unittest-try-name
[2] https://queue.taskcluster.net/v1/task/DeVpSCI1Sp-H2SVz7PGL_w/runs/0/artifacts/public/full-task-graph.json
[3] https://dxr.mozilla.org/mozilla-central/source/taskcluster/taskgraph/try_option_syntax.py#59
[4] https://hg.mozilla.org/mozilla-central/rev/71346fd5e1ec
Flags: needinfo?(jmaher)
:dustin, this has been my complaint for the last 6 months about the differences between buildbot and taskcluster- if we change this then windows/osx will not run tests, only linux will.  I assume we can support both: mochitest-e10s-browser-chrome-X and mochitest-browser-chrome-e10s-X
Flags: needinfo?(jmaher)
So I told Blake to file here but I think I misunderstood his problem. This might actually be a bug in ./mach try. For example:

$ ./mach try -b o -p linux64 -u mochitest-browser-chrome-e10s
# successfully runs tests

$ ./mach try -b o -p linux64 -u mochitest-browser-chrome-e10s --and browser/
# no tests found matching filters

There are obviously e10s enabled browser-chrome tests under the /browser directory, but mach try errors out. So while I agree that the difference between taskcluster and buildbot is confusing, I believe that is not the issue Blake has (though please correct me if I'm wrong).
Product: Taskcluster → Testing
you should be able to support both with aliases, yes -- but not by making an alias that expands to nothing :)
Comment on attachment 8841095 [details]
Bug 1342510: fix -u mochitest-e10s-foo aliases to point to nonempty sets of tests;

https://reviewboard.mozilla.org/r/115428/#review116950

nice and simple!
Attachment #8841095 - Flags: review?(jmaher) → review+
Assignee: nobody → dustin
Andrew, if you would like to fix the mach `try issue`, feel free to take the bug and mark leave-open.
Flags: needinfo?(ahalberstadt)
Pushed by dmitchell@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/46e57ca6f77e
fix -u mochitest-e10s-foo aliases to point to nonempty sets of tests; r=jmaher
Probably cleaner to just move this back to Taskcluster::Task Configuration and file another bug for the `mach try` issue.
Component: General → Task Configuration
Flags: needinfo?(ahalberstadt)
Product: Testing → Taskcluster
See Also: → 1342909
https://hg.mozilla.org/mozilla-central/rev/46e57ca6f77e
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla54
Product: TaskCluster → Firefox Build System
You need to log in before you can comment on or make changes to this bug.