1274578 - Try push appearing to pick up extra task cluster tasks: "-u mochitest-1" runs mochitest-bc1, etc

Status: RESOLVED FIXED

(Taskcluster :: General, defect)

Description

[Geoff Brown [:gbrown]]

+++ This bug was initially created as a clone of Bug #1267848 +++

For buildbot, "try: -u mochitest-1" runs mochitest-1.

For taskcluster, "try: -u mochitest-1" seems to run mochitest*1, which includes mochitest-1, mochitest-bc1, mochitest-dt1, etc.

The discrepancy is unexpected and can lead to many more tests running than intended.

I suspect this is related to how aliases are interpreted: https://hg.mozilla.org/mozilla-central/annotate/c67dc1f9fab8/testing/taskcluster/tasks/branches/base_job_flags.yml#l5

Comment 1

[Geoff Brown [:gbrown]]

Here's another example: https://treeherder.mozilla.org/#/jobs?repo=try&revision=47751985813b

try: -b o -p android-api-15 -u mochitest-1 -t none

For the buildbot build, only mochitest-1 was run.

For the TC build, mochitest-1, mochitest-chrome, mochitest-clipboard, mochitest-gl1, mochitest-gpu, and mochitest-mda1 were run. (There's no mochitest-dt or mochitest-bc on Android -- 'mochitest-1' runs even more tests on desktop.)

Comment 2

[Geoff Brown [:gbrown]]

Attachment: do not search for alias when <name>-<chunk> specified

This just skips the alias handling when the <name>-<chunk> form is matched.

Now "-u mochitest-1" only selects mochitest-1: https://treeherder.mozilla.org/#/jobs?repo=try&revision=67dead256193, https://treeherder.mozilla.org/#/jobs?repo=try&revision=59b1a7444a2e. This is consistent with traditional, buildbot try syntax, and I think this is more intuitive.

"-u mochitest" continues to select all mochitests, as before: https://treeherder.mozilla.org/#/jobs?repo=try&revision=9339c9598eaa, https://treeherder.mozilla.org/#/jobs?repo=try&revision=11a514871014

This change breaks one test case: "e10s-<chunk>". That is not recognized for buildbot and is, IMO, not very useful. I've removed that test case.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=8495ba393d83

"-u e10s" continues to act as before, and is consistent with buildbot: https://treeherder.mozilla.org/#/jobs?repo=try&revision=7245e7c02164

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8781230 [details] [diff] [review]
do not search for alias when <name>-<chunk> specified

Review of attachment 8781230 [details] [diff] [review]:
-----------------------------------------------------------------

Bonus: it's simpler than the old version :)

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/4d1fa46b05cb
Do not use alias when specific try -u test requested; r=dustin

Comment 5

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/4d1fa46b05cb