1555005 - mach try fuzzy does not run same jobs on try server as local

Status: NEW

(Firefox Build System :: Task Configuration, defect)

Description

[Greg Tatum [:gregtatum]]

Steps to reproduce:

I ran mach try fuzzy locally. Then entered a query interactively to select the tasks I wanted.

> test-windows10-64/debug-mochitest-browser-chrome-e10s-7
  test-windows10-64/debug-mochitest-browser-chrome-e10s-6
  test-windows10-64/debug-mochitest-browser-chrome-e10s-5
  test-windows10-64/debug-mochitest-browser-chrome-e10s-4
  test-windows10-64/debug-mochitest-browser-chrome-e10s-3
  test-windows10-64/debug-mochitest-browser-chrome-e10s-2
  test-windows10-64/debug-mochitest-browser-chrome-e10s-1
  test-windows10-64/opt-mochitest-devtools-chrome-e10s-5
  test-windows10-64/opt-mochitest-devtools-chrome-e10s-4
  test-windows10-64/opt-mochitest-devtools-chrome-e10s-3
  test-windows10-64/opt-mochitest-devtools-chrome-e10s-2
  test-windows10-64/opt-mochitest-devtools-chrome-e10s-1
  test-windows10-64/opt-mochitest-browser-chrome-e10s-7
  test-windows10-64/opt-mochitest-browser-chrome-e10s-6
  test-windows10-64/opt-mochitest-browser-chrome-e10s-5
  test-windows10-64/opt-mochitest-browser-chrome-e10s-4
  test-windows10-64/opt-mochitest-browser-chrome-e10s-3
  test-windows10-64/opt-mochitest-browser-chrome-e10s-2
  test-windows10-64/opt-mochitest-browser-chrome-e10s-1
  test-windows10-64/debug-mochitest-media-spi-e10s
  test-windows10-64/debug-mochitest-chrome-1proc-3
  test-windows10-64/debug-mochitest-chrome-1proc-2
  test-windows10-64/debug-mochitest-chrome-1proc-1
  test-windows10-64/opt-mochitest-media-spi-e10s
  test-windows10-64/opt-mochitest-chrome-1proc-3
  test-windows10-64/opt-mochitest-chrome-1proc-2
  test-windows10-64/opt-mochitest-chrome-1proc-1
  test-windows10-64/debug-mochitest-media-e10s
  test-windows10-64/debug-mochitest-a11y-1proc
  test-windows10-64/opt-mochitest-media-e10s
  test-windows10-64/opt-mochitest-a11y-1proc
  test-windows10-64/debug-mochitest-gpu-e10s
  test-windows10-64/opt-mochitest-gpu-e10s
  test-windows10-64/debug-mochitest-e10s-5
  test-windows10-64/debug-mochitest-e10s-4
  test-windows10-64/debug-mochitest-e10s-3
  test-windows10-64/debug-mochitest-e10s-2
  test-windows10-64/debug-mochitest-e10s-1
  test-windows10-64/opt-mochitest-e10s-5
  test-windows10-64/opt-mochitest-e10s-4
  test-windows10-64/opt-mochitest-e10s-3
  test-windows10-64/opt-mochitest-e10s-2
  test-windows10-64/opt-mochitest-e10s-1
  For more shortcuts, see mach help try fuzzy and man fzf
  accept: <enter>, cancel: <ctrl-c>, cursor-down: <ctrl-j>, cursor-up: <ctrl-k>, select-all: <ctrl-a>, toggle-select: <tab>
  249/3047
> 'linux64 | 'macos | 'windows10 !-qr !talos mochitest | xpcshell !shippable !asan !webgl !source-test

I then exited out and run the query from the command line.

./mach try fuzzy -q "'linux64/ | 'macosx64/ | 'windows10-64/ "'!'"shippable mochitest-browser opt | debug"

The following try run was generated:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=a7ed9ec931659bd71af9d012a8c35c116749f19d

Expected results

It to generate the same task list as what was displayed locally through the interactive interface, as the -q documentation suggests that would be the case.

  -q STR, --query STR   Use the given query instead of entering the selection
                        interface. Equivalent to typing <query><ctrl-a><enter>
                        from the interface. Specifying multiple times
                        schedules the union of computed tasks.

Actual results:

It generated the following try_task_config.json, which did not include all of the tasks.

new file mode 100644
--- /dev/null
+++ b/try_task_config.json
@@ -0,0 +1,14 @@
+{
+    "tasks": [
+        "test-macosx1010-64/debug-xpcshell-e10s-2"
+    ],
+    "templates": {
+        "artifact": {
+            "enabled": "1"
+        },
+        "env": {
+            "TRY_SELECTOR": "fuzzy"
+        }
+    },
+    "version": 1
+}