2055727 - Need a "mach try" to schedule betasim and run a try preset

Status: NEW

(Developer Infrastructure :: Try, enhancement)

Description

[Rob Wu [:robwu]]

I am trying to verify a fix to a betasim failure at bug 2051934, and needed a way to run a betasim and run specific tests (from a try preset). But I did not find a way to do so, except by hacking the try logic locally.

The documentation at https://wiki.mozilla.org/Sheriffing/How_To/Beta_simulations#TRUNK_AS_EARLY_BETA suggests to run this command to run a betasim locally:

./mach try release -v 154.0b1 --tasks release-sim --migration main-to-beta --stage-changes

When I run that, it stages some changes locally, comparable to https://hg-edge.mozilla.org/try/rev/464c830ccbed1a5c4d34736c5986c707c1ab5ac8 (I used b15 instead of b1).

I however do not find a way to select the try preset AND run a betasim at once.

So what I ended up doing is:

• Run betasim locally as shown above, and commit it: https://hg.mozilla.org/try/rev/464c830ccbed1a5c4d34736c5986c707c1ab5ac8
• Patch the preset to include the directory of the test that I want to run: https://hg-edge.mozilla.org/try/rev/d85791fff36202239b4ab88faba61273e9d1ceff
• Patch tools/tryselect/push.py to manually set the following parameters in the (generated) try_task_config.json file:
  • "try_mode": "try_task_config" as documented at https://firefox-source-docs.mozilla.org/taskcluster/try.html#complex-configuration
  • "release_type": "beta", as shown in the suggested try_task_config when running the betasim command (./mach try release ... above).
  • Patch I made: https://hg.mozilla.org/try/rev/1891a2d5858b7356383d0b4b95d0c1a553a46201
• run ./mach try --preset=webextensions --no-artifact to schedule the try push: https://treeherder.mozilla.org/jobs?repo=try&revision=971a7c16ae05ebbac9878127a95d6f5808df86a3

Surely there is a better way to do it?

I also tried generating the try_task_config.json file manually and committed it, but then ./mach try --no-push showed that it did not work (my custom try_task_config.json would apparently be overridden), contrary to what is documented at https://firefox-source-docs.mozilla.org/taskcluster/try.html#empty-try .