1908345 - [mach-try-perf] Enable triggering a specific test through mach try perf

Status: RESOLVED FIXED

(Testing :: Performance, task, P2)

Description

[Greg Mierzwinski [:sparky]]

This bug is for making it possible to trigger specific tests through mach try perf (with or without --alert). By specific test, I mean something like amazon.

Comment 1

[Greg Mierzwinski [:sparky]]

Attachment: Bug 1908345 - Add --tests option to mach try perf. r?#perftest

This patch adds the abiltiy to run specific tests through mach try perf using the --tests option. When a test is specified, we search through the full-task-graph to find all tasks that run the test based on the command that is running in the task. Test finders are implemented on a per-suite basis to determine if the task contains the requested test. These are then used to (1) create new categories that only run this specific test, and (2) find all tasks that run the requested test. The query for the category is based on the longest common substring between all the tasks that were found, and some heuristics modifying the task label are needed within the test-finders to ensure that we find unique strings that only run the requested tasks/test. These tasks, and categories are then used within the other standard options like the category fuzzy view, the show-all view, and when running specific tests in an alert.

Comment 2

[Greg Mierzwinski [:sparky]]

Attachment: Bug 1908345 - Add tests for --tests test selection. r?#perftest

This patch adds tests for the --tests option. It includes a sample full-task-graph for the task finding stage.

Depends on D216932

Comment 3

[Greg Mierzwinski [:sparky]]

Attachment: Bug 1908345 - Add documentation about --tests option. r?#perftest

This patch adds some documentation about the --tests option, and also updates a bit of the other documentation for mach-try-perf.

Depends on D216933

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/74f136cc9208
Add --tests option to mach try perf. r=perftest-reviewers,kshampur
https://hg.mozilla.org/integration/autoland/rev/11e9ea5824ba
Add tests for --tests test selection. r=perftest-reviewers,kshampur
https://hg.mozilla.org/integration/autoland/rev/a34bfe3d947d
Add documentation about --tests option. r=perftest-reviewers,aglavic

Comment 5

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/74f136cc9208
https://hg.mozilla.org/mozilla-central/rev/11e9ea5824ba
https://hg.mozilla.org/mozilla-central/rev/a34bfe3d947d