1277364 - Simplify talos regression template

Status: RESOLVED FIXED

(Tree Management :: Perfherder, defect)

Description

[William Lachance (:wlach)]

In preparation for bug 1260805, which will make it possible to define regression templates for different projects (arewefastyet). I'd like to simplify the talos regression template in a few ways:

1. Remove the direct links to test documentation for specific tests, instead just link to overall page and let users look stuff up.
2. Remove the specific try syntax, instead just link to the page on running Talos.

A few motivations for this:

1. These explanations and URLs are rather lengthy and verbose.
2. Some of the instructions (e.g. try) can go out of date for an older regression.
3. It's a lot of code to maintain, and it's not something I'm feeling like I want to make generic across the different types of performance frameworks we support.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

4) remove a lot of newlines in the template
5) add the branch to the summary

Comment 2

[Avi Halachmi (:avih)]

0. The actual regression numbers which we're claiming to have detected, sorted from most meaningful to the least meaningful (test name, platform, % change in performance). Or at least the top 5-10 of those.

N. A link to something which shows #0 in more details and allows further digging, be it an alert manager, compare view, etc (personally I think a compare view would be great).

Preferably, #0 or #N will also show improvements which were together with the regressions.

Comment 3

[William Lachance (:wlach)]

Discovered that the running talos wiki page is really out of date, filed bug 1277371 for that.

Comment 4

[William Lachance (:wlach)]

(In reply to Avi Halachmi (:avih) from comment #2)
> 0. The actual regression numbers which we're claiming to have detected,
> sorted from most meaningful to the least meaningful (test name, platform, %
> change in performance). Or at least the top 5-10 of those.

That's bug 1270524, which we should also do soon but is seperate from this bug. :)

Comment 5

[GitHub Autolander Bot]

Attachment: [treeherder] wlach:1277364 > mozilla:master

Comment 6

[William Lachance (:wlach)]

Comment on attachment 8758900 [details] [review]
[treeherder] wlach:1277364 > mozilla:master

Hey, so here's a proposal for the new simplified regression template. We probably don't want to land this before we fix/improve the running talos documentation (bug 1277371).

Does this simplification seem reasonable? I know there are other things we would like to *add* as well, will get to those soon enough. :) 

(trying to get feedback from a wider group, if you have nothing to add, or someone's already set the flag for you, feel free to just cancel the f?)

Comment 7

[Avi Halachmi (:avih)]

Comment on attachment 8758900 [details] [review]
[treeherder] wlach:1277364 > mozilla:master

Good simplification.

But the windows instructions are lacking. I'll update the wiki with windows info.

Comment 8

[William Lachance (:wlach)]

Now that :rwood has fixed up the talos running page, we can land this!

Comment 9

[GitHub Autolander Bot]

Pull request has landed in master: https://github.com/mozilla/treeherder/commit/f15b10bbac754a8fc17f3817b2ae3c57d3c0d8a2

Comment 10

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/8c5cc5c4dccf7365b4e5a36e3f10f9c7154c75bf
Bug 1277364 - Simplify talos bugzilla template

Remove try syntax and links to documentation for specific tests in favor
of just linking to general resources. Shorter and simpler.