1272433 - [Meta] Improve talos regression template

Status: RESOLVED FIXED

(Tree Management :: Perfherder, defect)

Description

[William Lachance (:wlach)]

We have a bunch of different ideas on how to improve the talos regression template, some easier to implement than others (although none are especially hard). Let's use a metabug to track them.

We should hopefully improve the talos regression template in ways that should make it easier to generalize regression templates to different projects (bug 1260805).

Comment 1

[William Lachance (:wlach)]

So we've implemented everything we discussed a month or so back. Is there anything further to do here with the bug filing template or should we close this out?

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

I believe there was the one issue :h4writer wanted where he could get a current text summary of the regressions on demand- I would like that as well if we could.

Comment 3

[Avi Halachmi (:avih)]

(In reply to Joel Maher (:jmaher) from comment #2)
> I believe there was the one issue :h4writer wanted where he could get a
> current text summary of the regressions on demand- I would like that as well
> if we could.

This ^ is important IMO too.


(In reply to Avi Halachmi (:avih) from bug 1277364 comment #2)
> 0. The actual regression numbers (hopefully improvements too).
> 
> N. A link to something which shows #0 in more details and allows further
> digging, be it an alert manager, compare view, etc (personally I think a
> compare view would be great).

- Bug 1277364 should have improvements too now, right?
- Is it possible to see an example of it in action, preferably with improvements too?
- Did we end up with a link to the data in more details?

Comment 4

[William Lachance (:wlach)]

Yup, this (In reply to Avi Halachmi (:avih) from comment #3)
> (In reply to Joel Maher (:jmaher) from comment #2)
> > I believe there was the one issue :h4writer wanted where he could get a
> > current text summary of the regressions on demand- I would like that as well
> > if we could.
> 
> This ^ is important IMO too.

Fixed in bug 1284325.

> (In reply to Avi Halachmi (:avih) from bug 1277364 comment #2)
> > 0. The actual regression numbers (hopefully improvements too).
> > 
> > N. A link to something which shows #0 in more details and allows further
> > digging, be it an alert manager, compare view, etc (personally I think a
> > compare view would be great).
> 
> - Bug 1277364 should have improvements too now, right?
> - Is it possible to see an example of it in action, preferably with
> improvements too?
> - Did we end up with a link to the data in more details?

Yup, this is all implemented as of today. You can test it yourself by going to an alert on treeherder stage and clicking on the classification in the top right corner (you will get options to file a bug or copy the alert contents to a clipboard)

https://treeherder.allizom.org/perf.html#/alerts?id=2859

Going to mark this bug as fixed.

Comment 5

[Avi Halachmi (:avih)]

Looks great. Two things which I'd have said earlier if I saw an example of the summary-copy-as-text before it landed:

1. The date would be useful to, because the values can change over time at the alerts manager, so at least know that the view is NN days/weeks old.

2. It would be useful to have also the before/after numbers, and not only the change in %, because on some tests the absolute value is as important as the change percentage. E.g. copy "<test name>: before: X   After: Y   Change: NN% improvement/regression"