1202718 - Add ability to retrigger talos tests directly from the "compare performance" UI

Status: RESOLVED DUPLICATE of bug 1571643

(Tree Management :: Perfherder, enhancement, P1)

Description

[Brian Grinstead [:bgrins]]

When opening a URL like https://treeherder.mozilla.org/perf.html#/compare?originalProject=fx-team&originalRevision=65e92b72d584&newProject=try&newRevision=2b95b3d44786, some of the runs have a warning:

"More base / new runs recommended for increased confidence in comparison"

It would be really handy if you could retrigger those runs right from this UI, maybe by clicking a button next each of the to the '# runs' column, or by deep linking into the BuildAPI with the particular tests filtered.

Comment 1

[William Lachance (:wlach)]

Yes, some kind of interface to do this easily would be nice. Note that you can access the jobs page for the push by clicking on the revision though.

Comment 2

[Brian Grinstead [:bgrins]]

(In reply to William Lachance (:wlach) from comment #1)
> Note that you
> can access the jobs page for the push by clicking on the revision though.

True.  I don't always know which talos test maps to which suite off the top of my head, but now I see that that it shows up in the "Talos" tab on treeherder so that works.

Comment 3

[MikeLing]

This idea sounds awesome!  I would like to deliver this feature :)

BTW, maybe we should change the summary from "....... from the comparechooser UI" to "...... from the compareview UI"? Because the compare chooser is where we choose the revision to compare rather than display the result ;)

Comment 4

[MikeLing]

Hi Will(:wlach), could you give me some hit about this? I think it's easy to fix and I can commit a workable PR very soon :)

Comment 5

[William Lachance (:wlach)]

(In reply to MikeLing from comment #3)
> This idea sounds awesome!  I would like to deliver this feature :)
> 
> BTW, maybe we should change the summary from "....... from the
> comparechooser UI" to "...... from the compareview UI"? Because the compare
> chooser is where we choose the revision to compare rather than display the
> result ;)

Good point.

Comment 6

[William Lachance (:wlach)]

(In reply to MikeLing from comment #4)
> Hi Will(:wlach), could you give me some hit about this? I think it's easy to
> fix and I can commit a workable PR very soon :)

You are right that actually retriggering a run should be fairly easy. But the devil is in the details. :) You might want to retrigger runs for either the base revision or the new revision, or both. We also need to be somewhat careful about not inadvertently encouraging people to trigger too many jobs redundantly, as that can be very expensive in terms of machine time.

So if I were going to do this, I'd probably add another option next to each row when user hovers it (in addition to 'subtests' and 'graph') called 'retrigger'. When it is clicked on, pop up a modal dialog (http://angular-ui.github.io/bootstrap/#/modal) which shows how many times the job associated with the test has been triggered (either pending or running) for both the base and new revisions, and provide some kind of simple way to schedule an arbitrary number of additional jobs.

If you're still interested in doing this, let me know and I can probably spec out a more complete user interface mockup and more implementation details.

Comment 7

[MikeLing]

(In reply to William Lachance (:wlach) from comment #6)
> (In reply to MikeLing from comment #4)
> If you're still interested in doing this, let me know and I can probably
> spec out a more complete user interface mockup and more implementation
> details.

of course! thank you very much!

> We also need to be somewhat careful about not inadvertently encouraging people to trigger too many jobs >redundantly, as that can be very expensive in terms of machine time. 

Maybe we can disable the button when it been clicked or the job is still running. But I don't know if we have api could help us query job status or trigger job. could you give some advices about that? :)

Comment 8

[William Lachance (:wlach)]

(In reply to MikeLing from comment #7)

> > We also need to be somewhat careful about not inadvertently encouraging people to trigger too many jobs >redundantly, as that can be very expensive in terms of machine time. 
> 
> Maybe we can disable the button when it been clicked or the job is still
> running. But I don't know if we have api could help us query job status or
> trigger job. could you give some advices about that? :)

Sorry, I haven't forgotten about this, just haven't had time to note down my thoughts here:

So what we need to do here is query the job id associated with the result for a particular datum (this is a property, along with value, etc.). From there, that should be enough to know how to retrigger a job (I don't know the details of how this is done, just basing that assertion on the fact that you can retrigger from the job detail panel from treeherder).

You should also be able to query treeherder for whether there are similar jobs scheduled or in progress for the result set in question on that particular branch and display that in the dialog. 

Most of the relevant APIs should be in the treeherder code under ui/. I don't know the details, sorry. :) But if you poke around the code to do with the job panel and retriggers you should be able to figure out how it works hopefully without too much trouble.

I think the dialog could look like this:

-------------------------------------------
| base               | new                |
| 4 results          | 2 results          |
| 1 scheduled        | 0 scheduled        |
| Schedule [  ] more | Schedule [  ] more |
|------------------------------------------
|                              |Schedule| |
-------------------------------------------

Something like that, anyway.