1410430 - Consider adding a visual summary of subtests on top-level compare view

Status: RESOLVED DUPLICATE of bug 1316590

(Tree Management :: Perfherder, task, P3)

Description

[Robert Wood [:rwood]]

Consider adding a subtests summary column to the top-level perferherder compare revisions page, so that without clicking 'subtests' you can get a quick visual idea of the subtest results.

For example, here on the top-level compare page [1] we see a regression, however if you click on 'subtests' you see a whole bunch of subtest improvements along with a few regressions.

Just an idea, possibly a 'subtests' column at that level [1] that has green with the number of subtest improvements and red with the number of subtest regressions.

[1] https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-central&newProject=try&newRevision=482d2433fe40ca1a618270a19a876ce1a762c590&framework=1&filter=speedo&showOnlyImportant=1&selectedTimeRange=172800

Comment 1

[William Lachance (:wlach)]

One easy thing to try would be to put a subtest summary in a tooltip that you get when hovering over the subtests (similar to the other tooltips in the compare view, e.g. for distribution)

Comment 2

[Robert Wood [:rwood]]

(In reply to William Lachance (:wlach) (use needinfo!) from comment #1)
> One easy thing to try would be to put a subtest summary in a tooltip that
> you get when hovering over the subtests (similar to the other tooltips in
> the compare view, e.g. for distribution)

I like that idea! I'll take this and give that a try when I have a chance.

Comment 3

[Robert Wood [:rwood]]

Hey Will, so I'm a bit rusty at perfherder...

In a patch I have the tooltip appearing on the 'subtests' link no problem, but now for the hard part, getting the summary values.

On the compare view, for the 'subtests' link, the actual subtest compare results are only calculated at the time when that link is actually clicked and the subtest ctrl table is presented (I think hah).

Which means if I want to have a 'subtests' link tooltip on the top-level compare view page, the subtest results aren't actually available yet, am I correct?

If so, I'm not sure it would be worth the overhead of calculating the subtest comparison results ahead of time just to get regression and improvement counts for the tooltip, and then having them calculated again if/when the 'subtests' link is actually clicked and the subtest page presented. Especially for a test like speedometer with a huge number of subtests.

What do you think? Thanks!

Comment 4

[William Lachance (:wlach)]

> If so, I'm not sure it would be worth the overhead of calculating the
> subtest comparison results ahead of time just to get regression and
> improvement counts for the tooltip, and then having them calculated again
> if/when the 'subtests' link is actually clicked and the subtest page
> presented. Especially for a test like speedometer with a huge number of
> subtests.

Right, I think it only makes sense to fetch the subtest information and display it when the user hovers over the subtest link (I think fetching the data should only take a few seconds at most). I would use a similar strategy that Shruti took with the inline distribution view last summer:

https://github.com/mozilla/treeherder/blob/master/ui/js/components/perf/compare.js#L114

Hopefully it shouldn't be too difficult to extend that to your use case. To reduce duplication, you may want to factor out the code that creates the comparison map out of CompareSubTestCtrl into a factory/directive.

Comment 5

[Robert Wood [:rwood]]

(In reply to William Lachance (:wlach) (use needinfo!) from comment #4)
...
Ok thanks for the feedback Will!