1569024 - select the right test for the graph backdrop when adding more test data to graph

Status: NEW

(Tree Management :: Perfherder, enhancement, P3)

Description

[Steve Fink [:sfink] [:s:]]

See eg https://treeherder.mozilla.org/perf.html#/graphs?series=try,fcefb979eac44d057f9c05434580ce7845f4c2d6,1,1&highlightedRevisions=476cf1aea384,2f1bef963074&timerange=1209600

If I want to get rid of the useless try repo comparisons and switch to mozilla-central, it makes me manually find the same test name. I would expect the common case to be choosing the exact same test I'm already looking at.

Next behind that would be looking at the same test with different configurations, but I guess that would be harder.

It's not just because it's a nuisance to find the matching test name in the list. It's also a UI issue, where I was confused for a long time what the added test data was all about. (Especially since the graph doesn't currently highlight my chosen revisions at all, and I suspect might not even display them if they are out of the date range selected?) I now think it's supposed to be to get an overview at a glance of the historical values, their variance, any multimodality, etc., and see your results in the context of those data. (Which is why I'm calling it the "backdrop".) That's really useful, but unclear from the UI. When I experimented with adding in more test data, the main thing to see was a long list of tests, most of them not related to the test I had just clicked on "graph" for, so I thought I must be doing something wrong.

Comment 1

[Sarah Clements [:sclements]]

(In reply to Steve Fink [:sfink] [:s:] from comment #0)

See eg https://treeherder.mozilla.org/perf.html#/graphs?series=try,fcefb979eac44d057f9c05434580ce7845f4c2d6,1,1&highlightedRevisions=476cf1aea384,2f1bef963074&timerange=1209600

If I want to get rid of the useless try repo comparisons and switch to mozilla-central, it makes me manually find the same test name. I would expect the common case to be choosing the exact same test I'm already looking at.

Next behind that would be looking at the same test with different configurations, but I guess that would be harder.

If you hover over the test name, repository and platform of the legend card in your example link, you'll see tooltips explaining a few options. Clicking on one of those will open the test data modal with pre-filled fields. This should probably be made more obvious. Does this solve the issue?

I noticed hovering over linux64 shows a 'Add related branches' note which should be 'Add related platforms'. I'll get that fixed in my next pr.

It's not just because it's a nuisance to find the matching test name in the list. It's also a UI issue, where I was confused for a long time what the added test data was all about. (Especially since the graph doesn't currently highlight my chosen revisions at all, and I suspect might not even > display them if they are out of the date range selected?)

I added a comment in your bug 1569021. It's true that the way the UI is designed now, you don't get any sort of indication that a revision is outside of the x axis date range. I'll see about adding a helpful message.