1819634 - It's too difficult to look at m-c results for try graphs

Status: NEW

(Tree Management :: Perfherder, defect)

Description

[:Gijs (he/him)]

Sometimes perf compare shows you changes that look... suspect... even with a few replicates (e.g. 5). If you run enough tests, pure statistics (and the noisyness of the tests) mean some of them will show differences. To quickly check if these are representative, it would be nice if you could easily doublecheck the result on central.

Maybe having a dedicated button for this would be possible, I don't know how difficult that is.

Given there is no such button, I figured I'd copy-paste the full description of the test.

But the current UI makes the name of the test a link. If you click that link (e.g. on this graphs page, the dialog to add more tests opens, but it does not preselect anything (unsure if that's a bug, or what the link is supposed to do).

So the next best thing would be to be able to copy-paste text from the existing data series to filter for m-c instead. So I tried to select the text while pressing alt, which normally allows text selection inside <a href> style links. However, although the name of the test looks like a link, it's actually a <button> and so you can't select the text on it at all.

So then I tried to find the name manually. But when you click "add test data", the dialog obscures the existing test data, and the dialog cannot be moved. So I have to basically memorize the extended test description, and then try to retype it into the filter textbox and pick out the right subtest.

This meant that just "get the data for the exact same subtest on central for a given dataset on try", which feels like something that should take seconds, take many minutes and some frustration (not helped by bug 1819632).

Comment 1

[:Gijs (he/him)]

(Embarrassingly, this difficulty also meant I ended up looking at the wrong data for a while, and was puzzled as to why it was so different! Turned out that although the descriptions of the new data I added were very similar, some of them were for "cold" runs and some for "warm" runs, and of course the performance results were quite different...)