1352207 - Compare view should show a tooltip on bars saying whether the result was better or worse

Status: RESOLVED FIXED

(Tree Management :: Perfherder, enhancement)

Description

[William Lachance (:wlach)]

We don't color bars when the comparison result is uncertain to reduce visual noise, but it's hard to know whether the result is actually better or worse at a glance that way (the only way to really know is the direction of the bar, which is pretty counter intuitive). For example these linux64 compositor cases that came up for dbaron:

https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-central&originalRevision=03d602fd723a&newProject=try&newRevision=4243fd25743565f31fa933f3d006cb8d00c382c6&framework=1&filter=basic%20linux&showOnlyImportant=0

Since we already have the information, we could at least add a tooltip to the bar; that would probably be enough for the target audience for this feature.

Comment 1

[William Lachance (:wlach)]

Looking more closely, let's not add a tooltip to the bar. Instead, let's change the text for the percentage change from:

"Mean difference: +/- XXX (= +/- Y%)" to "Mean difference: +/- XXX (= Y% better/worse)

(i.e. add a better/worse description, use absolute value for percentage)

Examples:

* "Mean difference: -0.12 (2.38% better)"
* "Mean difference: 1.12 (5.25% better)"

You will need to modify the code around here:

https://github.com/mozilla/treeherder/blob/master/ui/partials/perf/comparetable.html#L85

We will show better or worse depending on whether this value is true:

compareResult.isRegression

And change the percentage to absolute value (I think you can just use Math.abs for this but am not sure)

Shruti, would you like to do this? Should be a relatively easy one.

Comment 2

[Shruti Jasoria [:ShrutiJ]]

Sure, I'll do it. Assigning myself to it.

Comment 3

[GitHub Autolander Bot]

Attachment: [treeherder] SJasoria:Bug_1352207 > mozilla:master

Comment 4

[Shruti Jasoria [:ShrutiJ]]

Comment on attachment 8856659 [details] [review]
[treeherder] SJasoria:Bug_1352207 > mozilla:master

I have made a few changes to the code. While testing, I found that it is showing "better" for all the tests. 

Should I use compareResult.isImprovement instead of !compareResult.isRegression for the case when test results are better? 

https://github.com/mozilla/treeherder/blob/master/ui/js/services/perf/compare.js#L288

Comment 5

[William Lachance (:wlach)]

Comment on attachment 8856659 [details] [review]
[treeherder] SJasoria:Bug_1352207 > mozilla:master

Based on the documentation you linked to, it looks like "newIsBetter" is the one you want (since it gives a result irrespective of confidence). Give that a try!

Comment 6

[Shruti Jasoria [:ShrutiJ]]

Comment on attachment 8856659 [details] [review]
[treeherder] SJasoria:Bug_1352207 > mozilla:master

I have made a few changes. I think it works well now.

Comment 7

[William Lachance (:wlach)]

Comment on attachment 8856659 [details] [review]
[treeherder] SJasoria:Bug_1352207 > mozilla:master

One small nit :)

Comment 8

[Shruti Jasoria [:ShrutiJ]]

Comment on attachment 8856659 [details] [review]
[treeherder] SJasoria:Bug_1352207 > mozilla:master

Removed the extra space. :)

Comment 9

[William Lachance (:wlach)]

Comment on attachment 8856659 [details] [review]
[treeherder] SJasoria:Bug_1352207 > mozilla:master

Thank you! Feel free to resolve the bug.

Comment 10

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/39e5e7c4ea3adae6aec3a39973113d35aef218f0
Bug 1352207 - Compare view should show a tooltip on bars saying whether the result was better or worse (#2333)