Closed
Bug 1927493
Opened 5 days ago
Closed 5 days ago
Delta and Score appear to disagree across platforms
Categories
(Testing :: PerfCompare, defect)
Testing
PerfCompare
Tracking
(Not tracked)
RESOLVED
DUPLICATE
of bug 1926213
People
(Reporter: mgaudet, Unassigned)
Details
Using this perfCompre as an example:
This is the same benchmark, on Linux vs Windows
- Windows Base has a higher score Windows New
- Linux Base has a lower score than Linux New
Yet both report a negative Delta(%). One of those presumably has a sign error? Unfortunately, I can't really reverse-engineer the math on the delta percent either? E.g. for windows, the Delta(%) is reported as -0.57%, but...
- (100*20.51/20.42)-100 = 0.4407443683
- (100*20.42/20.51)-100 = -0.4388103364
So not sure where the sign error is here.
Comment 1•5 days ago
|
||
I think this comes from the fact that we currently show the Median score, not the Mean score, but the delta is the delta of Means. This is clearly confusing to the users.
We will change it shortly to show the Mean (like the old tool) instead of the Median. And in the future we'll gather feedback about which one is the most interesting.
Updated•5 days ago
|
You need to log in
before you can comment on or make changes to this bug.
Description
•