Open Bug 2026354 Opened 1 month ago Updated 1 month ago

Mixing together scores and times leads to wierd sign-flip-flops on CLES and Cliff's Delta.

Categories

(Testing :: PerfCompare, enhancement)

Product:
Testing
Component:
PerfCompare
Type:
enhancement

Tracking

(Not tracked)

People

(Reporter: mgaudet, Unassigned)

References

Details

(Whiteboard: [pcf])

Basically: Score and Time get different ends of the normalized range for a regression vs. an improvement:

  • Score improvements get low CLES and regressions get high CLES
  • Time improvements get high CLES and regressions get low CLES

I think something similar is true for Cliff's delta though, I think Cliff's delta already sorts by magnitude so it's sort of extra confusing.

Whiteboard: [pcf]
You need to log in before you can comment on or make changes to this bug.