Open Bug 1655019 Opened 2 months ago Updated 2 months ago

Display a name/unit for summary results

Categories

(Tree Management :: Perfherder, enhancement, P3)

enhancement

Tracking

(Not tracked)

People

(Reporter: davehunt, Unassigned)

Details

(Whiteboard: [perf:workflow])

Summary results are used when multiple test results are summarised, for example the various technical pageload results are summarised as a geometric mean. This summary result is primarily used for regression detection.

Currently, when displaying the summary results in Perfherder or other dashboards there is no way to know the method used to summarise the results. We should require summary results to provide a name/unit that can be displayed to provide this extra context.

For example, the following tests are shown in the Add Test Data modal in the graphs view:

  • amazon opt cold
  • amazon SpeedIndex opt cold
  • amazon FirstVisualChange opt cold
  • etc

The first test (amazon opt cold) is a summary result, but this is not immediately obvious. If you include subtests you'll see the additional tests that are used in the summary result, but it's still not very clear:

  • amazon opt cold
  • amazon dcf opt cold
  • amazon fcp opt cold
  • amazon fnbpaint opt cold
  • amazon loadtime opt cold
  • amazon SpeedIndex opt cold
  • amazon FirstVisualChange opt cold
  • etc

Proposed improvement when subtests are not displayed:

  • amazon opt cold summary
  • amazon SpeedIndex opt cold
  • amazon FirstVisualChange opt cold
  • etc

Proposed improvement when subtests are displayed:

  • amazon opt cold summary
    • dcf
    • fcp
    • fnbpaint
    • loadtime
  • amazon SpeedIndex opt cold
  • amazon FirstVisualChange opt cold
  • etc

When displaying the results in graphs, the above summary example could display the unit as "geometric mean".

You need to log in before you can comment on or make changes to this bug.