Showing average test times is pretty useless: should show average of maximum test times instead

NEW
Unassigned

Status

Testing Graveyard
GoFaster
7 years ago
2 years ago

People

(Reporter: wlach, Unassigned)

Tracking

Trunk
x86_64
Linux

Details

Right now the dashboard has this view for seeing what the average time for a test is:

http://brasstacks.mozilla.com/gofaster/#/executiontime/test

In actual fact, this is kinda useless though, as our main interest is in test times that are holding us back. It doesn't matter if our average is 30 minutes (e.g.) per test if there's a single test that regularly takes 90 minutes.

I propose that we revise this metric to be an average of maximum test times (that is, take the maximum test time on each platform for each run, and then divide that by the number of runs in the day).

Comment 1

6 years ago
Perhaps pointing the name of the worst offender also helps (see bug 700815).
(Assignee)

Updated

2 years ago
Product: Testing → Testing Graveyard
You need to log in before you can comment on or make changes to this bug.