733468 - Summarize telemetry data by including "acceptable" cutoffs for key metrics.

Status: RESOLVED WONTFIX

(Mozilla Metrics :: Frontend Reports, defect)

Description

[[:Cww]]

In telemetry, for each measure (or some key measures) we should have a cutoff and then we should report what % of submissions are greater than cutoff. It's a much better way of counting than using mean or median or even percentiles and can boil metrics down to a single easy-to-track number.

Say the cutoff for GC_MS is 300. Just tell me what percent of hits are > 300 and let me slice and dice that on a graph to show how that changes over time or between releases or between OSes.

Comment 1

[[:Cww]]

To clarify, the metrics team isn't responsible for coming up with cutoffs, that's the job of product/engineering/whoever is requesting the measure.  This probably only applies to a number of key summary metrics (like GC_MS, but not GC_SWEEP for example).

Comment 2

[Annie Elliott]

Marking: in group of > 33 asks for Telemetry that need PM priority before triage/scheduling.

Comment 3

[Annie Elliott]

Triaged.