Closed Bug 1426163 Opened 6 years ago Closed 6 years ago

Provide a CLI for generating summary metrics for shield pref-flipping experiments

Categories

(Data Platform and Tools :: General, enhancement, P3)

Product:
Data Platform and Tools
Component:
General
Platform:
x86_64
Linux
Type:
enhancement
Points:
3

Tracking

(Not tracked)

Status:
RESOLVED DUPLICATE of bug 1485804

People

(Reporter: harter, Assigned: harter)

References

Details

I want to be able to easily generate summary metrics for a shield pref-flipping experiment.  I'd like the metrics to be output in a textual format so they can be included in a report and annotated.

I'd like the initial set of metrics to be percentage changes between experiment and base for 4-5 metrics with an included confidence interval.  The confidence interval would ideally not rely on a normal hypothesis.

An initial set of metrics could include:

* Session Length
* URI Counts
* Total Searches
* Crashes

Eventually, I'd like there to be similar functionality for all experiment results used to make a decision.  I.e. I'd like all experiment summary tools to have textual export functions.
See Also: → 1397298
Unlikely I'll make progress in Q1. Hoping to tackle this in Q2
Priority: P1 → P3
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → DUPLICATE
Component: Telemetry APIs for Analysis → General
You need to log in before you can comment on or make changes to this bug.