Closed Bug 1426163 Opened 3 years ago Closed 2 years ago

Provide a CLI for generating summary metrics for shield pref-flipping experiments

Categories

(Data Platform and Tools :: Telemetry APIs for Analysis, enhancement, P3)

x86_64
Linux
enhancement
Points:
3

Tracking

(Not tracked)

RESOLVED DUPLICATE of bug 1485804

People

(Reporter: harter, Assigned: harter)

References

Details

I want to be able to easily generate summary metrics for a shield pref-flipping experiment.  I'd like the metrics to be output in a textual format so they can be included in a report and annotated.

I'd like the initial set of metrics to be percentage changes between experiment and base for 4-5 metrics with an included confidence interval.  The confidence interval would ideally not rely on a normal hypothesis.

An initial set of metrics could include:

* Session Length
* URI Counts
* Total Searches
* Crashes

Eventually, I'd like there to be similar functionality for all experiment results used to make a decision.  I.e. I'd like all experiment summary tools to have textual export functions.
See Also: → 1397298
Duplicate of this bug: 1394612
Unlikely I'll make progress in Q1. Hoping to tackle this in Q2
Priority: P1 → P3
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1485804
You need to log in before you can comment on or make changes to this bug.