Closed Bug 1337081 Opened 4 years ago Closed 3 years ago

Add/improve interface to mark active experiments in TelemetryEnvironment

Categories

(Toolkit :: Telemetry, defect, P2)

defect

Tracking

()

RESOLVED DUPLICATE of bug 1348748
Tracking Status
firefox54 --- affected

People

(Reporter: gfritzsche, Unassigned)

References

Details

(Whiteboard: [measurement:client])

We should standardize on one field in the environment data that specifies which experiment(s) are running.
This allows for common tooling/faceting/filtering/...

Constraints:
- allow multiple experiments (in case we need that later)
- allow specifying optional cohorts

Pre-existing: environment/activeExperiments, TelemetryExperiments.jsm
We should integrate this properly if we supersede it.
We need to specify the properties and data format we want here.
From there we can add an API that makes it convenient for test pilot, SHIELD & Telemetry experiments to consistently track active experiments.
It would be great to facet on this field on telemetry.mozilla.org for easy comparison with the base population (as well as automatically detecting significant changes in distribution of histograms and scalars as we do now by channel+build).
Note that some datasets and dashboards already depend on this field as it currently works:

Dashboard: http://bsmedberg.github.io/telemetry-experiments-dashboard/
CrashSummaryView (proposed): https://github.com/mozilla/telemetry-batch-view/blob/master/src/main/scala/com/mozilla/telemetry/views/CrashSummaryView.scala

As for faceting tmo on it, there's a prior bug on file with bug 1224264.
s/experiments/phased rollout of possibly experimental code.

1.  Does this seem like a good statement of the problem... 

We want to know what clientId's are in SPECIAL COHORTS so that we can more quickly identify those cohorts during analysis?

Standardizing on this field seems like one solution to that problem.
And also, branches within the population (by which I mean, branch should definitely be a field :))

Also, we'll want some way to keep track of when experiments were enabled/disabled
Blocks: 1324436
Other "cohort"-like uses include funnelcake & distributionid.
I think i'll have to summarize this bug so far and post to fhr-dev for broader input.
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1348748
You need to log in before you can comment on or make changes to this bug.