1337081 - Add/improve interface to mark active experiments in TelemetryEnvironment

Status: RESOLVED DUPLICATE of bug 1348748

(Toolkit :: Telemetry, defect, P2)

Description

[Georg Fritzsche [:gfritzsche]]

We should standardize on one field in the environment data that specifies which experiment(s) are running.
This allows for common tooling/faceting/filtering/...

Constraints:
- allow multiple experiments (in case we need that later)
- allow specifying optional cohorts

Pre-existing: environment/activeExperiments, TelemetryExperiments.jsm
We should integrate this properly if we supersede it.

Comment 1

[Georg Fritzsche [:gfritzsche]]

We need to specify the properties and data format we want here.
From there we can add an API that makes it convenient for test pilot, SHIELD & Telemetry experiments to consistently track active experiments.

Comment 2

[Mark Reid [:mreid]]

It would be great to facet on this field on telemetry.mozilla.org for easy comparison with the base population (as well as automatically detecting significant changes in distribution of histograms and scalars as we do now by channel+build).

Comment 3

[Chris H-C :chutten]

Note that some datasets and dashboards already depend on this field as it currently works:

Dashboard: http://bsmedberg.github.io/telemetry-experiments-dashboard/
CrashSummaryView (proposed): https://github.com/mozilla/telemetry-batch-view/blob/master/src/main/scala/com/mozilla/telemetry/views/CrashSummaryView.scala

As for faceting tmo on it, there's a prior bug on file with bug 1224264.

Comment 4

[Gregg Lind (Fx Strategy and Insights - Shield - Heartbeat )]

s/experiments/phased rollout of possibly experimental code.

1.  Does this seem like a good statement of the problem... 

We want to know what clientId's are in SPECIAL COHORTS so that we can more quickly identify those cohorts during analysis?

Standardizing on this field seems like one solution to that problem.

Comment 5

[bugzilla]

And also, branches within the population (by which I mean, branch should definitely be a field :))

Also, we'll want some way to keep track of when experiments were enabled/disabled

Comment 7

[Georg Fritzsche [:gfritzsche]]

Other "cohort"-like uses include funnelcake & distributionid.
I think i'll have to summarize this bug so far and post to fhr-dev for broader input.