Closed Bug 1555172 Opened 3 years ago Closed 3 years ago

Extend `setExperimentActive` to support enrollment IDs

Categories

(Toolkit :: Telemetry, enhancement, P2)

enhancement

Tracking

()

RESOLVED FIXED
mozilla70
Tracking Status
firefox70 --- fixed

People

(Reporter: mythmon, Assigned: chutten)

References

(Blocks 1 open bug)

Details

Attachments

(2 files)

In order to help correlate enrollments, unenrollments and other events, and help distinguish between cloned profiles, Normandy would like to associate each call to setExperimentActive with a random, unique id that is generated when a user first enrolls in an experiment. This data should then be accessible by analysts in some way.

One way this could be done is extending the setExperimentActive method to allow for another option, enrollmentId. Normandy could then generate that data, persist it, and pass it to setExperimentActive when needed.

Blocks: 1555178

Is this needed independently of (earlier than?) the broader project in bug 1549760?

Flags: needinfo?(mcooper)

This is independent from 1549760. It's a separate request for more useful Telemetry from Normandy that happened to be at the same time as bug 1549760.

Flags: needinfo?(mcooper)
Assignee: nobody → chutten
Status: NEW → ASSIGNED
Priority: -- → P2

This id, to be generated at runtime by the client, will be used to distinguish
when a client enrolls multiple times in the same id/branch/type.

:mythmon, would you mind writing up the Data Collection Review for this? I'm not 100% clear on how this is intended to be used.

Alternatively, I suppose we could attach the Data Review to bug 1555178 where the IDs will actually be generated and submitted.

Flags: needinfo?(mcooper)

Hey Sunah, does this look like a useful shape for this to be in for Data Engineering? Anything else needing to be done?

Flags: needinfo?(ssuh)
Blocks: 1565600

Looks good to me! I just filed a dependent bug for the data engineering side.

Flags: needinfo?(ssuh)
Flags: needinfo?(mcooper)
Flags: needinfo?(chutten)
Flags: needinfo?(chutten)
Flags: needinfo?(teon)
Attachment #9077814 - Flags: data-review?(teon)

just need a bit of clarification:

  • for the inclusion of the enrollment_id, will this be added as a scalar to the telemetry collection?
  • and for the enrollment_id, this is an existing id, but will just be included in the environment?
Flags: needinfo?(teon) → needinfo?(mcooper)

enrollment_id will be newly generated for each enrollment, and won't be re-used beyond a single experiment. I don't know how exactly this will be represented in the telemetry ping, for that I'd direct you towards :chutten.

We also plan to include the enrollment ID in other pings related to a users enrollment in an experiment, such as Telemetry Events.

Flags: needinfo?(mcooper) → needinfo?(chutten)

It'll be present in the Environment of every ping that contains an Environment. Documentation can be found here: https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/data/environment.html

(( It'll be included in the example and explained in the experiments subsection. It will also have a section detailing its usage in the API documentation here: https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/collection/experiments.html#the-js-api ))

Flags: needinfo?(chutten)
Comment on attachment 9077814 [details]
data-collection-review-1555172.md

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes, it will be present in the Environment of every ping that contains an Environment. Documentation can be found here: https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/data/environment.html

    Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

    If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, :mythmon is responsible.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction.

    Is the data collection request for default-on or default-off?

Default on for all channels with Normandy enabled.

    Does the instrumentation include the addition of any new identifiers?

Yes, enrollment_id. The ID will expire after the experiment is completed.

    Is the data collection covered by the existing Firefox privacy notice?

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data?

No. This collection is permanent.

---
Result: datareview
Attachment #9077814 - Flags: review+
Pushed by chutten@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/acd1ffd459f7
Allow annotating experiments with Enrollment Ids. r=janerik,mythmon
Attachment #9077814 - Flags: review+
Attachment #9077814 - Flags: data-review?(teon)
Attachment #9077814 - Flags: data-review+
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla70

Alicia, I noticed that this collection contains a new persistent ("life of an experiment") random identifier, and one possible parsing of the data collection review form is "escalate to legal if the data includes new identifiers"—I don't have any concerns but wanted to put your eyes on it. AIUI, Bug 1555176 will actually start reporting this identifier.

Flags: needinfo?(agray)
Blocks: 1584337
No longer blocks: 1584337

Hi Tim,

Thanks for the escalation. Trust & Security review = approved; Cat 2 interaction data. Bug 1555176 has also been updated accordingly with Trust signoff.

Flags: needinfo?(agray)
You need to log in before you can comment on or make changes to this bug.