1512938 - Data review for the core metrics in the baseline ping

Status: RESOLVED FIXED

(Toolkit :: Telemetry, enhancement, P1)

Description

[Alessio Placitelli [:Dexter]]

Glean has its own metrics and, for this reason, it should probably have a data-review about them before starting sending any data.

Comment 1

[Alessio Placitelli [:Dexter]]

The documentation for the current pings is now living at: https://github.com/mozilla-mobile/android-components/blob/master/components/service/glean/docs/pings.md

Comment 2

[Alessio Placitelli [:Dexter]]

Attachment: glean-baseline-request.md

Hi Chenxia,

this is a follow-up to the email thread and the discussion we had on vidyo. I left a few question empty as they relate to the product more than the library, and I wasn't really able to classify the "client_id" field. Please let me know if there's anything else that I need to do here.

Comment 3

[Chenxia Liu [:liuche]]

Comment on attachment 9032199 [details]
glean-baseline-request.md

1) Is there or will there be **documentation** that describes the schema for the ultimate data set available publicly, complete and accurate?
Yes, the default fields in the pings are documented in the repo as *.md files
https://github.com/Dexterp37/android-components/blob/61a271806bc3955049587cebdc050caca42b0ba5/components/service/glean/docs/baseline.md
https://github.com/Dexterp37/android-components/blob/61a271806bc3955049587cebdc050caca42b0ba5/components/service/glean/docs/pings.md

2) Is there a control mechanism that allows the user to turn the data collection on and off? (Note, for data collection not needed for security purposes, Mozilla provides such a control mechanism) Provide details as to the control mechanism available.

Yes, Mozilla products that use this library must submit data reviews for additional telemetry collected, and add a toggle for users to turn this off. Glean SDK does not need to provide this as it is just a library.

3) If the request is for permanent data collection, is there someone who will monitor the data over time?**
This will be consumed by products and they will handle data monitoring.
The baseline data is listed in the docs (sessions, os, etc).

4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under?  **
Baseline: primarily Type 1 (os, build version) and also Type 2 for session data (duration, experiments, etc)
Ping: Type 1

5) Is the data collection request for default-on or default-off?
Depends on the library consumer

6) Does the instrumentation include the addition of **any *new* identifiers** (whether anonymous or otherwise; e.g., username, random IDs, etc.  See the appendix for more details)?
Includes Client ID, but this is used only for telemetry, and should not be used anywhere else.

7) Is the data collection covered by the existing Firefox privacy notice? **If unsure: escalate to legal if:**
Yes, adding a library to consolidate data collection tools for Mobile - it ports over the baseline data collection that is currently collected, and should obsolete the other telemetry libraries.

* The data includes new identifiers; OR

* The data falls within the Web activity category AND is default-on.

8) Does there need to be a check-in in the future to determine whether to renew the data? (Yes/No) (If yes, set a todo reminder or file a bug if appropriate)**

Comment 4

[Chenxia Liu [:liuche]]

Thanks for the clear communication with the Data Stewards while this project is going on, Alessio! It has been very helpful and I appreciate the transparency in getting this right :)

A few more need-info points, to follow up from our earlier conversation:
- Can you add a section at the top in the Glean SDK README to users of the library, that for the products that they use this library with they MUST both a) document new telemetry that is collected with this library, b) provide a way for users to turn the data off, and c) go through data review? Please link to the data review form: https://github.com/mozilla/data-review/blob/master/request.md
- A line in the README (or somewhere else if there's a better place) for contributors to the library that adding new collection fields to Glean SDK needs to be both documented and data-reviewed
- I also noticed that there are still two sections of the documentation that are missing: "events" and "metrics"
https://github.com/Dexterp37/android-components/blob/61a271806bc3955049587cebdc050caca42b0ba5/components/service/glean/docs/pings.md
Can you let me know when those will be updated?

Thanks again!

Comment 5

[Alessio Placitelli [:Dexter]]

(In reply to Chenxia Liu [:liuche] from comment #4)
> Thanks for the clear communication with the Data Stewards while this project
> is going on, Alessio! It has been very helpful and I appreciate the
> transparency in getting this right :)

My pleasure, I'm glad this worked out so well! Thank you so much for all your efforts in helping move this forward!

> A few more need-info points, to follow up from our earlier conversation:
> - Can you add a section at the top in the Glean SDK README to users of the
> library, that for the products that they use this library with they MUST
> both a) document new telemetry that is collected with this library, b)
> provide a way for users to turn the data off, and c) go through data review?
> Please link to the data review form:
> https://github.com/mozilla/data-review/blob/master/request.md
> - A line in the README (or somewhere else if there's a better place) for
> contributors to the library that adding new collection fields to Glean SDK
> needs to be both documented and data-reviewed

I've just changed the readme file to address these points, feel free to take a look at the PR here: https://github.com/mozilla-mobile/android-components/pull/1655

> - I also noticed that there are still two sections of the documentation that
> are missing: "events" and "metrics"
> https://github.com/Dexterp37/android-components/blob/
> 61a271806bc3955049587cebdc050caca42b0ba5/components/service/glean/docs/pings.
> md
> Can you let me know when those will be updated?

The documentation for the events ping is in our priority list, and happening as part of bug 1515974. The metrics ping is not being sent yet, but documentation will come as soon as it lands.

Since, by themselves, they are only containers for events and metrics that need to be data-reviewed separately, and we're not sending nothing more than what's already in the "ping_info" section for the baseline ping, does these pings require data review as well?

Comment 6

[Chenxia Liu [:liuche]]

Okay! That makes sense, that nothing else sent in the events and metrics, and only baseline by default (other than what's already been reviewed in ping) so that's fine. Thanks! I think this bug can be closed now.