1557048 - Investigate adding "reason codes" to the `metrics` ping

Status: RESOLVED FIXED

(Data Platform and Tools :: Glean: SDK, enhancement, P3)

Description

[Michael Droettboom [:mdroettboom]]

:chutten commented "I kinda/sorta miss the "reason" field when performing these analyses. I liked being able to tell why the ping was sent, and it would help answer questions like these without guessing."

We should investigate whether adding this to Glean is a good idea and probably begin with a design document.

Comment 1

[Travis Long [:travis_]]

Chris, what would you like to see, specifically?

Comment 2

[Chris H-C :chutten]

I think it would be beneficial for the "events" and "metrics" pings a reason field explaining why they were submitted when they were submitted. Was the "metrics" ping sent because the app was awake at 4AM? Was it sent because it was the first session? Was it sent because we woke up and found that 4AM had passed?

The comment specifically was referencing "events" pings where I couldn't tell which ones were sent at which times without doing some sort of "start_time is close enough to end_time" fuzzy math.

I originally thought it'd be useful in ping_info, but seeing as "baseline" has only one reason to be sent and custom pings have custom reasons, it's only really relevant to "events" and "metrics".

Comment 3

[Alessio Placitelli [:Dexter]]

I'd like to propose the following set of reasons for the metrics ping:

• overdue: the ping was collected immediately because it was past 4AM local when the check was performed.
• today: the ping was to be collected in the same calendar day the scheduling occurred (e.g. this would cover the case of the check happening at 3AM and the ping generated at 4AM)
• tomorrow: the ping was to be collected on the next calendar day compared to when the scheduling occurred (e.g. check happening at 5AM and the collection happening at 4AM on the next calendar day).

Note that all of the above might be tricky to get right :)

Thoughts?

Comment 4

[Chris H-C :chutten]

Sounds fine by me. What would make it tricky?

Comment 5

[Travis Long [:travis_]]

I'm okay with the 3 'reasons' that you propose, so long as the explanations are clearly documented so we know what they represent. It would also be nice to have examples or a list of situations that could trigger each case to help explain why a ping is scheduled.

Comment 6

[Jan-Erik Rediger [:janerik]]

Sign-off on these 3 reasons.
Also like to see :chutten's question answered. I figure we might be able to just set a string metric right before we collect the ping?

Comment 7

[Michael Droettboom [:mdroettboom]]

Sounds good to me as well.

Another use case for the enumeration metric we don't have... :)

Comment 8

[Alessio Placitelli [:Dexter]]

(In reply to Chris H-C :chutten from comment #4)

Sounds fine by me. What would make it tricky?

The fact that, depending on when the workmanager is triggered, we might not really know why it was triggered (did we get a 4am signal from WM today? or is it for "tomorrow"?)

Comment 9

[Michael Droettboom [:mdroettboom]]

Adding a note (and whiteboard tag) that this will be useful for investigating issues like the "baseline incident".

Comment 10

[Alessio Placitelli [:Dexter]]

Untaking this, it might change depending on where we go with the metrics ping.

Comment 11

[Michael Droettboom [:mdroettboom]]

Attachment: Data review request

Comment 12

[Michael Droettboom [:mdroettboom]]

Attachment: GitHub Pull Request

Comment 13

[Chris H-C :chutten]

Comment on attachment 9120877 [details]
Data review request

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. This collection is documented in a couple of places. The ping reasons will be documented with the "metrics" ping documentation and in the Glean SDK's [pings.yaml](https://github.com/mozilla/glean/blob/master/glean-core/pings.yaml). The reason metric that will contain the reason until it's sent is documented with the [metrics that the Glean SDK collects](https://mozilla.github.io/glean/book/user/collected-metrics/metrics.html#metrics-1) and also in the Glean SDK's [metrics.yaml](https://github.com/mozilla/glean/blob/master/glean-core/metrics.yaml)

    Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. All applications embedding the Glean SDK must provide an opt-out mechanism. For example, Fenix has its opt-out in its Settings UI.

    If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, Michael Droettboom is responsible.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 1, Technical.

    Is the data collection request for default-on or default-off?

Default on for all channels.

    Does the instrumentation include the addition of any new identifiers?

No.

    Is the data collection covered by the existing Firefox privacy notice?

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data?

No. This collection is permanent.

---
Result: datareview+