1586764 - Consider reporting the number of pings sent

Status: RESOLVED FIXED

(Data Platform and Tools :: Glean: SDK, task, P3)

Description

[Chris H-C :chutten]

I was thinking of corner cases that could prevent the "baseline" ping from satisfying analysis needs for baseline engagement measurement, and thought of kiosks. Maybe Firefox is used to show a map in an Airport Terminal and thus never gets sent to the background and spends days on the same page.

Now, the "metrics" and "events" pings should still be sent. But how will we know whether any "baseline" pings have been sent in between?

I propose we report the number of pings we submit and monitor:

• the difference between the number we report and the number we receive
• the number of pings that come without any other pings in between

That'll help us recognize the shape of any "kiosk problem", and also act as a good resource for introspective Glean Health evaluation and monitoring.

I see it taking the shape of a ping-lifetime labelled counter glean.pings_submitted reported on both "metrics" and "baseline" pings where the labels are the names of the pings. It'd be a dynamic list of ping names, unfortunately (unless we want to get fancy in our build step with multiple glean_parser steps to generate the ping list and substitute it into our own metrics.yaml. (and this won't help us if we start supporting dynamic registration of pings (e.g. from addons))). We could then detect kiosks by looking for "metrics" pings without any "baseline" pings having been sent, and compare the values between "metrics" and "baseline" pings to ensure they're consistent.

Comment 1

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/glean/pull/1478

Comment 2

[Michael Droettboom [:mdroettboom]]

Attachment: Data review request

Comment 3

[Travis Long [:travis_]]

Comment on attachment 9200520 [details]
Data review request

1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?
   Yes, in the Glean Book

2. Is there a control mechanism that allows the user to turn the data collection on and off?
   N/A, the opt out is per product that uses Glean and is controlled by the standard Glean opt-out mechanism.

3. If the request is for permanent data collection, is there someone who will monitor the data over time?
   Yes, Michael Droettboom

4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?
   Category 1

5. Is the data collection request for default-on or default-off?
   default-on

6. Does the instrumentation include the addition of any new identifiers?
   No

7. Is the data collection covered by the existing Firefox privacy notice?
   Yes

8. Does there need to be a check-in in the future to determine whether to renew the data?
   No

9. Does the data collection use a third-party collection tool?
   No

Result: data-review+