1825863 - Initial parallel Glean telemetry for spotlight and onboarding

Status: RESOLVED FIXED

(Firefox :: Messaging System, task, P1)

Description

[Perry McManis [:perry.mcmanis]]

Also let this bug serve as a standin for glean-parser as a temporary data review link.

This bug covers a patch that has instrumentation for About:Welcome and Telemetry Feed in the Onboarding Messaging component. It uses Glean to instrument the same data as exists in PingCentre, and follows the Pings as Events (name subject to change) paradigm to do so. Please refer to the latest phabricator patch if you'd like to know more, as it contains significant context about specific decisions.

Comment 1

[Perry McManis [:perry.mcmanis]]

Attachment: Bug 1825863 - Add a Glean version of About Welcome telemetry r?dmose

Comment 2

[Perry McManis [:perry.mcmanis]]

Attachment: WIP: Bug 1825863 - Add a Glean version of Spotlight telemetry

Depends on D174679

Comment 3

[Dan Mosedale (:dmosedale, :dmose)]

(In reply to Perry McManis [:perry.mcmanis] from comment #0)

Also let this bug serve as a standin for glean-parser.

What does this mean?

Comment 4

[Perry McManis [:perry.mcmanis]]

(In reply to Dan Mosedale (:dmosedale, :dmose) from comment #3)

(In reply to Perry McManis [:perry.mcmanis] from comment #0)

Also let this bug serve as a standin for glean-parser.

What does this mean?

One of the requirements that Glean places on data is that there is a link to a work tracking bug (the bug field in *.yaml) and one link to a data review (the data review field in the same). Because I will not actually be collecting any data as part of this effort, at least not that is just information off of my own computer/dev env, I am using this bug to satisfy the check. Without it, you actually get build errors.

Comment 5

[Perry McManis [:perry.mcmanis]]

Attachment: Bug 1825863 - Add a Glean version of Spotlight telemetry r?dmose

This stack is an example of how you might instrument telemetry found in spotlight.jsm in Glean. It aims to closely mirror the PingCentre format.

This follows the "pings as events" paradigm. An entirely new ping is created, defined in a pings.yaml file, and to this we add the metrics we like. However, this ping represents a single instance of an event in a sense; by sending the ping as soon as we attach data to it, each ping contains only the data on it from that interaction.

Each metric is registered in an appropriate metrics.yaml file (it doesnt particularly matter which one you choose as glean parser will handle it as long as there is an entry), and the send_in_pings section is used to attach the metric to your new custom ping.

Comment 6

[Perry McManis [:perry.mcmanis]]

Attachment: Bug 1825863 - Add a Glean version of Spotlight telemetry r?dmose

Comment 7

[Chris H-C :chutten]

Attachment: Bug 1825863 - Add some prototype Glean instrumentation tests for asrouter r?dmose!,perry.mcmanis!

Comment 8

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: Bug 1825863 - WIP: Add some prototype Glean instrumentation tests for asrouter r?perry.mcmanis

This Work-In-Progress patch stack for feedback:

• Has been mostly written by Perry McManis, except for the npm patch, which was written by Chris Hutten, despite what Phab says.
• Captures key kinds of telemetry that need to be migrated
• Is expected to evolve into the (somewhat larger) final patch set
• Targeted to land by May 19th (end of 115.1), with work being primarily done by Perry and (most likely) me
• Have a variety of small changes/errors that need to be cleaned up; at least one test currently fails

High level-strategy:

• Context:
  ** For general Glean/FOG context applies here, see Glean/FOG see Appendix A of the remediation plan.
  ** Structure (and to a large extent, the content) of Glean pings needs to be pre-defined in YAML files
  *** Any given ping must predefine whether it sends (both legacy & Glean) clientIds, so we have two different pings for ASRouter events. Note that as written, this patch only sends Glean clientIds, NOT legacy clientIds, though that could be changed.
• Get PingCentre JSON structure after complete (ie policies applied etc); submit
• Create Glean Ping by converting JSON & submit
• Misc:
  ** Data Science has reviewed and OKed the overall structure of the pings this code generates
  ** After data starts flowing into raw tables, Data Science will propose table views for querying against, which will be updated daily.
  ** We started iterating using more typical Glean idioms but ultimately chose to minimize risk with simpler code and similar data structure at the cost of using Glean in a less typical way. Glean instrumentation built up over time from scratch would look substantially different.

Feedback desired - high-level questions & comments, among other things:

• Motivations for non-obvious things

• Structure of code

• The pings the code generates (timing, structure, information...)

• Things we’ve missed

Before fleshing out the YAML and other code bits we need to verify:

• There shouldn’t be issues around the proposed JSON flattening
• This work covers all existing types of data (review against existing docs + some src))
• That attribution (& any other middleware) is being correctly captured
• Should we consider sending the legacy clientId in addition to the Glean one? If so, why?

Fleshing out these patches to get them to a landable state:

• Fix broken tests in the patches, add assertions
• Add Glean assertions to (some subset of?) existing tests (particularly adding assertions about not having invalid keys).
• Fix the failures
• …

Comment 9

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: Bug 1825863 - WIP: Add a Glean version of Spotlight telemetry r?chutten

Comment 10

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: Bug 1825863 - WIP: add a Glean version of About Welcome telemetry r?chutten

Comment 11

[Perry McManis [:perry.mcmanis]]

Attachment: Bug 1825863 - Add a Glean version of Spotlight telemetry r?dmose

New patch that contains all relevant changes

Comment 12

[Perry McManis [:perry.mcmanis]]

Attachment: Bug 1825863 - WIP: Add a Glean version of Spotlight telemetry r?chutten

Comment 13

[Perry McManis [:perry.mcmanis]]

Attachment: Bug 1825863 - WIP: Add some prototype Glean instrumentation tests for asrouter r?dmose

Comment 14

[Chris H-C :chutten]

Attachment: Bug 1825863 - Cover asrouter ping with unit tests r?dmose!

Comment 15

[Chris H-C :chutten]

Attachment: Bug 1825863 - Add a Glean version of onboarding and ASRouter telemetry r?dmose!

We consolidate all the "messaging-system"-namespace pings into a single Glean
ping to provide a holistic view of all events in that system.

Comment 16

[Perry McManis [:perry.mcmanis]]

Attachment: Bug 1825863 - DO NOT LAND VERSION Add a Glean version of onboarding and ASRouter telemetry r?dmose!

Test patch

Comment 17

[Chris H-C :chutten]

Attachment: data collection review request

A note that all the other data collections being added are not new and are merely reimplementations of existing PingCentre data collections. This review request is for only the new stuff (error checks) as (per Trust) the existing stuff does not require further review.

Comment 18

[Travis Long [:travis_]]

Comment on attachment 9335475 [details]
data collection review request

Data Review

1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, through the metrics.yaml file and the Glean Dictionary.

2. Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, through the telemetry preferences in the application settings.

3. If the request is for permanent data collection, is there someone who will monitor the data over time?

Permanent collection to be monitored by [e-mail], [e-mail], [e-mail]

4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 1, Technical data

5. Is the data collection request for default-on or default-off?

Default-on

6. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No

7. Is the data collection covered by the existing Firefox privacy notice?

Yes

8. Does the data collection use a third-party collection tool?

No

Result

data-review+

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/30b2192cb8d6
Add a Glean version of onboarding and ASRouter telemetry r=dmose,perry.mcmanis,omc-reviewers

Comment 20

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/30b2192cb8d6