1889111 - Prototype instrumenting the Firefox Desktop frontend with events by using BrowserUsageTelemetry

Status: RESOLVED FIXED

(Toolkit :: Telemetry, task, P1)

Description

[Chris H-C :chutten]

As part of our exploration into the lightspeed instrumentation of Firefox Desktop, we've determined we could learn a lot by designing, implementing, testing, shipping, and validating a prototype instrumentation system within BrowserUsageTelemetry.

Comment 1

[Chris H-C :chutten]

Okay, Travis, what do you think of these design notes? https://docs.google.com/document/d/1W-N3PnyayfOvIsGKDNJyTvvjg0VMrE2QmZWH3-1WXaw/edit

I was going to just put them here in the bug as a comment, but I realized I wanted feedback on it first. So I intend to copy this stuff out and put it here later, but first I'd love if you could take a look and offer your opinions.

Comment 2

[Travis Long [:travis_]]

I agree, Option 2 with fewer events and using the event_extras to encode the BrowserUsageTelemetry id and context is going to be the best way to move fast on this (and still give us some balance between fine and coarse grained server knobs control, depending on the "few" number of events defined).

Comment 3

[Chris H-C :chutten]

Attachment: Proposed Design (markdown)

Comment 4

[Chris H-C :chutten]

Repeating "option 2" here:

---

Option 2: Generate a single event (or small number of events) and put everything in extras

We keep coming back to this discussion of whether "we" might "prefer" to have specific or generic events (for definitions of "we" and "prefer"). For Glean.js automatic instrumentation we went with "generic events": where there is a small number of event definitions whose various and several uses are determined through the values of its extra fields. We have a single automatic click event, and its extras will tell you what was clicked and under what context.

For no-code events this would work fairly well and mimic the existing keyed-scalar instrumentation fairly clearly as there's a small number of keyed scalars (one per location or UI interaction method) but the big discrimination is done in the information encoded in those compound keys.

However, there is not yet nor any future plans to create any mechanism by which Server Knobs can turn on or off the recording of a specific event record based on its contents. If we wish to be able to turn pieces of this on and off remotely, we cannot go down this route.

---

We're not sure that there is a valid use-case for enabling only some events, so the caveat at the end might not be relevant. It is now one of the purposes of the prototype to investigate that.

I think that's everything I need to start fiddling with some code. Let's get to it.

Comment 5

[Chris H-C :chutten]

Attachment: data collection review request

Comment 6

[Travis Long [:travis_]]

Comment on attachment 9395135 [details]
data collection review request

Data Review

1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, through the metrics.yaml file and the Glean Dictionary.

2. Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, through the data preferences in the application settings.

3. If the request is for permanent data collection, is there someone who will monitor the data over time?

N/A, collection to end in version 132

4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction data (from out of band conversations with :chutten)

5. Is the data collection request for default-on or default-off?

Default-on

6. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No

7. Is the data collection covered by the existing Firefox privacy notice?

Yes

8. Does the data collection use a third-party collection tool?

No

Result

data-review+

Comment 7

[Chris H-C :chutten]

Attachment: Bug 1889111 - Instrument Firefox Desktop UI Interactions with events r?TravisLong!,kcochrane!

Assumption: Browser Usage Telemetry (ideally) records only and all interesting
interactions with Firefox Desktop's UI, and preserving syntax and semantics
when instrumenting using events is valuable.

Options we could consider:

• A single interaction event instead of one per source, with the source put as
  an extra (like in unknown).
• Putting events in the "events" ping instead of a custom ping
  (Though it makes it harder to turn things on/off remotely)

Value this provides over existing keyed scalars:

• Order of operations (did three tabs open and then three tabs close, or did
  a single tab open-close three times?)
• Flow control (several atomic interactions combine to a user task. flow_id
  grouping allows us to see that easily in analysis. e.g. Open a tab, open
  prefs, privacy prefs, change a setting.)
• Glean

This is aiming for prototype quality and a prototype lifetime, to see if it's
worth investing more than just a week or two into.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/16cab86ffacf
Instrument Firefox Desktop UI Interactions with events r=TravisLong,kcochrane,Gijs

Comment 9

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/16cab86ffacf