1441070 - Instrument toolbox panel opening with event telemetry

Status: RESOLVED FIXED

(DevTools :: General, enhancement)

Description

[:Harald Kirschner :digitarald]

The events are documented in the Amplitude ping format: https://docs.google.com/spreadsheets/d/1PvLMiSQeq-XMPhgUr1GpVGCtg64XwGhOMQ8hZSYR07Q/edit#gid=879531055

:gfritzsche owns the mapping from Event Telemetry pings into Amplitude. So far we agreed to use the first set of parameters to break down the Amplitude event name, leave `value` empty and use the `extra` object for user and event properties.

For row 22 the event telemetry entry ([timestamp, category, method, object, value, extra]) might look like:

[timestamp, "dt", "enter", "console", "", {host: "bottom", width: 956, message_count: 100, cold: true}]

We should get a patch up for review first that reports the telemetry, which will help us find edge cases where our simple assumptions on formatting fell short.

Comment 1

[:Harald Kirschner :digitarald]

Assigning to me to figure out resources with the team.

Comment 2

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

In the spreadsheet we currently have this e.g. "dt_console - enter":

Category        Method  Object  Value     extra
devtools.main   enter   console null      { host: "bottom", width: 1450, message_count: 27, cold: true }

So a couple of questions:
1. So which is my source of truth, this bug or the spreadsheet?
2. In the spreadsheet we use devtools.main as the category but in comment 0 you use dt... which is correct? For the toolbox open event we used devtools.main.
3. If we are dropping value will we be doing the same for exit, activate and deactivate?

Comment 3

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

I get what digitarald wants: We are changing enter.pane.console to e.g. enter.console (moving the value to the object column and setting value to null).

So the following events:
dt_console - enter
dt_inspect - enter
dt_debug - enter
dt_style - enter
dt_net - enter
dt_storage - enter
dt_misc - enter

Will share some combination of:
- host
- width
- message_count (int)
- start_state (inspect_dom, paused_on_exception, jump_to_file)
- panel_name
- cold (bool)

@digitarald Do you have a more complete definition of start_state? I mean, it could just be a tab switch, menu selection, session restore... there is an almost endless list.

Comment 4

[:Harald Kirschner :digitarald]

> @digitarald Do you have a more complete definition of start_state? I mean, it could just be a tab switch, menu selection, session restore... there is an almost endless list.

This would be free-form as it might change over time as panels evolve.

Comment 5

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

Attachment: data-review.txt

Comment 6

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

So we need to gather the following:

1. Services.telemetry.recordEvent("devtools.main", "enter", "console", null, {
     "host": "...",
     "width": "...",
     "start_state": "tab_click",
     "message_count": "...",
     "panel_name": "webconsole",
     "cold": "..."
   });
2. Services.telemetry.recordEvent("devtools.main", "enter", "inspector", null, {
     "host": "...",
     "width": "...",
     "start_state": "inspect_dom, tab_click",
     "panel_name": "inspector",
     "cold": "..."
   });
3. Services.telemetry.recordEvent("devtools.main", "enter", "debugger", null, {
     "host": "...",
     "width": "...",
     "start_state": "paused_on_exception, jump_to_file, jump_to_line, debugger_statement, breakpoint, conditional_breakpoint, tab_click etc.",
     "panel_name": "jsdebugger",
     "cold": "..."
   });
4. Services.telemetry.recordEvent("devtools.main", "enter", "style_editor", null, {
     "host": "...",
     "width": "...",
     "start_state": "jump_to_file, jump_to_line, tab_click",
     "panel_name": "styleeditor",
     "cold": "..."
   });
5. Services.telemetry.recordEvent("devtools.main", "enter", "network_monitor", null, {
     "host": "...",
     "width": "...",
     "start_state": "tab_click",
     "panel_name": "netmonitor",
     "cold": "..."
   });
6. Services.telemetry.recordEvent("devtools.main", "enter", "storage_inspector", null, {
     "host": "...",
     "width": "...",
     "start_state": "tab_click",
     "panel_name": "storage",
     "cold": "..."
   });
7. Services.telemetry.recordEvent("devtools.main", "enter", "other", null, {
     "host": "...",
     "width": "...",
     "start_state": "tab_click",
     "panel_name": "other",
     "cold": "..."
   });

Comment 7

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

tab_click should be tab_switch

Comment 8

[François Marier [:francois]]

Comment on attachment 8969226 [details]
data-review.txt

> -  panel_name: The name of the panel opened, console, inspector, debugger, style_editor, network_monitor, storage_inspector or other

By "other" here, do you mean the literal string 'other' or that it could be some other string?

Comment 9

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

The literal string "other"

Comment 10

[François Marier [:francois]]

Comment on attachment 8969226 [details]
data-review.txt

1) Is there or will there be **documentation** that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes, in Events.yaml.

2) Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, telemetry setting.

3) If the request is for permanent data collection, is there someone who will monitor the data over time?**

Yes, Harald Kirschner.

4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under?  **

Category 2.

5) Is the data collection request for default-on or default-off?

Default-on, all channels.

6) Does the instrumentation include the addition of **any *new* identifiers** (whether anonymous or otherwise; e.g., username, random IDs, etc.  See the appendix for more details)?

No.

7) Is the data collection covered by the existing Firefox privacy notice?

Yes.

8) Does there need to be a check-in in the future to determine whether to renew the data?

No, permanent.

Comment 11

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

Attachment: Bug 1441070 - Instrument toolbox panel navigation with event telemetry

This is pretty simple... finding the best locations for the probes was the
struggle here.

In order to land these probes before the freeze the test for these events will
be created as part of bug 1456073.

Review commit: https://reviewboard.mozilla.org/r/238908/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/238908/

Comment 12

[Yulia Startsev [:yulia] OOO until July 2026]

Comment on attachment 8970115 [details]
Bug 1441070 - Instrument toolbox panel navigation with event telemetry

https://reviewboard.mozilla.org/r/238908/#review244542

Looks good to me

Comment 13

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

Comment on attachment 8970115 [details]
Bug 1441070 - Instrument toolbox panel navigation with event telemetry

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/238908/diff/1-2/

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3577a6b9b1a1
Instrument toolbox panel navigation with event telemetry r=yulia

Comment 15

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/3577a6b9b1a1

Comment 16

[:Harald Kirschner :digitarald]

Moving to M1 for splitting off M2 work.