1529883 - Add telemetry to the element picker

Status: RESOLVED FIXED

(DevTools :: Inspector, enhancement, P2)

Description

[Martin Balfanz [:mbalfanz]]

In preparation to the visual picker project, I'd like to get a baseline of data for our current element picker. This will make it easier for us to judge how the new feature performs.

I'd like to add a probe for the number of times the picker is used, similar to scalars_devtools.accessibility.picker_used_count. This should track general usage irrespective of the entry point (e.g. toolbar icon vs. keyboard shortcut).

Comment 1

[Razvan Caliman [:rcaliman]]

Attachment: Bug 1529883 - Add telemetry instrumentation to count the number of invocations of the element picker. r=gl

Agreed with PM ([e-mail]) about the intended behavior: number of invocations, not actual picks is considered sufficient to answer the questions.
Temporary telemetry will be removed in Firefox 70.

Comment 2

[Razvan Caliman [:rcaliman]]

Attachment: data-review-element-picker-used.md

Hi Chris,

May I please get your review of this data collection proposal? We want to count the number of times the element picker is used in the Inspector to judge whether this is a viable interaction model for an upcoming feature in DevTools.

Comment 3

[Chris H-C :chutten]

Comment on attachment 9048899 [details]
data-review-element-picker-used.md

Preliminary note:

Have you given thought to how you will normalize this value? Will it be "number of element picker uses per.... day? devtools toolbox session? client?" . Also, it may help to have a definition ahead of time of what you think "success" looks like for your questions.

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. This collection is Telemetry so is documented in its definitions file [Scalars.yaml](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/telemetry/Scalars.yaml) and the [Probe Dictionary](https://telemetry.mozilla.org/probe-dictionary/).

    Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

    If the request is for permanent data collection, is there someone who will monitor the data over time?

No. This collection will expire in Firefox 70.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction.

    Is the data collection request for default-on or default-off?

Default on for all channels.

    Does the instrumentation include the addition of any new identifiers?

No.

    Is the data collection covered by the existing Firefox privacy notice?

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data?

Yes. :rcaliman is responsible for renewing or removing the collection before it expires in Firefox 70.

---
Result: datareview+

Comment 4

[Martin Balfanz [:mbalfanz]]

Hey Chris! We thought about normalization for the data. We want this to be comparable to our other panel/feature usage numbers, which means "number of element picker uses per day".

The success criteria will be defined in the PRD. We are currently debating two different implementations, both with different criteria. But both depend on understanding our current usage as baseline. That's the main reason we want to understand it before we do any adjustments.

Comment 5

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/f115573cf9a8

Comment 6

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/integration/autoland/rev/f115573cf9a82f025084210a0379a0fc84e2436a