1428796 - DEVTOOLS_ENTRY_POINT will expire in 60.0a1

Status: RESOLVED FIXED

(DevTools :: General, enhancement)

Description

[Alexandre Poirot [:ochameau]]

This telemetry tracks how users open the tools: key shortcut, menu, ...
It sounds like the kind of probe we want to be permanent.

Comment 1

[Alexandre Poirot [:ochameau]]

Attachment: Bug 1428796 - Make devtools entry point telemetry never expire.

Review commit: https://reviewboard.mozilla.org/r/211030/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/211030/

Comment 2

[Alexandre Poirot [:ochameau]]

Comment on attachment 8940758 [details]
Bug 1428796 - Make devtools entry point telemetry never expire.

Harald, I imagine we always want this data, right?
https://telemetry.mozilla.org/new-pipeline/dist.html#!cumulative=0&end_date=2018-01-07&keys=__none__!__none__!__none__&max_channel_version=nightly%252F59&measure=DEVTOOLS_ENTRY_POINT&min_channel_version=nightly%252F59&processType=*&product=Firefox&sanitize=0&sort_keys=submissions&start_date=2017-11-13&table=0&trim=1&use_submission_date=0

Comment 3

[:Harald Kirschner :digitarald]

Comment on attachment 8940758 [details]
Bug 1428796 - Make devtools entry point telemetry never expire.

https://reviewboard.mozilla.org/r/211030/#review216978

LGTM. This should to be permanent. Do we need a data peer review when we make probes permanent?

Comment 4

[Alexandre Poirot [:ochameau]]

Comment on attachment 8940758 [details]
Bug 1428796 - Make devtools entry point telemetry never expire.

I have no idea, let's ask.

Chenxia, could you review this change? Do we have to ask for data review when making one probe record permanently?

Comment 5

[Chenxia Liu [:liuche]]

Hi Alexandre, I know this has already gone through data review, but can you fill out this form? And please also make a case for why this should be monitored permanently.
https://github.com/mozilla/data-review/blob/master/request.md

Having a clear reason for collecting the data makes it clear when to remove a probe (even if the original person who adds the probe leaves, or changes projects, etc).

From out data policies:
https://wiki.mozilla.org/Firefox/Data_Collection

> Necessity - We collect only as much data as is necessary when we can demonstrate a clear business case for that data

Since we probably won't be removing devtools, this seems like it should be fine, but please document here in this bug! (I'll also  clarify this with the other data peers in our monthly meeting if this is necessary, but please bear with my erring on the side of caution!)

Comment 6

[Alexandre Poirot [:ochameau]]

1. What questions will you answer with this data?

How users enabled the DevTools?

2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements? Some example responses:

* Better understand how developers use DevTools. Knowing how they open it is a key factor.
* When we compare this to other measurements, it would help use understanding people who aren't developer and trigger the tools by accident

3. What alternative methods did you consider to answer these questions? Why were they not sufficient?

I'm not sure we considered any other method.

4. Can current instrumentation answer these questions?

Yes

5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories on the found on the Mozilla wiki.

"Records how the user is triggering Developer Tools startup." - Category 2 - bug 1378863

6. How long will this data be collected? Choose one of the following:

I want to permanently monitor this data.

7. What populations will you measure?

Everyone, but the action triggering this probe is very specific to DevTools.

8. Please provide a general description of how you will analyze this data.

See what are the most used entry points and focus quality work on them.
Then, ensure that we understand any change if the most popular entry point change over time.
Any major change may highlight a performance issue, an unreported breakage or a UX issue.

9. Where do you intend to share the results of your analysis?

I think Harald is using this in product dashboard, but may be he is only using telemetry dashboard.

Comment 7

[Chenxia Liu [:liuche]]

Comment on attachment 8940758 [details]
Bug 1428796 - Make devtools entry point telemetry never expire.

https://reviewboard.mozilla.org/r/211030/#review219470

Data-review only

> Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?
Yes, Histograms

> Is there a control mechanism that allows the user to turn the data collection on and off? (Note, for data collection not needed for security purposes, Mozilla provides such a control mechanism) Provide details as to the control mechanism available.
Yes, Firefox data controls

> If the request is for permanent data collection, is there someone who will monitor the data over time?**
Yes, apoirot will monitor - requested permanent data collection in order to monitor any changes to DevTools entry points for changes

> Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under? **
Type 2

> Is the data collection request for default-on or default-off?
Default on

> Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?
No

> Is the data collection covered by the existing Firefox privacy notice?
Yes

> Does there need to be a check-in in the future to determine whether to renew the data? (Yes/No) (If yes, set a todo reminder or file a bug if appropriate)**
No, okay-ing this for permanent collection because DevTools is basic browser functionality and team would like to permanently monitor entry points

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7c67b6dd2450
Make devtools entry point telemetry never expire. r=Harald,liuche

Comment 9

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/7c67b6dd2450