1639454 - Introduce telemetry for the Compatibility side panel

Status: RESOLVED FIXED

(DevTools :: Inspector: Compatibility, task, P2)

Description

[Jan Honza Odvarko [:Honza] (always need-info? me)]

This is a follow up for bug 1625132.

The Compatibility side panel usage is currently using the following telemetry:

DEVTOOLS_CUSTOM_TIME_ACTIVE_SECONDS;
DEVTOOLS_CUSTOM_OPENED_COUNT;

We should introduce specific telemetry for it:
DEVTOOLS_COMPATIBILITYVIEW_OPENED_COUNT
DEVTOOLS_COMPATIBILITYVIEW_TIME_ACTIVE_SECONDS

See this function:
https://searchfox.org/mozilla-central/rev/61fceb7c0729773f544a9656f474e36cd636e5ea/devtools/client/shared/telemetry.js#720-796

Honza

Comment 1

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Martin, do you see any data for: DEVTOOLS_CUSTOM_TIME_ACTIVE_SECONDS & DEVTOOLS_CUSTOM_OPENED_COUNT ? Perhaps this could be useful until we introduce the new telemetry.

Honza

Comment 2

[Martin Balfanz [:mbalfanz]]

Yes, I do see data for this probe. I'll add it to the LayoutTools Dashboard

Comment 3

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Cool

I think that we should still introduce specific probe for the Compat panel since DEVTOOLS_CUSTOM_TIME_ACTIVE_SECONDS & DEVTOOLS_CUSTOM_OPENED_COUNT are used for all custom panels (some panels could be added by extensions). It's a fallback used for all panels that don't have their own probe.

Martin, WDYT?

Honza

Comment 4

[Martin Balfanz [:mbalfanz]]

Yes, we definitely need to introduce a probe specific to the webcompat panel. The data I see is flawed in that there is more history than I should see, so I can't be sure what exactly collected the data.

Comment 5

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Great, thanks Martin.

Daisuke, it would be awesome to land this yet in 78 (if possible) to start collecting the data.
(note that the 78 soft freeze is next week Thursday).

Honza

Comment 6

[Daisuke Akatsuka (:daisuke)]

Thank you for the notice!
I will do soon.

Comment 7

[Daisuke Akatsuka (:daisuke)]

Attachment: Bug 1639454: Introduce OPENED_COUNT and TIME_ACTIVE_SECONDS telemetry for the compatibility panel.

Comment 8

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Attachment: data-review.md

Comment 9

[Tim Smith 👨‍🔬 [:tdsmith]]

Comment on attachment 9150676 [details]
data-review.md

1) Is there or will there be **documentation** that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, in Histograms.json and Events.yaml as aggregated by the probe dictionary.

2) Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, the Firefox telemetry opt-out.

3) If the request is for permanent data collection, is there someone who will monitor the data over time?

Honza will monitor the probes.

4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, interaction data.

5) Is the data collection request for default-on or default-off?

Default-on.

6) Does the instrumentation include the addition of **any *new* identifiers** (whether anonymous or otherwise; e.g., username, random IDs, etc.  See the appendix for more details)?

No.

7) Is the data collection covered by the existing Firefox privacy notice?

Yes.

8) Does there need to be a check-in in the future to determine whether to renew the data?

No, this is a permanent collection.

9) Does the data collection use a third-party collection tool?

No.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2cb92b52fb42
Introduce OPENED_COUNT and TIME_ACTIVE_SECONDS telemetry for the compatibility panel. r=Honza

Comment 11

[Bogdan Tara[:bogdan_tara | bogdant]]

Backed out changeset 2cb92b52fb42 (bug 1639454) for telemetry related bustages

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&fromchange=2cb92b52fb428bfadb5c8d868531bfad95db2abb&searchStr=build&tochange=4a46d2e96770571d041d4a27f93f6b8a12999134

Backout link: https://hg.mozilla.org/integration/autoland/rev/4a46d2e96770571d041d4a27f93f6b8a12999134

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=303854287&repo=autoland&lineNumber=3280

[task 2020-05-26T23:11:02.646Z] 23:11:02     INFO -  make[4]: Entering directory '/builds/worker/workspace/obj-build/toolkit/components/telemetry'
[task 2020-05-26T23:11:02.646Z] 23:11:02     INFO -  toolkit/components/telemetry/TelemetryHistogramData.inc.stub
[task 2020-05-26T23:11:02.646Z] 23:11:02     INFO -  /builds/worker/workspace/obj-build/_virtualenvs/init_py3/bin/python -m mozbuild.action.file_generate /builds/worker/checkouts/gecko/toolkit/components/telemetry/build_scripts/gen_histogram_data.py main TelemetryHistogramData.inc .deps/TelemetryHistogramData.inc.pp .deps/TelemetryHistogramData.inc.stub /builds/worker/checkouts/gecko/toolkit/components/telemetry/Histograms.json /builds/worker/checkouts/gecko/dom/base/UseCounters.conf /builds/worker/checkouts/gecko/dom/base/nsDeprecatedOperationList.h ../../../layout/style/ServoCSSPropList.py /builds/worker/checkouts/gecko/servo/components/style/properties/counted_unknown_properties.py /builds/worker/checkouts/gecko/dom/base/UseCountersWorker.conf
[task 2020-05-26T23:11:02.646Z] 23:11:02     INFO -  Unsupported kind "count" for histogram "DEVTOOLS_COMPATIBILITYVIEW_OPENED_COUNT":
[task 2020-05-26T23:11:02.647Z] 23:11:02     INFO -  New "count" histograms are not supported on Desktop, you should use scalars instead:
[task 2020-05-26T23:11:02.647Z] 23:11:02     INFO -  https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/collection/scalars.html
[task 2020-05-26T23:11:02.647Z] 23:11:02     INFO -  Are you trying to add a histogram on Android? Add "operating_systems": ["android"] to your histogram definition.
[task 2020-05-26T23:11:02.647Z] 23:11:02     INFO -  backend.mk:12: recipe for target '.deps/TelemetryHistogramData.inc.stub' failed
[task 2020-05-26T23:11:02.647Z] 23:11:02    ERROR -  make[4]: *** [.deps/TelemetryHistogramData.inc.stub] Error 1
[task 2020-05-26T23:11:02.647Z] 23:11:02     INFO -  make[4]: Leaving directory '/builds/worker/workspace/obj-build/toolkit/components/telemetry'
[task 2020-05-26T23:11:02.647Z] 23:11:02     INFO -  make[4]: *** Waiting for unfinished jobs....
[task 2020-05-26T23:11:02.647Z] 23:11:02     INFO -  make[4]: Entering directory '/builds/worker/workspace/obj-build/dom/bindings'
[task 2020-05-26T23:11:02.648Z] 23:11:02     INFO -  rm -f Node.webidl
[task 2020-05-26T23:11:02.648Z] 23:11:02     INFO -  make[4]: Leaving directory '/builds/worker/workspace/obj-build/dom/bindings'

Comment 12

[Bogdan Tara[:bogdan_tara | bogdant]]

Also seeing this starting with the backed out changes:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=303854232&repo=autoland&lineNumber=9876

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1109bd345ac2
Introduce OPENED_COUNT and TIME_ACTIVE_SECONDS telemetry for the compatibility panel. r=Honza

Comment 14

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/1109bd345ac2