[remote-dbg-next] Telemetry: record runtime updates
Categories
(DevTools :: about:debugging, enhancement, P1)
Tracking
(firefox67 fixed)
Tracking | Status | |
---|---|---|
firefox67 | --- | fixed |
People
(Reporter: jdescottes, Assigned: jdescottes)
References
(Blocks 2 open bugs)
Details
Attachments
(2 files, 4 obsolete files)
47 bytes,
text/x-phabricator-request
|
Details | Review | |
2.80 KB,
text/plain
|
chutten
:
data-review+
|
Details |
We should record events when the device/runtime list is updated. This should include:
- when a device is detected,
- when a new runtime is detected
- when a runtime disappears
- when a device disappears
- when a connection to a runtime is established
- when a connection to a runtime is lost
The events should give us an idea of how frequently the list of devices is updated, and should also give insights about the type of devices available to the user. I don't think we can record complex properties (arrays of objects? even if easily serializable?).
One option would be to serialize such an object, but it would probably be hard to query and reuse from amplitude. So alternatively (or additionally) we could have as properties:
- the total number of devices
- number of devices by type
- number of devices by runtime type (GeckoView? Fennec?)
- number of devices by connection status (Connected? Not Connected)
Or we can record one event per device every time the list is updated. E.g. something like device_available, with some basic properties about the device (connection type, runtime, connected...)
Assignee | ||
Updated•5 years ago
|
Assignee | ||
Updated•5 years ago
|
Assignee | ||
Comment 1•5 years ago
|
||
Harald, before I ask for data review, can you check if you would like more data to be logged alongside those events. For now I just log:
- connection_type: how the device was connected ("usb", "network" or "wifi")
- device_name: the manufacturer's device name in case of phones connected via USB (eg "Moto G(4)")
- runtime_name: the name of the detected runtime (eg "Firefox", "Firefox Nightly")
Thanks!
Assignee | ||
Comment 2•5 years ago
|
||
Will ask for r? from janerik once product has approved the data collected here.
Assignee | ||
Comment 3•5 years ago
|
||
Updated information about the extras logged for device_
events.
Assignee | ||
Comment 4•5 years ago
|
||
Added the runtime id to the list of information collected for runtime events.
Comment 5•5 years ago
|
||
Comment on attachment 9043658 [details] data_review_1521507_v3.txt My take on making this a bit more concrete: > 1. What questions will you answer with this data? Which devices are users trying to debug and how? Which devices are users struggling to connect to? Is the numbers of developers increasing, that debug GeckoView-based browsers? > 2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements? We can focus on better supporting the devices users actually remote-debug to provide compatibility.
Assignee | ||
Comment 6•5 years ago
|
||
Applied suggestions from Harald, requesting data-review.
Comment 7•5 years ago
|
||
Comment on attachment 9043901 [details]
data_review_1521507_v4.txt
Network host names are not be eligible for default-on collection in release as they can contain arbitrary strings of unknown Category. I recommend removing that collection and resubmitting for review, or designing and having approved a mitigation for the possibility of those names containing personal data.
Please don't hesitate to reach out to me if you have any questions about this.
Assignee | ||
Comment 8•5 years ago
|
||
Thanks for the review! I kept a runtime_id data collection to allow us to link together events occurring on a given runtime (and the runtime name is not 1/ specific enough, 2/ always available), but this new "runtime_id" is just a random unique id that should not leak any user information (unique for a given about:debugging session).
I will update the patch so that you and jan-erik can check the implementation as well. Let me know if this approach is ok from a data collection perspective.
Comment 9•5 years ago
|
||
Comment on attachment 9044628 [details]
data_review_1521507_v5.txt
A random id would be perfectly fine from a data collection POV, though I'd love to take a look at the implementation as I give review.
Assignee | ||
Comment 10•5 years ago
|
||
(In reply to Chris H-C :chutten from comment #9)
Comment on attachment 9044628 [details]
data_review_1521507_v5.txtA random id would be perfectly fine from a data collection POV, though I'd
love to take a look at the implementation as I give review.
Sure! flagged you for r? on Phabricator.
Comment 11•5 years ago
|
||
Comment on attachment 9044628 [details] data_review_1521507_v5.txt DATA COLLECTION REVIEW RESPONSE: Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate? Yes. This collection is Telemetry so is documented in its definitions file ([Events.yaml](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/telemetry/Events.yaml)) and the [Probe Dictionary](https://telemetry.mozilla.org/probe-dictionary/). Is there a control mechanism that allows the user to turn the data collection on and off? Yes. This collection is Telemetry so can be controlled through Firefox's Preferences. If the request is for permanent data collection, is there someone who will monitor the data over time? Yes. Harald Kirschner is responsible (I gather from Comment #5 that he is okay with being named). Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under? Category 2, Interaction. Is the data collection request for default-on or default-off? Default on for all channels. Does the instrumentation include the addition of any new identifiers? Yes. It introduces a temporary runtime identifier which is only stable within the Firefox session. It also records the manufacturer-specified device name if the attachment is to a mobile device. Neither of these are of high category or are used in other datasets of high category. Is the data collection covered by the existing Firefox privacy notice? Yes. Does there need to be a check-in in the future to determine whether to renew the data? No. This collection is permanent. --- Result: datareview+
Comment 12•5 years ago
|
||
Pushed by jdescottes@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/531c11330f0c Add telemetry events for runtime update;r=daisuke,janerik,chutten
Comment 13•5 years ago
|
||
bugherder |
Description
•