1786864 - Add telemetry for how long we spend gathering memory telemetry

Status: RESOLVED FIXED

(Core :: XPCOM, task)

Description

[Paul Bone [:pbone]]

It sounds like a joke, to add telemetry about our telemetry, but it's a good idea. We've had a regression in how long it takes to gather memory telemetry and we might catch this sooner if we measured that.

See Bug 1779138 comment 4 and Bug 1779138 comment 11

Comment 1

[Paul Bone [:pbone]]

Attachment: Bug 1786864 - Add telemetry for how long we spend gathering memory telemetry r=kmag

Comment 2

[Paul Bone [:pbone]]

Comment on attachment 9291947 [details]
Bug 1786864 - Add telemetry for how long we spend gathering memory telemetry r=kmag

1. What questions will you answer with this data?

   This telemetry will measure how long it takes to calculate some data for use in telemetry. Changes to the calculation, including changes in the Operating System could change how long this takes.

2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements? Some example responses:

   Establish baselines or measure changes in product or platform quality or performance. Specifically we almost shipped a performance regression in this area. Capturing telemetry here will allow us to notice these problems sooner if they happen in the future.

3. What alternative methods did you consider to answer these questions? Why were they not sufficient?

It's possible that we might just "notice" such a regression in the future, like we did this time. However I think we were lucky this time both noticing it and being able to link it to a recent change. Even if we notice regressions in the future it may take time to perform the regression analysis. Having telemetry helps speed up this process.

4. Can current instrumentation answer these questions?

   No, there's no relevant existing telemetry.

5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.

   MEMORY_COLLECTION_TIME - the time in milliseconds taken to gether memory telemetry. It is Category 1 “Technical data”.

6. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.

   This collection is documented in its definitions files Histograms.json and in the Probe Dictionary at https://probes.telemetry.mozilla.org."

7. How long will this data be collected? Choose one of the following:

   Forever. I wish to perminantly monitor this data. [e-mail].

8. What populations will you measure?

   Whoever is providing telemetry in general. I beleive that's nightly and beta by default and opt-in for release.

9. If this data collection is default on, what is the opt-out mechanism for users?

   Same as the rest of telemetry.

10. Please provide a general description of how you will analyze this data.

    I will view it on the telemetry website. I'd also like to receive alert e-mails for large fluctuations but I think we stopped doing that.

11. Where do you intend to share the results of your analysis?

    If there's something woth reporting I intend to report it in new bugs on bugzilla.mozilla.org.

12. Is there a third-party tool (i.e. not Glean or Telemetry) that you are proposing to use for this data collection? If so:

    No.

Comment 3

[Chris H-C :chutten]

Attachment: data collection review request

In future, please attach data review requests like so.

Comment 4

[Chris H-C :chutten]

Comment on attachment 9292844 [details]
data collection review request

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, :pbone is responsible.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 1, Technical.

Is the data collection request for default-on or default-off?

Default on for pre-release channels only.

Does the instrumentation include the addition of any new identifiers?

No.

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does the data collection use a third-party collection tool?

No.

---

Result: datareview+

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ad0b177e131d
Add telemetry for how long we spend gathering memory telemetry r=kmag

Comment 6

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/ad0b177e131d