1636419 - There's no GC telemetry for mobile platforms

Status: RESOLVED FIXED

(Core :: JavaScript: GC, task, P1)

Description

[Jon Coppeard (:jonco)]

After talking to the telemetry team, it turns out we don't have any GC telemetry on mobile.

histograms.json specifies "fennec" and "geckoview" products for our telemetry histograms, but:

• the "fennec" product doesn't work for us (not sure why)
• the "geckoview" product has been deprecated and only applies to Focus now, as Fenix has migrated to new telemetry system based on Glean
• Glean requires the "geckoview_streaming" product instead plus new entry in metrics.yaml

Comment 1

[Jon Coppeard (:jonco)]

How to add existing gecko telemetry to glean is described here:

https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/start/report-gecko-telemetry-in-glean.html

Comment 2

[Jon Coppeard (:jonco)]

Attachment: Bug 1636419 - Add GC_MS telemetry to geckoview r?sfink

As a first step, add geckoview_streaming for GC_MS telemetry.

Comment 3

[Alessio Placitelli [:Dexter]]

(In reply to Jon Coppeard (:jonco) from comment #0)

histograms.json specifies "fennec" and "geckoview" products for our telemetry histograms, but:

• the "fennec" product doesn't work for us (not sure why)

Pardon my fly-by :-) It doesn't work for you because Fennec is a totally different product/telemetry system than Fenix, so what the histogram.json definition is saying is: "Please collect this data for Fennec". When running in Fenix, unless otherwise stated, data is dropped to the floor.

Comment 4

[Jon Coppeard (:jonco)]

Comment on attachment 9147589 [details]
Bug 1636419 - Add GC_MS telemetry to geckoview r?sfink

Requesting data review.

Comment 5

[Jon Coppeard (:jonco)]

Attachment: data_collection_request.txt

Requesting data review.

Comment 6

[Alessio Placitelli [:Dexter]]

Comment on attachment 9147592 [details]
data_collection_request.txt

I'm afraid I'm not a data-reviewer! The list of [data reviewers is here](https://wiki.mozilla.org/Firefox/Data_Collection). I redirected this to :chutten :-)

Comment 7

[Olli Pettay [:smaug][[e-mail]]]

(we are missing also all the CC metrics on Fenix)

Comment 8

[Chris H-C :chutten]

Comment on attachment 9147592 [details]
data_collection_request.txt

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. This collection is Telemetry so is documented in its definitions file [Histograms.json](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/telemetry/Histograms.json) and the [Probe Dictionary](https://telemetry.mozilla.org/probe-dictionary/). The GeckoView portion is documented in [metrics.yaml](https://searchfox.org/mozilla-central/source/toolkit/components/telemetry/geckoview/streaming/metrics.yaml) and engine-gecko's [metrics documentation](https://github.com/mozilla-mobile/android-components/blob/master/components/browser/engine-gecko-nightly/docs/metrics.md).

    Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

    If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, :jonco is responsible.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 1, Technical.

    Is the data collection request for default-on or default-off?

Default on for all channels.

    Does the instrumentation include the addition of any new identifiers?

No.

    Is the data collection covered by the existing Firefox privacy notice?

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data?

No. This collection is permanent.

---
Result: datareview+

Comment 9

[Jon Coppeard (:jonco)]

(In reply to Chris H-C :chutten from comment #8)
Thanks for the quick review!

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/92debb8a68a6
Add GC_MS telemetry to geckoview r=sfink,Dexter

Comment 11

[Cristina Coroiu [:ccoroiu]]

Backed out changeset 92debb8a68a6 (bug 1636419) for build bustages at deps/TelemetryHistogramData.inc.stu

Backout: https://hg.mozilla.org/integration/autoland/rev/e4dc54b392c86a456cd4bb3a665c483d165da9eb

Failure push: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=92debb8a68a69e514642f4d08249c16c3272534e

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=301942925&repo=autoland&lineNumber=3237

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8686fd3da6e7
Add GC_MS telemetry to geckoview r=sfink,Dexter

Comment 13

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/8686fd3da6e7

Comment 14

[Jon Coppeard (:jonco)]

Looks like this worked! Here's the results for Fenix so far:

https://sql.telemetry.mozilla.org/queries/71106#178835

This looks pretty similar to our results on desktop which is good sign:

https://telemetry.mozilla.org/new-pipeline/dist.html#!cumulative=0&end_date=2020-05-10&include_spill=0&keys=__none__!__none__!__none__&max_channel_version=nightly%252F78&measure=GC_MS&min_channel_version=nightly%252F75&processType=*&product=Firefox&sanitize=1&sort_by_value=0&sort_keys=submissions&start_date=2020-05-04&table=0&trim=1&use_submission_date=0

Comment 15

[Jon Coppeard (:jonco)]

Attachment: Bug 1636419 - Enable main GC timing telemetry on geckoview r?sfink

This adds data for the time taking by the different phases plus slice time.

Comment 16

[Jon Coppeard (:jonco)]

Attachment: data_collection_request_2.txt

Requesting data review.

Comment 17

[Chris H-C :chutten]

Comment on attachment 9150796 [details]
data_collection_request_2.txt

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. This collection is Telemetry and glean so is documented in its definitions file [Histograms.json](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/telemetry/Histograms.json) and the [Probe Dictionary](https://telemetry.mozilla.org/probe-dictionary/) as well as in [metrics.yaml](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/telemetry/geckoview/streaming/metrics.yaml) and the [engine-gecko metrics documentation](https://github.com/mozilla-mobile/android-components/tree/master/components/browser/engine-gecko-nightly/docs).

    Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences in Firefox and is Glean so can be controlled through Fenix's Preferences in Fenix.

    If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, :jonco is responsible.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 1, Technical.

    Is the data collection request for default-on or default-off?

Default on for all channels.

    Does the instrumentation include the addition of any new identifiers?

No.

    Is the data collection covered by the existing Firefox privacy notice?

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data?

No. This collection is permanent.

---
Result: datareview+

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8fbcd4668bae
Enable main GC timing telemetry on geckoview r=sfink

Comment 19

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/8fbcd4668bae

Comment 20

[Jon Coppeard (:jonco)]

Attachment: Bug 1636419 - Fix misconfigured geckoview telemetry r?sfink

Sadly I messed up the minor GC time telemetry by getting the unit wrong.

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/bd586dcc7733
Fix misconfigured geckoview telemetry r=sfink

Comment 22

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/bd586dcc7733

Comment 23

[Jon Coppeard (:jonco)]

Attachment: Bug 1636419 - Add GC_MARK_MS telemetry on mobile too r?sfink

I missed this from the previous patch. It's covered by the previous data-review which says it's included.

Comment 24

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/bd587844f882
Add GC_MARK_MS telemetry on mobile too r=sfink

Comment 25

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/bd587844f882