1529232 - Implement pre-account ping

Status: RESOLVED FIXED

(Toolkit :: Telemetry, enhancement, P1)

Description

[Jan-Erik Rediger [:janerik]]

With the specification mostly finished, we can start working on the implementation of the pre-account (name subject to change) ping as a custom ping

Comment 1

[Jan-Erik Rediger [:janerik]]

Attachment: Bug 1529232 - Implement pre-account ping

It's missing some details and is disabled by default (gated by the
toolkit.telemetry.ecosystemping.enabled preference)

Comment 2

[Jan-Erik Rediger [:janerik]]

Attachment: Bug 1529232 - Enable Ecosystem Telemetry on Nightly

Depends on D21521

Comment 3

[Jan-Erik Rediger [:janerik]]

Attachment: Bug 1529232 - Add browser.engagement.total_uri_count scalar to ecosystem telemetry

Depends on D23293

Comment 4

[Jan-Erik Rediger [:janerik]]

Attachment: data-review-request-preaccount.txt

I'd like to get data-review for a new ping, the Ecosystem pre-account ping.
This will eventually be sent from all products to analyze cross-product behavior.

This is mostly about the environment/general data.
For testing purposes I added a single existing probe. Further probes will be added later (bug 1529233).
The ecosystem client ID is the static string "unknown" for now (when bug 1530655 is implemented, it will also get data-review)

Comment 5

[Jan-Erik Rediger [:janerik]]

Comment on attachment 9050628 [details]
data-review-request-preaccount.txt

Missing a small but important detail. Will reupload.

Comment 6

[Jan-Erik Rediger [:janerik]]

Attachment: data-review-request-preaccount.txt

Comment 7

[Jan-Erik Rediger [:janerik]]

I'm gonna remove the unclear data point (uid) for now (plus some other changes coming), but the general part stays the same.

Comment 8

[Chenxia Liu [:liuche]]

:chutten do you want to pick this up again since you're more familiar with the Ecosystem ping? I've been following up on some Fenix/Mobile things that are starting to pile up, but can take it if you want.

Comment 9

[Chris H-C :chutten]

Sure, I'll reassign the review to myself.

Comment 10

[Chris H-C :chutten]

Comment on attachment 9050934 [details]
data-review-request-preaccount.txt

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. This collection is documented in the [in-tree documentation for Ecosystem Telemetry](https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/collection/events.html).

The browser.engagement.total_uri_count probe is also documented in its definitions file [Scalars.yaml](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/telemetry/Scalars.yaml) and the [Probe Dictionary](https://telemetry.mozilla.org/probe-dictionary/).

    Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

    If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, Leif Oines is responsible for monitoring the probe. The General/Environment data is ongoing collection for which the Firefox Telemetry team is responsible. Future probes will be monitored by their owners.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction (for total_uri_count. All the rest is Cat 1)

    Is the data collection request for default-on or default-off?

Default on for only Nightly.

    Does the instrumentation include the addition of any new identifiers?

No.

    Is the data collection covered by the existing Firefox privacy notice?

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data?

No. This collection is permanent.

---
Result: datareview+

(( Note that future probes and future additions to the ping will be subject to future Data Reviews )).

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ed20a4120896
Implement pre-account ping r=chutten,gfritzsche,markh
https://hg.mozilla.org/integration/autoland/rev/8ca8be52ca57
Enable Ecosystem Telemetry on Nightly r=gfritzsche
https://hg.mozilla.org/integration/autoland/rev/f2935b75f9dc
Add browser.engagement.total_uri_count scalar to ecosystem telemetry r=gfritzsche

Comment 12

[Arthur Iakab [arthur_iakab]]

Backed out 3 changesets (bug 1529232) for xpcshell failures on test_TelemetryController.js CLOSED TREE

Backout revision https://hg.mozilla.org/integration/autoland/rev/677a1d7108cd9c43c69f527b34cba45a6074b704

Failed push https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=f2935b75f9dcf8b5816192d7d2b6a01f43f2d029&searchStr=android%2C4.3%2Capi16%2B%2Cdebug%2Cxpcshell%2Ctests%2Ctest-android-em-4.3-arm7-api-16%2Fdebug-xpcshell-12%2Cx%28x12%29

Failure log https://treeherder.mozilla.org/logviewer.html#?job_id=235535597&repo=autoland
https://treeherder.mozilla.org/logviewer.html#?job_id=235544457&repo=autoland
https://treeherder.mozilla.org/logviewer.html#?job_id=235540156&repo=autoland

:janerik could you please take a look?

Comment 13

[Jan-Erik Rediger [:janerik]]

Found the culprit. A small readability optimization causes problems on Android.
New try run on multiple platforms: https://treeherder.mozilla.org/#/jobs?repo=try&revision=cf3ff1a04b6951987fd96afaf4da1cd7fe3a3e91

Keeping the ni? so that I remember to check the try run and re-land this later.

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9c9c6c36db7b
Implement pre-account ping r=chutten,gfritzsche,markh
https://hg.mozilla.org/integration/autoland/rev/3c600bf98e37
Enable Ecosystem Telemetry on Nightly r=gfritzsche
https://hg.mozilla.org/integration/autoland/rev/1a0669e032ff
Add browser.engagement.total_uri_count scalar to ecosystem telemetry r=gfritzsche

Comment 15

[Andreea Pavel [:apavel]]

Backed out for failing xpacshell at test_EcosystemTelemetry.js

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=success%2Ctestfailed%2Cbusted%2Cexception&selectedJob=235784319&searchStr=linux%2Copt%2Cxpcshell%2Ctests%2Ctest-linux32%2Fopt-xpcshell-8%2Cx%28x8%29&revision=1a0669e032ffb71baac240ecad8453195ac9634d

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=235784314&repo=autoland&lineNumber=2290

Backout: https://hg.mozilla.org/integration/autoland/rev/32062b26934c1767701f6c0420999cb9025ab2f2

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c2fbc9f732e0
Implement pre-account ping r=chutten,gfritzsche,markh
https://hg.mozilla.org/integration/autoland/rev/0e09b0ab01d9
Enable Ecosystem Telemetry on Nightly r=gfritzsche
https://hg.mozilla.org/integration/autoland/rev/2885c6add178
Add browser.engagement.total_uri_count scalar to ecosystem telemetry r=gfritzsche

Comment 17

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/c2fbc9f732e0
https://hg.mozilla.org/mozilla-central/rev/0e09b0ab01d9
https://hg.mozilla.org/mozilla-central/rev/2885c6add178