1762652 - Enhanced telemetry around unenrollment and schema validation failures

Status: RESOLVED FIXED

(Firefox :: Nimbus Desktop Client, enhancement, P1)

Description

[Beth Rennie [:beth] (she/her)]

Right now we get unenroll telemetry for schema validation failures, but we only know that it happened, not why (i.e., we don't know what recipe failed). Additionally, we don't collect telemetry if the recipe failed validation but was not previously enrolled.

Comment 1

[Beth Rennie [:beth] (she/her)]

Attachment: Bug 1762652 - Add detailed telemetry around schema validation in Nimbus r?TravisLong

Depends on D150693

Comment 2

[Beth Rennie [:beth] (she/her)]

Attachment: data-review-1762652.md

Comment 3

[Chris H-C :chutten]

Comment on attachment 9284424 [details]
data-review-1762652.md

PRELIMINARY NOTES:
Since this is a mirror from Glean to Telemetry and they behave identically wrt opt-out in Firefox Desktop, you can in future submit a data collection review just for the Glean portion. And to make that easier, in Firefox Desktop we have mach data-review <bug number> which can grab the information you already entered for the Glean metric definition and use it to generate a data review request form template.

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, Barret Rennie is responsible.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 1, Technical.

Is the data collection request for default-on or default-off?

Default on for all channels.

Does the instrumentation include the addition of any new identifiers?

No.

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does the data collection use a third-party collection tool?

No.

---

Result: datareview+

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/033d8825f710
Add detailed telemetry around schema validation in Nimbus r=TravisLong

Comment 5

[Iulian Moraru]

Backed out for causing multiple failures.

Push with failures

Failure log for xpcshell jobs that fail on test_ExperimentManager_lifecycle.js
Failure log for bc jobs that fail on browser_remotesettingsexperimentloader_remote_defaults.js
Failure log for TV jobs that fail on test_RemoteSettingsExperimentLoader_updateRecipes.js

Backout link

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5e5c34c193b6
Add detailed telemetry around schema validation in Nimbus r=TravisLong

Comment 7

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/5e5c34c193b6