1955169 - Record missing enrollment status telemetry

Status: RESOLVED FIXED

(Firefox :: Nimbus Desktop Client, task, P3)

Description

[Beth Rennie [:beth] (she/her)]

We need to record the NotEnrolled enrollment status when we dont match targeting or bucketing when we unenroll from an experiment or end up not enrolling.

Comment 2

[Beth Rennie [:beth] (she/her)]

Attachment: Bug 1955169 - Simplify recording of enrollment status telemetry r?#nimbus-reviewers!,#omc-reviewers

Previously we were recording enrollment status telemetry everywhere we
were calling into enroll and _unenroll from onRecipe and
updateEnrollment. Now we record the enrollment status telemetry from
inside NimbusTelemetry.recordEnrollment and
NimbusTelemetry.recordUnenrollment, so that the enrollment status
telemetry is recorded from every enrollment event, not just during
RemoteSettingsExperimentLoader.updateRecipes().

To help with this, a new helper has been added, UnenrollmentCause,
which carries all the metadata required to unenroll from an experiment
(similar how CheckRecipeResult carries all the metadata for enrollment
and updating enrollment) and to submit the corresponding telemetry. Now
instead of calling unenroll() with a reason string, you must call it
with an object returned from one of the UenrollmentCause utilities.
All tests that were calling unenroll() with a string like
"test" or "cleanup" have been updated to remove the reason string. When
it is not present, the UnenrollmentCause will default to an "unknown"
reason, which is good enough for test cleanup.

Comment 3

[Beth Rennie [:beth] (she/her)]

Attachment: Bug 1955169 - Record NOT_ENROLLED status telemetry r?#nimbus-reviewers

Comment 4

[Beth Rennie [:beth] (she/her)]

Attachment: Bug 1955169 - Record OPT_IN for Firefox Labs enrollment r?#nimbus-reviewers

Comment 5

[Beth Rennie [:beth] (she/her)]

Attachment: Bug 1955169 - Record OPT_OUT for Firefox Labs and about:studies opt-outs r?#nimbus-reviewers

Comment 6

[Beth Rennie [:beth] (she/her)]

Attachment: Bug 1955169 - Record EnrollmentStatus telemetry for all remaining unenrollment reasons r?#nimbus-reviewers

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/64ae848c4a2b
Simplify recording of enrollment status telemetry r=omc-reviewers,nimbus-reviewers,aminomancer,chumphreys
https://hg.mozilla.org/integration/autoland/rev/ac2b71a2ef17
Record NOT_ENROLLED status telemetry r=nimbus-reviewers,chumphreys
https://hg.mozilla.org/integration/autoland/rev/32059aa54193
Record OPT_IN for Firefox Labs enrollment r=nimbus-reviewers,chumphreys
https://hg.mozilla.org/integration/autoland/rev/c717d831bbc4
Record OPT_OUT for Firefox Labs and about:studies opt-outs r=nimbus-reviewers,chumphreys
https://hg.mozilla.org/integration/autoland/rev/a07069ddb118
Record EnrollmentStatus telemetry for all remaining unenrollment reasons r=nimbus-reviewers,chumphreys

Comment 8

[agoloman]

Backed out for causing xpc failures @test_ExperimentManager_enroll.js and bc failures @browser_about_studies.js.

• Backout link
• Push with failures browser_about_studies.js
• Push with failures xpc failures
• Failure Log

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/669a93c90421
Simplify recording of enrollment status telemetry r=omc-reviewers,nimbus-reviewers,aminomancer,chumphreys,application-update-reviewers,nalexander
https://hg.mozilla.org/integration/autoland/rev/6c18b6e1178a
Record NOT_ENROLLED status telemetry r=nimbus-reviewers,chumphreys
https://hg.mozilla.org/integration/autoland/rev/04666fa6a9a4
Record OPT_IN for Firefox Labs enrollment r=nimbus-reviewers,chumphreys
https://hg.mozilla.org/integration/autoland/rev/6f249fd9c97e
Record OPT_OUT for Firefox Labs and about:studies opt-outs r=nimbus-reviewers,chumphreys
https://hg.mozilla.org/integration/autoland/rev/2260624d1529
Record EnrollmentStatus telemetry for all remaining unenrollment reasons r=nimbus-reviewers,chumphreys

Comment 10

[Cosmin Sabou [:CosminS]]

Backed out 5 changesets (Bug 1955169) for causing xpcshell failures on test_RemoteSettingsExperimentLoader_updateRecipes.

Push with failures

Failure log

Backout link

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7523c66cf741
Simplify recording of enrollment status telemetry r=omc-reviewers,nimbus-reviewers,aminomancer,chumphreys,application-update-reviewers,nalexander
https://hg.mozilla.org/integration/autoland/rev/dec22291ce40
Record NOT_ENROLLED status telemetry r=nimbus-reviewers,chumphreys
https://hg.mozilla.org/integration/autoland/rev/527cbe48536b
Record OPT_IN for Firefox Labs enrollment r=nimbus-reviewers,chumphreys
https://hg.mozilla.org/integration/autoland/rev/aec590ec0dcc
Record OPT_OUT for Firefox Labs and about:studies opt-outs r=nimbus-reviewers,chumphreys
https://hg.mozilla.org/integration/autoland/rev/c4164da0418a
Record EnrollmentStatus telemetry for all remaining unenrollment reasons r=nimbus-reviewers,chumphreys

Comment 12

[Atila Butkovits]

Backed out for causing failures at test_nimbusTelemetry.js.

Backout link: https://hg.mozilla.org/integration/autoland/rev/23a4dd4ede52772cbc0a21733b4099b02fb90752

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=c4164da0418a115c3bfa23ff1d6c94d3c794935d&selectedTaskRun=Dzd-R9yMQ2Cg3YLE868eYA.0

Failure log: https://treeherder.mozilla.org/logviewer?job_id=502091741&repo=autoland&lineNumber=2793

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d114660d9175
Simplify recording of enrollment status telemetry r=omc-reviewers,nimbus-reviewers,aminomancer,chumphreys,application-update-reviewers,nalexander
https://hg.mozilla.org/integration/autoland/rev/123eb3f4b8fe
Record NOT_ENROLLED status telemetry r=nimbus-reviewers,chumphreys
https://hg.mozilla.org/integration/autoland/rev/93f874c3e8b9
Record OPT_IN for Firefox Labs enrollment r=nimbus-reviewers,chumphreys
https://hg.mozilla.org/integration/autoland/rev/fe2c7eda57df
Record OPT_OUT for Firefox Labs and about:studies opt-outs r=nimbus-reviewers,chumphreys
https://hg.mozilla.org/integration/autoland/rev/a612a7074eb8
Record EnrollmentStatus telemetry for all remaining unenrollment reasons r=nimbus-reviewers,chumphreys

Comment 14

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/d114660d9175
https://hg.mozilla.org/mozilla-central/rev/123eb3f4b8fe
https://hg.mozilla.org/mozilla-central/rev/93f874c3e8b9
https://hg.mozilla.org/mozilla-central/rev/fe2c7eda57df
https://hg.mozilla.org/mozilla-central/rev/a612a7074eb8

Comment 15

[Beth Rennie [:beth] (she/her)]

Attachment: Bug 1955169 - Request longer timeout for various nimbus tests r?#nimbus-reviewers

Comment 16

[Phabricator Automation]

Comment on attachment 9518477 [details]
Bug 1955169 - Request longer timeout for various nimbus tests r?#nimbus-reviewers

Revision D267658 was moved to bug 1991153. Setting attachment 9518477 [details] to obsolete.