Closed Bug 1685741 Opened 5 years ago Closed 5 years ago

[Experiment] Pref-Flip Experiment: unenrollment normandy test with no-op pref in Firefox - string Fx 86.0 to 100.0 Nightly

Categories

(Shield :: Shield Study, task, P3)

Product:
Shield
Component:
Shield Study
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: experimenter, Assigned: shell)

References

(
URL
)

Details

User Story

Delivery Type: Pref Flip Experiment

    What is the preference we will be changing

app.normandy.test-prefs.string

    What are the branches of the experiment and what values should
    each branch be set to?

- Treatment treatment string 50%:

Value: treatment

treatment is treatment
        
- Control control string 50%:

Value: empty

default empty string
        

    What version and channel do you intend to ship to?

30% of Nightly Firefox 86.0 to 100.0

    Are there specific criteria for participants?

Prefs:

Experiments:

Any additional filters:  add minimum nightly build that has the prefs
Countries: all

Locales: all

    What is your intended go live date and how long will the experiment run?

Jan 12, 2021 - Feb 09, 2021 (28 days)

    What is the main effect you are looking for and what data will you use to
    make these decisions?

look for uneven un-enrollment in a case there should be none.

We'll check the new meta data on standard normandy telemetry that says which pref caused the change landed in 85.

    Who is the owner of the data analysis for this experiment?

tismith@mozilla.com

    Will this experiment require uplift?

False

    QA Status of your code:

Discussed in Lightning advising with QA and DS (Krupa and Tim).  This is just no-op prefs on Nightly + Beta for system test.  Engineer QA is sufficient for this minimal risk.

    Link to more information about this experiment:

https://experimenter.services.mozilla.com/experiments/unenrollment-normandy-test-with-no-op-pref-in-firefox-string/

unenrollment normandy test with no-op pref in Firefox - string

Changing a pref that does nothing - to check enrollment and unenrollment reliability.

Experimenter is the source of truth for details and delivery. Changes to Bugzilla are not reflected in Experimenter and will not change delivery configuration.

Data Science Issue: https://jira.mozilla.com/browse/DO-397
More information: https://experimenter.services.mozilla.com/experiments/unenrollment-normandy-test-with-no-op-pref-in-firefox-string/

User Story: (updated)
Summary: [Experiment]: Pref-Flip Experiment: unenrollment normandy test with no-op pref in Firefox - string → [Experiment] Pref-Flip Experiment: unenrollment normandy test with no-op pref in Firefox - string Fx 86.0 to 100.0 Nightly
Start Date: 2021-01-12 End Date: 2021-02-09
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.