Closed Bug 1590831 Opened 5 years ago Closed 5 years ago

[Experiment] Pref-Flip Experiment: DoH US Engagement Study V2 Fx 69.0 to 71.0 Release

Categories

(Shield :: Shield Study, task, P3)

Product:
Shield
Component:
Shield Study
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: experimenter, Assigned: wthayer)

References

(Blocks 1 open bug,
URL
)

Details

User Story

Experiment Type: Pref Flip Experiment

    What is the preference we will be changing

doh-rollout.enabled

    What are the branches of the experiment and what values should
    each branch be set to?

- Treatment disabled 50%:

Value: false

disabled
        
- Treatment enabled 50%:

Value: true

enabled
        

    What version and channel do you intend to ship to?

1% of Release Firefox 69.0 to 71.0

    Are there specific criteria for participants?

- 69.0.3 or higher (including 70.*)
- Enrollment should be sticky over country
- System addon doh-rollout@mozilla.org is installed

The staged rollout will want to avoid this experiment https://experimenter.services.mozilla.com/experiments/doh-us-staged-rollout-to-all-us-desktop-users/edit/
Countries: United States of America (US) 

Locales: English (British) (en-GB) English (Canadian) (en-CA) English (US) (en-US) 

    What is your intended go live date and how long will the experiment run?

Oct 29, 2019 - Dec 03, 2019 (35 days)

    What is the main effect you are looking for and what data will you use to
    make these decisions?

What is the main effect you are looking for and what data will
you use to make these decisions? What metrics are you using to measure success

We will be analyzing telemetry for anomalies as we slowly enable DoH, and also watching user sentiment on social media and in bug reports. This experiment will supplement these other feedback channels with engagement data. If engagement hold steady, we will continue to roll out DoH. If it drops, we will stop and possibly roll back the change.

Do you plan on surveying users at the end of the experiment? No.
Strategy and Insights can help create surveys if needed

    Who is the owner of the data analysis for this experiment?

tismith@mozilla.com

    Will this experiment require uplift?

False

    QA Status of your code:

New jira request is PI-326.

The original addon that this pref will enable was tested at https://jira.mozilla.com/browse/PI-244 - but the add-on was updated to respect the default branch instead of user branch prefs.

Signed Off as GREEN, for more info see https://experimenter.services.mozilla.com/experiments/doh-us-engagement-study-v2/#comment502

    Link to more information about this experiment:

https://experimenter.services.mozilla.com/experiments/doh-us-engagement-study-v2/

DoH US Engagement Study V2

We plan to conduct a study to monitor engagement and retention in conjunction with a staged rollout of DNS over HTTPS to US users.

Experimenter is the source of truth for details and delivery. Changes to Bugzilla are not reflected in Experimenter and will not change delivery configuration.

More information: https://experimenter.services.mozilla.com/experiments/doh-us-engagement-study-v2/

User Story: (updated)
Summary: [Experiment]: Pref-Flip Experiment: DoH US Engagement Study V2 → [Experiment] Pref-Flip Experiment: DoH US Engagement Study V2 Fx 69.0 to 71.0 Release
Start Date: 2019-10-30 End Date: 2019-12-04
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.