Closed Bug 1599650 Opened 5 years ago Closed 4 years ago

[Experiment] Add-On Experiment: DoH US Engagement Study V4 Fx 70.0 to 72.0 Release

Categories

(Shield :: Shield Study, task, P3)

Product:
Shield
Component:
Shield Study
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: experimenter, Assigned: wthayer)

References

(Blocks 1 open bug,
URL
)

Details

User Story

Experiment Type: Add-on experiment

    What are the branches of the experiment:

- Control doh rollout heuristics 50%:

DoH heuristics are enabled and DoH may be enabled automatically
        
- Treatment doh rollout disabled 50%:

DoH heuristics are not enabled, and DoH will not be enabled automatically
        

    What version and channel do you intend to ship to?

1.5% of Release Firefox 70.0 to 72.0

    Are there specific criteria for participants?


- Enrollment should be sticky over country
- Users that were ever enrolled in the V2 or V3 studies should be excluded: https://experimenter.services.mozilla.com/experiments/doh-us-engagement-study-v2/ and https://experimenter.services.mozilla.com/experiments/doh-us-engagement-study-v3/

The staged rollout will want to avoid this experiment https://experimenter.services.mozilla.com/experiments/doh-us-staged-rollout-to-all-us-desktop-users/edit/

Using browser.search.region instead of geoIP - and making sticky.

Countries: United States of America (US) 

Locales: English (Australian) (en-AU) English (British) (en-GB) English (Canadian) (en-CA) English (US) (en-US) 

    What is your intended go live date and how long will the experiment run?

Nov 27, 2019 - Jan 01, 2020 (35 days)

    What is the main effect you are looking for and what data will you use to
    make these decisions?

What is the main effect you are looking for and what data will
you use to make these decisions? What metrics are you using to measure success

We will be analyzing telemetry for anomalies as we slowly enable DoH, and also watching user sentiment on social media and in bug reports. This experiment will supplement these other feedback channels with engagement data. If engagement hold steady, we will continue to roll out DoH. If it drops, we will stop and possibly roll back the change.

Do you plan on surveying users at the end of the experiment? No.
Strategy and Insights can help create surveys if needed

    Who is the owner of the data analysis for this experiment?

tismith@mozilla.com

    Will this experiment require uplift?

False

    QA Status of your code:

New jira request is PI-356.

The original addon that this pref will enable was tested at https://jira.mozilla.com/browse/PI-244 and https://jira.mozilla.com/browse/PI-326, but the add-on has been updated for use in an add-on style experiment.

    Link to more information about this experiment:

https://experimenter.services.mozilla.com/experiments/doh-us-engagement-study-v4/

DoH US Engagement Study V4

Re-launching due to un-enrollment issue - explained here https://docs.google.com/document/d/1E-bi9KBpfx7NlQ_xPSG-NyPthmFqmFYObEpGxGH5Thk/edit

We plan to conduct a study to monitor engagement and retention in conjunction with a staged rollout of DNS over HTTPS to US users.

Experimenter is the source of truth for details and delivery. Changes to Bugzilla are not reflected in Experimenter and will not change delivery configuration.

More information: https://experimenter.services.mozilla.com/experiments/doh-us-engagement-study-v4/

User Story: (updated)
Summary: [Experiment]: Add-On Experiment: DoH US Engagement Study V4 → [Experiment] Add-On Experiment: DoH US Engagement Study V4 Fx 70.0 to 72.0 Release
Start Date: 2019-11-27 End Date: 2020-01-01
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.