1397418 - [Shield] Pref Flip Study: Tracking Protection

Status: RESOLVED FIXED

(Shield :: Shield Study, enhancement)

Description

[Luke Crouch [:groovecoder]]

Basic description of experiment: 
Auto-enable Tracking Protection for new users
What is the preference we will be changing?
privacy.trackingProtection.enabled
What are the branches of the study and what values should each branch be set to?
false  (control)
true
What percentage of users do you want in each branch?
50%
What Channels and locales do you intend to ship to?
en-US, de
Release
What is your intended go live date and how long will the study run?
September 6 - 27
At the end of the study, the pref will be flipped back
Users will be shown a heartbeat message that tells them how to stay safe online (content TBD)
Are there specific criteria for participants?
Users with new profile creation dates (on day 0)
Cannot already have privacy.trackingProtection.enabled true
Cannot have any of the following addons:
{d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d} (Adblock-plus)
[e-mail] (uBlock Origin)
[e-mail] (Ghostery)
jid1-MnnxcxisBPnSXQ@jetpack (Privacy Badger)
[e-mail] (Disconnect)
[e-mail] (uMatrix)
{73a6fe31-595d-460b-a920-fcc0f8843232} (noscript [really?])
What is the main effect you are looking for and what data will you use to make these decisions?
More day 0 usage hours vs. control
Better week 3 retention vs. control
Who is the owner of the data analysis for this study?
Peter Dolanjski
Will this experiment require uplift?
No
QA Status of your code:
Already built into Firefox with regular QA process
Do you plan on surveying users at the end of the study?
Yes(?)
Link to any relevant google docs / Drive files that describe the project. Links to prior art if it exists:
Tracking Protection Test Pilot
https://testpilot.firefox.com/experiments/tracking-protection 
https://sql.telemetry.mozilla.org/dashboard/blok-executive-summary 
Privacy Settings Shield Study
https://addons.mozilla.org/en-US/firefox/shield_study_14 
https://sql.telemetry.mozilla.org/dashboard/shield-study-improve-privacy-settings

Comment 1

[Matt Grimes [:Matt_G]]

Already reviewed the PHD. Ilana will be doing analysis on the study. I think you're ready for your intent to ship. Once we get sign off from Relman we can push.

Comment 2

[Luke Crouch [:groovecoder]]

Emailed RelMan

Comment 3

[Matt Grimes [:Matt_G]]

Study is live on Release for new users only in en_US and DE. We will see how many participants we have tomorrow and decide if we need to adjust sample rates.

Comment 4

[Matt Grimes [:Matt_G]]

This study is set to end in a few days. Have we sent the Heartbeat surveys yet?

Comment 5

[Robert (rrayborn, he/him)]

Emailed early survey data.  Will close the recipes and bug in a few days.

Comment 6

[Luke Crouch [:groovecoder]]

I think we can close this bug now, right?