1415967 - [Shield] Pref Flip Study: Activity Stream Enabled for New Users in 57 Release

Status: RESOLVED FIXED

(Shield :: Shield Study, enhancement)

Description

[Jared Lockhart [:jlockhart]]

Basic description of experiment: 

Enable Activity Stream for a subset of the 57 release population of new users

What is the preference we will be changing? 

browser.newtabpage.activity-stream.enabled

What are the branches of the study and what values should each branch be set to? 

Control: false
Variant: true

What percentage of users do you want in each branch? 

2% Overall
1 Control: 1 Variant

What Channels and locales do you intend to ship to? 

57 Release all locales

What is your intended go live date and how long will the study run? 

Start Date: November 14th 2017
End Date: January 23rd 2018

Are there specific criteria for participants? 

New users (profile creation date >= Nov 14 2017)

What is the main effect you are looking for and what data will you use to make these decisions? 

The effect we are looking for is how users engage with Activity Stream and how that affects overall browser usage and retention.

We will measure Firefox Engagement and Retention using the Telemetry Experiments dataset to measure any regressions.

We will measure Activity Stream engagement using Ping Centre to measure the rates at which users interact with each feature.

Who is the owner of the data analysis for this study? 

Marina Samuel (emtwo)

Will this experiment require uplift? 

No

QA Status of your code: 

Green.  This code has been rigorously tested in Nightly and Beta.

Do you plan on surveying users at the end of the study? 

Yes.

Link to any relevant google docs / Drive files that describe the project. Links to prior art if it exists:

https://github.com/mozilla/activity-stream

Previous studies:
https://docs.google.com/a/mozilla.com/spreadsheets/d/1aExRidtN4fWs-sQr9YPXmH7mZ5BowLbCT88rw-jW5Po/edit?usp=sharing

Comment 1

[Jared Lockhart [:jlockhart]]

Recipe slug:

pref-flip-activity-stream-57-release-enabled-new-users-bug-1415967

Comment 2

[Matt Grimes [:Matt_G]]

This study is now live. It should not return true for anyone until the Release 57 updates go live.

Comment 3

[Matt Grimes [:Matt_G]]

@jkerim same here. Can we get this closed and a recap of the study end state?

Comment 4

[:shell escalante]

closing all experiment bugs - for experiments that have ended.