1443649 - [Shield] Pref Flip Study: Activity Stream wide layout (60 Beta)

Status: RESOLVED FIXED

(Shield :: Shield Study, enhancement)

Description

[Maria Popova]

Basic description of experiment: Responsive Layout/ Wide Screen (Show more content when wide screens are viewing New Tab).  Full design can be found here.  Below is a comparison of new and old design.  No other functional changes are introduced.

What are the branches of the study? 50% of Beta with 50/50 split on control vs. experiment

What percentage of users do you want in each branch?  50% of Beta with 50/50 split on control vs. experiment

What Channels and locales do you intend to ship to? Beta, all locales

What is your intended go live date and how long will the study run? Firefox 60, the experiment should run for 2 weeks from Monday, 3/12 through Monday, 3/26 .

Are there specific criteria for participants? No

What is the main effect you are looking for and what data will you use to make these decisions? User Engagement, user retention

Who is the owner of the data analysis for this study? Tim Spurway and Nan Jiang 

Who will have access to the data? Activity Stream and Shields Team

Do you plan on surveying users at the end of the study? Yes, we would like to create a Heartbeat survey with help from Strategy and Insights 

User facing title of the experiment: Activity Stream Wide Screens

User facing description of the experiment: Responsive Layout/ Wide Screen for Activity Stream provides better use of the screen space and reduces the white space thus making better use of the available screen real estate. 

Code Review performed by: Name of Firefox peer reviewer(s)?

QA Status of your add-on: Green  

Link to any relevant google docs / Drive files that describe the project. Links to prior art if it exists:
https://trello.com/c/IMfmkovo/101-i-would-like-to-see-a-4-or-5-column-layout-when-viewing-the-new-tab-on-a-wide-screen

Comment 1

[Maria Popova]

Here is the link to PHD:
https://docs.google.com/document/d/13eSw8yUNRsSardFa8SEIRYEkzhskN2f2kr6ZzgnESE4/edit

Comment 2

[Ilana]

Hi Maria, PHDs now need to be submitted to [e-mail] and put through a PHD review cycle in order to be rolled out via shield. Please do this step and we'll review it and follow up with the Wednesday PHD review session where we can schedule it and make sure it's ready for rollout.

Comment 3

[Ciprian Muresan [:cmuresan], Ecosystem QA]

I have verified that with the "browser.newtabpage.activity-stream.enableWideLayout" pref set to false, the new tab page reverts to the initial implementation. There have been no start-up crashes, hangs or lag of any kind.

The pref was also verified in Nightly in bug 1385944.

Testing was performed on the latest Firefox Beta (60.0b4) on Windows 10 x64, Mac 10.13.3 and Ubuntu 14.04 x64.

Considering this, from QA's point of view, we are GREEN to release the experiment.

Comment 4

[Ed Lee :Mardak]

I reviewed the code for the wider layout and the preference in https://github.com/mozilla/activity-stream/pull/3969

Comment 5

[Ilana]

Science review: R+

Comment 6

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Commenting on Julien's behalf (he owns 60), this exp is good to go. Julien asked for QA sign off and peer review (in email) which are both addressed in comment 3 & 4. 

Julien, FYI

Comment 7

[Robert (rrayborn, he/him)]

We're live!

Comment 8

[klong]

Where can I find the shield_id for this study? Want to keep an eye on Pocket Recs CTR.

Comment 9

[Nan Jiang [:nanj]]

(In reply to kenny from comment #8)
> Where can I find the shield_id for this study? Want to keep an eye on Pocket
> Recs CTR.

You can find the shield_id for this study in the Firefox Experiments Viewer (Mozilla ldap auth required)

https://moz-experiments-viewer.herokuapp.com/?pop=six&showOutliers=false

Comment 10

[Jared Lockhart [:jlockhart]]

Hi!  We missed a step in deploying this study, so unfortunately we need to close the existing recipe, clone it with the following changes, and relaunch it:


Pref Branch: User        //// VERY IMPORTANT

Recipe Slug:

pref-flip-activity-stream-60-beta-wide-layout-bug-1443649



All other information and configuration should be identical, let me know if you need any other steps or information.

Comment 11

[Robert (rrayborn, he/him)]

This was fixed (2 hours ago).  Thanks for the corrections.

Comment 12

[Maria Popova]

The experiment will close on Friday, 4/6 allowing the team to collect more data in survey:
https://data.surveygizmo.com/r/28049_5ac3a667706900.87436738