1455483 - [Shield] Pref Rollout: Pocket Ghost Signup Removal

Status: RESOLVED WONTFIX

(Shield :: Shield Study, enhancement)

Description

[Hugo Romano]

By setting the “extensions.pocket.settings.test.panelSignUp” pref to “control” by default for new Pocket users in Firefox that don’t have the pref already set, we expect to see a drop of DAUs(~4%) in Pocket users on Firefox. This will mainly come from users that try Pocket and immediately churn. Most importantly, we expect to see limited impact overall in weekly Pocket readers. 

For details please see PRD doc: https://docs.google.com/document/d/1EXf-SJfBnNlAFCVWD6Uq2FnwN73uK8guEzipWU5FDNw/edit?ts=5ac66429

Comment 1

[Ilana]

Science review: R+

Comment 2

[Hugo Romano]

Quick update on this: 
- Scope: Confirmed this will not be a roll-out and don't need to wait until 61. Updated the PHD. 
- Release Management: I'm syncing up with Marnie today re: release management. 
- QA: I've followed up with Ciprian Muresan to restart conversations. 
- Intent to Ship: I've sent this email out and am working with Julien on clarification. 

Please let me know if I'm missing anything.

Comment 3

[MarniePW [:marnie]]

One clarification: I'm not release management, I'm program management. You'll want to get approval from the owner of 60, Julien, for release management.

Comment 4

[Ciprian Muresan [:cmuresan], Ecosystem QA]

We have finished testing the "Pocket Ghost Signup Removal" Shield Study experiment.

QA’s recommendation: GREEN - ship it

Reasoning: We haven't found any issues during testing.

Testing Summary:
	- Verified that all variants do what they are supposed to do.
	- Verified that the correct doorhanger is displayed if the user is already logged in a Sync account in the "control" variant.
	- Verified that all log in flows work as expected.
	- Verified that the doorhanger correctly appears regardless of where it is triggered from.
	- During testing, we haven't observed any crashes, lag or stuttering.

Tested Firefox versions:
	- Firefox Beta build v61.0b2;
	- Firefox Nightly build v62.0a1;
	
Tested Platforms:
	- Windows 10 x64;
	- Mac 10.13.3;
	- Arch Linux x64;

Comment 5

[Hugo Romano]

Quick clarification regarding my earlier comment on this experiment. After syncing up with Matt, we agree this should be done as a pref rollout in 61. Again, my earlier comment is incorrect.

Comment 6

[Matt Grimes [:Matt_G]]

Assigning this to mythmon for when you decide to kickoff the rollout.

Comment 7

[Michael Cooper [:mythmon]]

After reviewing the code involved here, and talking with the Pocket team, we've come to the conclusion that this cannot be done with Normandy preference rollout. Pocket only reads this preference from the user branch of preferences, and Normandy only writes to the default branch of preferences. Because of this, the Pocket team has decided not to pursue a different solution.