[Experiment] Pref-Flip Experiment: Activity-Stream Chrome Switchers card for Onboarding Triplets in Firefox 73 V3 Fx 73.0 to 75.0 Release
Categories
(Shield :: Shield Study, task, P3)
Tracking
(Not tracked)
People
(Reporter: experimenter, Assigned: mythmon)
References
()
Details
User Story
Delivery Type: Pref Flip Experiment
What is the preference we will be changing
trailhead.firstrun.branches
What are the branches of the experiment and what values should
each branch be set to?
- Control join-dynamic 50%:
Value: join-dynamic
cards are dynamic cards, minus chrome card
- Treatment join-dynamic_chrome 50%:
Value: join-dynamic_chrome
cards are dynamic cards, including chrome card
What version and channel do you intend to ship to?
25% of Release Firefox 73.0 to 75.0
Are there specific criteria for participants?
Prefs:
Experiments:
Any additional filters:
Targeting should be for new profiles only (profile creation date after start of experiment date
Sticky: EXCLUDES users who have trailhead.firstrun.didSeeAboutWelcome is true. This captures users that have not seen about:welcome page. new users on first startup may have this not set at normandy execution, or have it set to false, both of which will make them eligible for the experiment. non-new users will have this set to true. when they start the browser and normandy executes, the value for this pref should exist for normandy to filter them out of the experiment.
Countries: all
Locales: all
What is your intended go live date and how long will the experiment run?
Mar 17, 2020 - Apr 28, 2020 (42 days)
What is the main effect you are looking for and what data will you use to
make these decisions?
What is the main effect you are looking for and what data will
you use to make these decisions? What metrics are you using to measure success
* Overall usage metrics (retention, days used, URIs, activehours, searches)
* % of users who clicked chrome tile (variant only)
* % of users who started import
* % of users who completed import
Supplementary metrics to check:
* jank during import process
Do you plan on surveying users at the end of the delivery? Yes/No.
Strategy and Insights can help create surveys if needed
N/A
Who is the owner of the data analysis for this experiment?
shong@mozilla.com
Will this experiment require uplift?
False
QA Status of your code:
THIS IS A RELAUNCH. Only change from previous is a recipe filter - and that this runs into Firefox 65. Asking engineering if anything has changed in this code area- or if it should be OK to run in 65. They will give us the update on if it needs more testing.
Sent a GREEN sign-off as per comment: https://experimenter.services.mozilla.com/experiments/chrome-switchers-card-for-onboarding-triplets-in-firefox-73/#comment652
Link to more information about this experiment:
https://experimenter.services.mozilla.com/experiments/activity-stream-chrome-switchers-card-for-onboarding-triplets-in-firefox-73-v3/
Activity-Stream Chrome Switchers card for Onboarding Triplets in Firefox 73 V3
RELAUNCH of the same experiment. The chrome filters is too restrictive - so removing that (and can look at in analysis). This experiment population is still quite small (well under 1 M - only new users) - so no additional risk.
We would like to run a study targeting Windows only Firefox 73 users that downloaded Firefox using Chrome.
We would show part of these users a Chrome specific language card (as of of the 3 onboarding cards) inviting them to import their data from Chrome.
Experimenter is the source of truth for details and delivery. Changes to Bugzilla are not reflected in Experimenter and will not change delivery configuration.
Data Science Issue: https://bugzilla.mozilla.org/show_bug.cgi?id=1609070
More information: https://experimenter.services.mozilla.com/experiments/activity-stream-chrome-switchers-card-for-onboarding-triplets-in-firefox-73-v3/
|
Reporter | |
Updated•5 years ago
|
|
|
Reporter | |
Comment 1•5 years ago
|
||
|
|
Reporter | |
Updated•5 years ago
|
Description
•