A/B experiment in Focus 9.0 to monitor the Ad Blocking feature
Categories
(Data Science :: Experiment Collaboration, task)
Tracking
(Not tracked)
People
(Reporter: vzare, Unassigned)
Details
Brief Description of the request:
We are introducing an Ad Blocking feature in Focus 9.0 and we would like to run a A/B experiment for the first 3 weeks of the release in order to monitor how this feature impacts key product metrics:
-DAU
-1-day retention
-7-day retention
-median session time
-95% percentile session time
-avg page load
-% of unique daily active users in the experiment cohort that enable ad blocking against total users
-% of unique daily active users in the experiment cohort that took action to disable ad blocking
Here is a similar A/B experiment dashboard for reference:
https://sql.telemetry.mozilla.org/dashboard/focus-8-0-release-dashboard
Fretboard configuration will be handled by the Focus team: https://github.com/mozilla-mobile/focus-android/issues/4148
50/50 split
Experiment Cohort: All ad blocking features enabled
Control Cohort: Hide ad blocking (hide ad blocking toggle in settings, ad blocking homescreen tip, and ad blocking onboarding cards)
Any timelines for the request or how this fits into roadmaps:
Focus 9.0 release is scheduled on Feb 12. We would like to start the experiment on this date.
Links to any assets (e.g Start of a PHD, BRD; any document that helps describe the project):
Name of Data Scientist (If Applicable):
Megan McCorquodale
|
||
Comment 1•5 years ago
|
||
Commits pushed to master at https://github.com/mozilla-services/shavar-list-creation-config https://github.com/mozilla-services/shavar-list-creation-config/commit/4a4fcd62f11ba62232efc3c1c1f4bd7f7ad84f6a bug 1520532 - add focus 9.0 lists to prod creation https://github.com/mozilla-services/shavar-list-creation-config/commit/17f13976d2762fc90c8cc1d2efcfb1d1f8b7b24a Merge pull request #39 from mozilla-services/add-focus9-lists-to-prod-1520532 bug 1520532 - add focus 9.0 lists to prod creation
|
|
||
Comment 2•5 years ago
|
||
Commits pushed to master at https://github.com/mozilla-services/shavar-server-list-config https://github.com/mozilla-services/shavar-server-list-config/commit/7108ca569fe598713696da305018ee58bfaafc6b bug 1520532 - serve focus 9.0 lists in prod shavar https://github.com/mozilla-services/shavar-server-list-config/commit/4eaade66daa2b9558e69953c2917a6884ba1e9ce Merge pull request #22 from mozilla-services/serve-focus9-lists-in-prod-1520532 bug 1520532 - serve focus 9.0 lists in prod shavar
|
||
Updated•5 years ago
|
The Focus team has decided to not move forward with the A/B experiment so we no longer need this dashboard.
|
|
||
Comment 4•5 years ago
|
||
Commits pushed to master at https://github.com/mozilla-services/shavar-list-creation-config https://github.com/mozilla-services/shavar-list-creation-config/commit/c764cdf30e218b4cd9505d289637110b3d8aeab1 bug 1520532: fix s3 key names https://github.com/mozilla-services/shavar-list-creation-config/commit/250423dbefd6164459ec2940ca20a3a9411fac4e Merge pull request #41 from mozilla-services/focus90-s3-keys bug 1520532: fix s3 key names
Description
•