Brief Description of the request: We are introducing an Ad Blocking feature in Focus 9.0 and we would like to run a A/B experiment for the first 3 weeks of the release in order to monitor how this feature impacts our key product metrics. Bucket A: no ad blocking (disable ad blocking toggle in settings, disable related homescreen tip, disable related onboarding cards) Bucket B: Ad blocking enabled Fretboard configuration will be handled by the Focus team. Here is a similar A/B experiment dashboard that was created for Focus 8.0: https://sql.telemetry.mozilla.org/dashboard/focus-8-0-release-dashboard Any timelines for the request or how this fits into roadmaps: Focus 9.0 release is scheduled on Feb 12. Ideally we would like to start the experiment on this date. Links to any assets (e.g Start of a PHD, BRD; any document that helps describe the project): https://docs.google.com/presentation/d/1xya135uIeNtdMkjEoz-9ZpGIIeV0LMhTzhJRSoX2ahs/edit#slide=id.g25275a8168_0_260 Name of Data Scientist (If Applicable): Megan McCorquodale
Bug 1520532 Comment 0 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
Brief Description of the request: We are introducing an Ad Blocking feature in Focus 9.0 and we would like to run a A/B experiment for the first 3 weeks of the release in order to monitor how this feature impacts key product metrics: -DAU -1-day retention -7-day retention -median session time -95% percentile session time -avg page load -% of unique daily active users in the experiment cohort that enable ad blocking against total users -% of unique daily active users in the experiment cohort that took action to disable ad blocking Here is a similar A/B experiment dashboard for reference: https://sql.telemetry.mozilla.org/dashboard/focus-8-0-release-dashboard Fretboard configuration will be handled by the Focus team: 50/50 Cohort A: no ad blocking (disable ad blocking toggle in settings, disable related homescreen tip, disable related onboarding cards) Cohort B: Ad blocking enabled Any timelines for the request or how this fits into roadmaps: Focus 9.0 release is scheduled on Feb 12. Ideally we would like to start the experiment on this date. Links to any assets (e.g Start of a PHD, BRD; any document that helps describe the project): https://docs.google.com/presentation/d/1xya135uIeNtdMkjEoz-9ZpGIIeV0LMhTzhJRSoX2ahs/edit#slide=id.g25275a8168_0_260 Name of Data Scientist (If Applicable): Megan McCorquodale
Brief Description of the request: We are introducing an Ad Blocking feature in Focus 9.0 and we would like to run a A/B experiment for the first 3 weeks of the release in order to monitor how this feature impacts key product metrics: -DAU -1-day retention -7-day retention -median session time -95% percentile session time -avg page load -% of unique daily active users in the experiment cohort that enable ad blocking against total users -% of unique daily active users in the experiment cohort that took action to disable ad blocking Here is a similar A/B experiment dashboard for reference: https://sql.telemetry.mozilla.org/dashboard/focus-8-0-release-dashboard Fretboard configuration will be handled by the Focus team: https://github.com/mozilla-mobile/focus-android/issues/4148 50/50 split Experiment Cohort: All ad blocking features enabled Control Cohort: Hide ad blocking (hide ad blocking toggle in settings, ad blocking homescreen tip, and ad blocking onboarding cards) Any timelines for the request or how this fits into roadmaps: Focus 9.0 release is scheduled on Feb 12. Ideally we would like to start the experiment on this date. Links to any assets (e.g Start of a PHD, BRD; any document that helps describe the project): https://docs.google.com/presentation/d/1xya135uIeNtdMkjEoz-9ZpGIIeV0LMhTzhJRSoX2ahs/edit#slide=id.g25275a8168_0_260 Name of Data Scientist (If Applicable): Megan McCorquodale
Brief Description of the request: We are introducing an Ad Blocking feature in Focus 9.0 and we would like to run a A/B experiment for the first 3 weeks of the release in order to monitor how this feature impacts key product metrics: -DAU -1-day retention -7-day retention -median session time -95% percentile session time -avg page load -% of unique daily active users in the experiment cohort that enable ad blocking against total users -% of unique daily active users in the experiment cohort that took action to disable ad blocking Here is a similar A/B experiment dashboard for reference: https://sql.telemetry.mozilla.org/dashboard/focus-8-0-release-dashboard Fretboard configuration will be handled by the Focus team: https://github.com/mozilla-mobile/focus-android/issues/4148 50/50 split Experiment Cohort: All ad blocking features enabled Control Cohort: Hide ad blocking (hide ad blocking toggle in settings, ad blocking homescreen tip, and ad blocking onboarding cards) Any timelines for the request or how this fits into roadmaps: Focus 9.0 release is scheduled on Feb 12. We would like to start the experiment on this date. Links to any assets (e.g Start of a PHD, BRD; any document that helps describe the project): https://docs.google.com/presentation/d/1xya135uIeNtdMkjEoz-9ZpGIIeV0LMhTzhJRSoX2ahs/edit#slide=id.g25275a8168_0_260 Name of Data Scientist (If Applicable): Megan McCorquodale