Closed Bug 1520532 Opened 2 years ago Closed 2 years ago

A/B experiment in Focus 9.0 to monitor the Ad Blocking feature

Categories

(Data Science :: Experiment Collaboration, task)

x86_64
macOS
task
Not set
normal
Points:
2

Tracking

(Not tracked)

RESOLVED INVALID

People

(Reporter: vzare, Unassigned)

Details

Brief Description of the request:

We are introducing an Ad Blocking feature in Focus 9.0 and we would like to run a A/B experiment for the first 3 weeks of the release in order to monitor how this feature impacts key product metrics:

-DAU
-1-day retention
-7-day retention
-median session time
-95% percentile session time
-avg page load
-% of unique daily active users in the experiment cohort that enable ad blocking against total users
-% of unique daily active users in the experiment cohort that took action to disable ad blocking

Here is a similar A/B experiment dashboard for reference:
https://sql.telemetry.mozilla.org/dashboard/focus-8-0-release-dashboard

Fretboard configuration will be handled by the Focus team: https://github.com/mozilla-mobile/focus-android/issues/4148

50/50 split
Experiment Cohort: All ad blocking features enabled
Control Cohort: Hide ad blocking (hide ad blocking toggle in settings, ad blocking homescreen tip, and ad blocking onboarding cards)

Any timelines for the request or how this fits into roadmaps:

Focus 9.0 release is scheduled on Feb 12. We would like to start the experiment on this date.

Links to any assets (e.g Start of a PHD, BRD; any document that helps describe the project):

https://docs.google.com/presentation/d/1xya135uIeNtdMkjEoz-9ZpGIIeV0LMhTzhJRSoX2ahs/edit#slide=id.g25275a8168_0_260

Name of Data Scientist (If Applicable):

Megan McCorquodale

Points: --- → 2

The Focus team has decided to not move forward with the A/B experiment so we no longer need this dashboard.

Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.