1575318 - Define engagement success for the Interventions add-on experiment

Status: RESOLVED INACTIVE

(Data Science :: Experiment Collaboration, task)

Description

[Harry Twyford]

Brief Description of the request (required):

Interventions is an add-on experiment that will insert contextual items into the Urlbar autocomplete panel as the user types to intervene on their searches, e.g. offering a "Refresh Firefox" button if the user is typing firefox is slow. The contextual item uses the same APIs as and looks very similar to a Nudge, with a string explaining the intervention and offering a button so the user can act on it.

I am requesting that the Data Science team determine success metrics on this experiment so that they can be engineered.

Business purpose for this request (required):

This work must be completed before the Interventions experiment is shipped. Interventions is an important part of the work on Quantum Bar and its experiments.

Requested timelines for the request or how this fits into roadmaps or critical decisions (required):

The current plan calls for Interventions to land in 70 but that is unlikely given the number of uplifts that would be required. It is more likely that this is moved to 71, in which case we'd probably need this Data Science work done at least two weeks before the 71 beta merge date? That said, earlier is better :)

Links to any assets (e.g Start of a PHD, BRD; any document that helps describe the project):

InVision spec: https://mozilla.invisionapp.com/share/S5SQCACBVAF#/375525556_Search_Interventions_Experiment
Meeting notes: https://docs.google.com/document/d/1Rf3No0ECkeiQJXGzlM67XlueucvN1t4S8DZL8PMOVV4/edit

Name of Data Scientist (If Applicable):

Ben Miroglio, possibly others?

---

Seeing as the two experiments are fairly similar from a UX perspective, this experiment may require the same data as the Nudges experiment, as outlined in bug 1572174. It would also be good to know what kinds of interventions users are seeing most often, and the rate at which users act on an intervention by clicking its button.

Comment 1

[Ben Miroglio [:bmiroglio]]

Thanks for filing this!

Would you mind filing a bug under Data Science > Experiment Collaboration

A template is there for that component that helps our team size / triage these requests.

Comment 2

[Harry Twyford]

I've updated the bug description with the template.

Comment 3

[Ben Miroglio [:bmiroglio]]

Megan, do you want to work on this experiment? We can work on it together if you'd like.

Comment 4

[Megan McCorquodale]

Sure, let's collaborate on this one @bmiroglio

Comment 5

[Experimenter Bot]

[Experiment]Search Interventions status has been changed to: Ship
            url:https://experimenter.services.mozilla.com/experiments/search-interventions/

Comment 6

[Experimenter Bot]

[Experiment]Search Interventions status has been changed to: Live
            url:https://experimenter.services.mozilla.com/experiments/search-interventions/

Comment 7

[Experimenter Bot]

[Experiment]Search Interventions V2 status has been changed to: Ship
            url:https://experimenter.services.mozilla.com/experiments/search-interventions-v2/

Comment 8

[Experimenter Bot]

[Experiment]Search Interventions V2 status has been changed to: Live
            url:https://experimenter.services.mozilla.com/experiments/search-interventions-v2/

Comment 9

[Jess Mokrzecki [:jmok]]

Work for the DS team is now tracked in Jira. You can search with the Data Science Jira project for the corresponding ticket.