1543770 - Reporting for Return-to-AMO feature

Status: RESOLVED FIXED

(Data Science :: Investigation, task)

Description

[Amy Tsay [:amyt]]

Brief Description of the request (required):

In Firefox 66, we launched the Return-to-AMO feature, which lands new users of Firefox back on the AMO extension page where they initiated the Firefox install.

Business purpose for this request (required):

This feature was implemented to shore up possible Firefox retention loss due to new users not being able to figure out how to install an extension that they installed Firefox to get. We'd like to know if the feature is making an impact by taking a look at the following:

• Number of times the feature has been triggered since launch; how many per day/week
• Click-through rate
• Install rate
• Impact on retention

Requested timelines for the request or how this fits into roadmaps or critical decisions (required):

The feature went live mid-March, so we'd like to see reporting by end of April/early May if possible. Critical decisions are not gated on this reporting.

Links to any assets (e.g Start of a PHD, BRD; any document that helps describe the project):

https://trello.com/c/fiSvzyRH/560-return-to-amo
https://bugzilla.mozilla.org/show_bug.cgi?id=1468680

Name of Data Scientist (If Applicable):

Please note if it is found that not enough information has been given this will delay the triage of this request.

Comment 1

[:shell escalante]

in case it helps - the previous "impact on retention" data we had can be found in this section of the PRD https://docs.google.com/document/d/1BukJcOsfXoP651IN8P2BTZ8jpc83KAa3caMKlpjTxDQ/edit#heading=h.p2843q3fz1e1.

Comment 2

[Amy Tsay [:amyt]]

Hi, just wanted to check in on this--what should we expect in terms of timeline with "Points: 2"? Thanks!

Comment 3

[Corey Dow-Hygelund [:ccd]]

Amy, I intend to send out a report early next week.

Comment 4

[Corey Dow-Hygelund [:ccd]]

Initial analysis: https://metrics.mozilla.com/protected/cdowhygelund/return_to_amo_anlysis.html

TBD:

• Click through rate
• retention (6 week?)
• fraction of add-on left enable over time.

Comment 5

[Corey Dow-Hygelund [:ccd]]

As per ddurst, we don't have adequate telemetry to determine click-through rate.

Comment 6

[Corey Dow-Hygelund [:ccd]]

:amyt Analysis complete and ready for your review.

Comment 7

[Corey Dow-Hygelund [:ccd]]

In addition, generated query for RTA analog similar to that referenced in PRD, which compares retention of RTA to footer installations

Comment 8

[Corey Dow-Hygelund [:ccd]]

Bug 1549770 will help address the significant uncertainty and unknowns in the estimates, by addressing instrumentation in AS.

Comment 9

[Corey Dow-Hygelund [:ccd]]

Activity Stream telemetry is being in added to enable much better (and simpler to calculate) estimations of:

• Number of times the feature has been triggered since launch; how many per day/week
• Click-through rate
• Install rate

Comment 10

[Corey Dow-Hygelund [:ccd]]

Requisite telemetry targeted for 68.

Comment 11

[Corey Dow-Hygelund [:ccd]]

Closing this bug. A new one should be submitted for the newly added v68 telemetry.