1579217 - New Fennec Sign-Up Buttons Telemetry

Status: RESOLVED WORKSFORME

(Data Science :: General, task)

Description

[athomas]

Brief Description of the request (required):

We're updating the onboarding screens & adding a new element to the new tab page as part of skyline. Looking for support in getting the right telemetry tracking in place to measure impact of this feature

Business purpose for this request (required):

Drive more FxA sign-ups (Skyline)

Requested timelines for the request or how this fits into roadmaps or critical decisions (required):

Part of Skyline

Links to any assets (e.g Start of a PHD, BRD; any document that helps describe the project):

PRD (with proposed success metrics): https://docs.google.com/document/d/12N9EM5BAx-lW6_oGSGLuVcX30VX17SitmZWqH2erg4Q/edit
Project plan: https://docs.google.com/document/d/1njMzQ4NGCN___ms2a2dkgVC4fi1OkWIaqxp3Y-vZLYw/edit#
Implementation tickets: https://bugzilla.mozilla.org/show_bug.cgi?id=1570878 , https://bugzilla.mozilla.org/show_bug.cgi?id=1570880

Name of Data Scientist (If Applicable):

Please note if it is found that not enough information has been given this will delay the triage of this request.

Comment 1

[Chris Peterson [:cpeterson]]

Link to this DS request in the "Mobile Prioritized DS/UR/UX Requests" spreadsheet:
https://docs.google.com/spreadsheets/d/1ofDg7dOMG4bqvb_dwaYakBleCmMlr5x6iIuVsrLf3x0/edit?ts=5d65d4cc#gid=0&range=4:4

Related Fennec bugs:

• Bug 1570878 - Add a sign-in button to each Fennec onboarding card
• Bug 1570880 - Add sign-in button and status to Fennec New Tab

Comment 2

[Emily Thompson]

Hi! just to understand the ask, is it to review a proposal for telemetry? (ie review the PRD)?

Comment 3

[athomas]

Yes, we have SV engineers to do the implementation. So I'm looking for a review of the proposed metrics & recommendation on how/where to implement

Comment 4

[Emily Thompson]

Thanks Ashley, sounds good! @Leif or @Kimmy, any chance this is this something you might be able to look into next week?

Comment 5

[Chris Peterson [:cpeterson]]

For reference, Grisha added a Leanplum event (in bug 1568605) to report when a Fennec user connects (i.e. "signup", "signin", or "reconnect") to a Firefox Account. We might be able to use that same Leanplum event but annotate it with a UI source:

• "Sign up" button on Onboarding card #1
• "Sign up" button on Onboarding card #2
• "Sign up" button on Onboarding card #3
• "Sign up" button on New Tab
• "Sign in" link on New Tab
• "Sign in" settings menu item

Comment 6

[Petru-Mugurel Lingurar [:petru]]

The firstrun-sync telemetry probe we already had for the Account onboarding screen was replaced in bug 1570878 with firstrun-signup for every onboarding screen's Sign up button.
For the New tab sign up button from bug 1570880 we were planning to add similar newtab-signup / newtab-signin telemetry probes.
For when the user signs-in from Settings we already have sync_setup.

This UIEvents are not mapped to MMA (Leanplum) and also are not tracked directly by Leanplum. But can easily be.

Grisha's ticket - bug 1568605 would apply to after the user actually signs-in and it would log to Leanplum (not also to our Telemetry) if the user just signed-up / signed-in for the first time / reconnected.

Comment 7

[Leif Oines [:loines]]

I can help with this. Ashley, would you like me to leave comments in the PRD? https://docs.google.com/document/d/12N9EM5BAx-lW6_oGSGLuVcX30VX17SitmZWqH2erg4Q/edit

Comment 8

[Leif Oines [:loines]]

Tagging Chris on this since Ashley's out ^

Comment 9

[Chris Peterson [:cpeterson]]

Stefan says the FxA team may already have all this FxA telemetry in Amplitude. I started an email thread with the FxA team.

We've been missing Leanplum telemetry from ARM64 Fennec (bug 1568674), but a fix should be shipping in a Fennec ESR 68.1.1 dot release later this week.

Comment 10

[Chris Peterson [:cpeterson]]

I think we can close this DS request. Alex Davis on the FxA team provided an Amplitude dashboard that shows Fennec FxA logins and registrations by Fennec entrypoint:

https://analytics.amplitude.com/mozilla-corp/dashboard/jac3w56

I think that dashboard will be adequate for monitoring the impact of the new Fennec Onboarding and New Tab sign-in buttons.