1568605 - Send a Leanplum event when the user signs up for a Firefox Account

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: Metrics, enhancement, P1)

Description

[Stefan Arentz | :st3fan | ⏰ EST | he/him]

This event should fire when a user signs up for FxA. This will help us figure which message creates higher conversions.

The event name should be E_User_Signed_Up_For_FxA and should be sent after the user has gone through the sign up flow.

This is related to the work in https://bugzilla.mozilla.org/show_bug.cgi?id=1380854

Comment 1

[Stefan Arentz | :st3fan | ⏰ EST | he/him]

This may include work to make sure that the Sign In event does not trigger on a Sign Up. It is not entirely clear from bug 1380854 if that is currently the case. If it is then that needs to be changed as part of this bug.

Comment 2

[:Grisha Kruglov]

What's required here:

• good news: fxa webchannel message fxaccounts:login comes with a LoginData object, which includes various detail about the login/sign-up event. https://github.com/mozilla/fxa/blob/master/packages/fxa-content-server/docs/relier-communication-protocols/fx-webchannel.md#fxaccountslogin
• bad news: the only way to figure out if that message is describing a "sign up" is by inspecting the LoginData object, and looking at the offeredSyncEngines list. Its absence indicates no sync engines were "offered" to the user, and since we only offer "what to sync" during sign-up, and never on login, presence of that list should indicate that a sign-up happened.

This should work; it assumes a lot about the underlying UI flow and product/eng decisions around how we present users with CWTS options.
Longer term, reliable approach would be to include a definitive action param in fxaccounts:login. I filed https://github.com/mozilla/fxa/issues/1998 to track this.

In the shorter term, we can inject that parameter starting at https://dxr.mozilla.org/mozilla-central/source/mobile/android/modules/FxAccountsWebChannel.jsm#307, based on presence/absence of offeredSyncEngines.

The rest should be straightforward: inspect action in the AccountsHelper.java, and fire off a correct mma message.

Comment 3

[:Grisha Kruglov]

Attachment: Bug 1568605 - Send corrent Leanplum events during signin, signup and reconnect r=nalexander

This patch augments FxA messages sent to native code with just enough information that we are able
to differentiate between "signin", "signup" and "reconnect" events.

Corresponding Leanplum events are sent on the receiving end of the FxA messages.

Comment 4

[:Grisha Kruglov]

I'm assuming that the two new Leanplum events will be automatically understood by Leanplum when it receives them. Trying to confirm that with folks more familiar with Leanplum.

Comment 5

[Chris Peterson [:cpeterson]]

[Tracking Requested - why for this release]:

Grisha, we'll need to uplift your Leanplum patch to mozilla-esr68 because Fennec Nightly, Beta, and Release channels are all built from Fennec ESR 68. We'll need to decide whether we can ship your Leanplum patch in Fennec ESR 68.1 (September 3) or must rush it out in an ESR 68.0.x dot release.

Fennec Release is ESR 68.0.x (from mozilla-esr68 branch FIREFOX_ESR_68_0_X_RELBRANCH). Fennec Nightly and Beta are pre-release builds of ESR 68.1 (from mozilla-esr68 default).

Comment 6

[David Bolter [:davidb] (NeedInfo me for attention)]

I'm told Loren might know?

(In reply to :Grisha Kruglov from comment #4)

I'm assuming that the two new Leanplum events will be automatically understood by Leanplum when it receives them. Trying to confirm that with folks more familiar with Leanplum.

Comment 7

[laustin]

We can "verify when an event is tracked in the Debugger or the Events tab in the Leanplum dashboard" (via https://docs.leanplum.com/reference#events).

Events tab for Fennec Dev here: https://www.leanplum.com/dashboard#/4739339185029120/content/events

I'm seeing these events showing up there:
E_User_Signed_In_To_FxA
E_User_Signed_Up_To_FxA

Those look like the event names you set up, so it appears that Leanplum is receiving them; however, I can't verify whether or not those events are generated at the appropriate time. Any chance that can be verified events are firing appropriately in the debugger?

Comment 8

[laustin]

Also noting that "USER_RECONNECTED_TO_FXA" is not showing up in the Leanplum Events dashboard.

Comment 9

[:Grisha Kruglov]

The patch hasn't landed yet, so you won't see the new event until it lands and at least makes its way to a nightly build (that'll take a day or two).

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/19de17673bcb
Send corrent Leanplum events during signin, signup and reconnect r=nalexander

Comment 11

[Wil Clouser [:clouserw]]

I'm CCing @rfkelly, @vbudhram, and @markh because comment 2 is talking about detecting this based on offeredSyncEngines and in Fx71 we're decoupling signing up for FxA from signing up for sync. I don't know if that affects this at all, but I'd rather flag it now than have this stop working unexpectedly.

Comment 12

[Ryan Kelly [:rfkelly]]

but I'd rather flag it now than have this stop working unexpectedly.

As described in this proposal doc we want to start offering choose-what-to-sync to existing account holders, which would break the assumptions you're depending on here. I think it would be reasonable to tackle fxa#1998 as part of that work in order to give you a more reliable indicator of signin vs signup.

Comment 13

[Ryan Kelly [:rfkelly]]

I think it would be reasonable to tackle fxa#1998 as part of that work in order to give you a more reliable indicator of signin vs signup.

Which is to say, perhaps you should pre-emptively look for an action parameter in the login message and fall back to offeredSyncEngines etc if it's missing, so that we don't need to update Fennec again when we make that change in FxA. Please summarize the expected values of the action parameter in the above FxA bug for our context (as I see you added some others here besides "signin" and "signup").

Comment 14

[:Grisha Kruglov]

Thanks Ryan, preemptively looking for action with a fallback makes sense. From the Slack/github threads, I assumed that the timeline for FxA changes is longer than what you suggest. But either way, being forward-compatible is a good thing regardless. I'll push a follow-up, so that we can uplift the forward-compatible solution here.

Comment 15

[Ryan Kelly [:rfkelly]]

I assumed that the timeline for FxA changes is longer than what you suggest.

Aye; I don't have a concrete timeline for this work yet, but when we do decide to do it, it could happen in a real hurry, so best to be prepared :-)

Comment 16

[:Grisha Kruglov]

(In reply to Ryan Kelly [:rfkelly] from comment #15)

I assumed that the timeline for FxA changes is longer than what you suggest.

Aye; I don't have a concrete timeline for this work yet, but when we do decide to do it, it could happen in a real hurry, so best to be prepared :-)

Landed a follow-up in Bug 1570496 which should be forward-compatible.

We'll need to uplift both this and 1570496 together.

Comment 17

[:Grisha Kruglov]

Updated https://github.com/mozilla/fxa/issues/1998#issuecomment-517032380 with a description of the current client behaviour.

Comment 18

[Razvan Maries]

https://hg.mozilla.org/mozilla-central/rev/19de17673bcb

Comment 19

[:Grisha Kruglov]

Comment on attachment 9081839 [details]
Bug 1568605 - Send corrent Leanplum events during signin, signup and reconnect r=nalexander

ESR Uplift Approval Request

• If this is not a sec:{high,crit} bug, please state case for ESR consideration: This is needed for the Skyline project, allowing us to better track performance of the accounts onboarding UIs.
• User impact if declined: None for the user; mozilla will be unable to differentiate sign-ups from signins in the leanplum telemetry.
• Fix Landed on Version: 70
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Not risky; to be considered together with 1570496 (and needs to be uplifted along with it!). Patch takes into the account current state of the webchannel, as well as the upcoming changes that will happen in fxa-web-content repos. FxA side is aware of these changes, and will act accordingly in the future to accommodate them.
• String or UUID changes made by this patch: n/a

Comment 20

[Julien Cristau [:jcristau]]

Comment on attachment 9081839 [details]
Bug 1568605 - Send corrent Leanplum events during signin, signup and reconnect r=nalexander

fennec leanplum/fxa update, approved for 68.1 so this gets into nightly and beta builds

Comment 21

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/releases/mozilla-esr68/rev/3bb57b1847b9

Comment 22

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9081839 [details]
Bug 1568605 - Send corrent Leanplum events during signin, signup and reconnect r=nalexander

We'll want this on Beta also for parity.

Comment 23

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/34eba66f4965