1241627 - Associate account conversions with onboarding experiments

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: First Run, defect)

Description

[:Margaret Leibovic]

We could do this with UI telemetry, or maybe we could pass a parameter to the FxA server so that the cloud services folks can do this association for us.

Comment 1

[Nick Alexander :nalexander [he/him]]

I think it's sensible to add UI telemetry for the major things that happen in response to FxA WebChannel messages.  See https://dxr.mozilla.org/mozilla-central/source/mobile/android/modules/FxAccountsWebChannel.jsm#29.

In particular, the following are relevant to the browser and are probably interesting to track in telemetry:

const COMMAND_LOGIN                = "fxaccounts:login";
const COMMAND_CHANGE_PASSWORD      = "fxaccounts:change_password";
const COMMAND_DELETE_ACCOUNT       = "fxaccounts:delete_account";
const COMMAND_SYNC_PREFERENCES     = "fxaccounts:sync_preferences";

Let's add a generic event for COMMAND_LOGIN (possibly with a flag to know if we created a new account or re-connected an existing account, or possibly two events for these cases).

Then we can associate the event to a particular UI telemetry session, which could include a long-lived "onboarding-X" session.

Comment 2

[Mark Finkle (:mfinkle) (use needinfo?)]

Attachment: fxaccount-probes v0.1

This patch adds a few probes for some actions.

Nick - I made assumptions about the spot to put the probes. Look right?
Margaret - I went with (ACTION, CONTENT, "fxaccount-*") which seemed OK. I don't want to make a new Event type for this and these are actions that happen in a web-based UI.

Nick - I also removed some imports you weren't using. I also made Prompt a lazy load, which is our convention since we never seem to Prompt on script load.

Comment 3

[Nick Alexander :nalexander [he/him]]

Comment on attachment 8726075 [details] [diff] [review]
fxaccount-probes v0.1

Review of attachment 8726075 [details] [diff] [review]:
-----------------------------------------------------------------

wfm.

::: mobile/android/modules/FxAccountsWebChannel.jsm
@@ +331,5 @@
>                return Accounts.deleteFirefoxAccount().then(success => {
>                  if (!success) {
>                    throw new Error("Could not delete Firefox Account!");
>                  }
> +                UITelemetry.addEvent("action.1", "content", null, "fxaccount-delete");

Just a note - there are other ways to delete a Firefox Account.  I'm sure you're aware, just needs to color this data.

Comment 4

[:Margaret Leibovic]

Comment on attachment 8726075 [details] [diff] [review]
fxaccount-probes v0.1

Review of attachment 8726075 [details] [diff] [review]:
-----------------------------------------------------------------

Looks fine from a telemetry probe naming perspective (I defer to Nick on the correctness for where to put these probes).

Comment 5

[Mark Finkle (:mfinkle) (use needinfo?)]

(In reply to Nick Alexander :nalexander from comment #3)

> Just a note - there are other ways to delete a Firefox Account.  I'm sure
> you're aware, just needs to color this data.

We decided to land this patch, scoped to meet the desired needs: track FxAccount signups from onboarding.

Nick provided links to other call sites that can allow the account to be deleted. If we need to track those, we can followup in a new bug.

https://dxr.mozilla.org/mozilla-central/source/mobile/android/services/src/main/java/org/mozilla/gecko/fxa/activities/FxAccountStatusFragment.java#242
https://dxr.mozilla.org/mozilla-central/source/mobile/android/services/src/main/java/org/mozilla/gecko/fxa/receivers/FxAccountDeletedService.java#40

Comment 6

[Pulsebot]

https://hg.mozilla.org/integration/fx-team/rev/4ad4e4652aa8

Comment 7

[Mark Finkle (:mfinkle) (use needinfo?)]

We should try to uplift these probes to Fx46 so we can better track the onboarding experiment.

Comment 8

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/4ad4e4652aa8

Comment 9

[Mark Finkle (:mfinkle) (use needinfo?)]

Comment on attachment 8726075 [details] [diff] [review]
fxaccount-probes v0.1

Approval Request Comment
[Feature/regressing bug #]: onboarding experiment #2
[User impact if declined]: We'd miss out learning if the onboarding experiments affected fx account sign-ups
[Describe test coverage new/current, TreeHerder]: Landed on Nightly
[Risks and why]: Low. Just adding some probes and removing unused imports
[String/UUID change made/needed]: none

Comment 10

[Liz Henry (:lizzard) (relman/hg->git project)]

Comment on attachment 8726075 [details] [diff] [review]
fxaccount-probes v0.1

Let's see if we can get this into 46 beta 1. please uplift.

Comment 11

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/3e77fa274604