Confirm that we have a sync probe for "have sync account"

RESOLVED WORKSFORME

Status

()

Firefox for Android
General
RESOLVED WORKSFORME
2 years ago
2 years ago

People

(Reporter: liuche, Unassigned)

Tracking

(Blocks: 1 bug)

Trunk
ARM
Android
Points:
---

Firefox Tracking Flags

(firefox49 affected)

Details

In order to measure how effective our Sync setup in firstrun is, we want to correlate firstrun experiment actions with whether people end up with a sync account. What telemetry data do we have for "user has a sync account", and is that accessible to the data analysis tools (redash?) that adavis and barbara use to analyze firstrun success?

Alex, Barbara, what sources/tools do you use to collect data [1] for deciding onboarding experiment success? Do they give you enough data about sync? Are there probes you still need in the tools that you use to make correlations?

mfinkle: do you know how to add more data to redash? [2]

[1] https://gecko.readthedocs.io/en/latest/toolkit/components/telemetry/telemetry/index.html
[2] https://wiki.mozilla.org/Mobile/Metrics/Redash#Data_Tables_.2F_Schemas
Flags: needinfo?(bbermes)
Flags: needinfo?(adavis)
Barbara and I have been using redash to access test data.

To my knowledge, we've only been able to tell if a user clicked on the button in onboarding to start the process to create an account but we never know if they actually finished making it. I am unaware of how I could find out if they successfully made an account. (or are logged into to one)

When reviewing our wiki, I am unsure if or where I could find the account information:
https://wiki.mozilla.org/Mobile/Metrics/Redash
Flags: needinfo?(adavis)
To clarify, we would like to know with certainty that someone completed making an account during onboarding (or during first session). I don't know if can tell that now in redash.
I'm sure there's somewhere where we track whether a user is signed into a sync account, but perhaps it's just not accessible from Redash.

Alex, what data have our previous experiment results been concluded from, the button click data, or a general trend in Sync accounts, or something else? I suspect that the sync account completion is tracked on the sync servers, which would have no idea where the sync setup was initiated from. Would it be sufficient to correlate whether people in experiments are signed into a sync account, or not?

mfinkle, do you know what determines whether a probe or telemetry is part of Redash? Who would be a good person to talk who knows the places where that sync account state is collected, and whether we can get that data into Redash to use in analysis?

If none of that is possible, we can add some kind of hacky probe for when we receive the "Your account has been confirmed - Sync preferences" page, and that would presumably be an event that triggers within the onboarding session.
Flags: needinfo?(mark.finkle)
adding nalexander and katie, as I believe we had a similar conversation with them previoulsy
Flags: needinfo?(nalexander)
Flags: needinfo?(capps)
Flags: needinfo?(bbermes)
(In reply to Chenxia Liu [:liuche] from comment #3)
> Alex, what data have our previous experiment results been concluded from,
> the button click data, or a general trend in Sync accounts, or something
> else? I suspect that the sync account completion is tracked on the sync
> servers, which would have no idea where the sync setup was initiated from.
> Would it be sufficient to correlate whether people in experiments are signed
> into a sync account, or not?

-We've been using the clicks on the button that's in the firstrun funnel.
-I'd be fine to know if people signed into an account (and when).
We added several event probes for interactions with a Fx Account:
fxaccount-changepassword: user changes a password
fxaccount-changeprofile: user changes their profile avatar (i think)
fxaccount-create	: user creates a fx account
fxaccount-delete	: user deletes a fx account
fxaccount-login: user logs into their fx account
fxaccount-syncprefs: user edits their sync preferences
Flags: needinfo?(mark.finkle)
It looks like we should be able to select these fxaccount action extras tagged with the firstrun session from a funnel of each onboarding experiment, so I'll close this bug. I'll play around with this later too, and see if I can make a query that does this.
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Flags: needinfo?(nalexander)
Flags: needinfo?(capps)
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.