Web metrics for the Hello onboarding campaign

RESOLVED FIXED

Status

www.mozilla.org
Analytics
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: ckprice, Unassigned)

Tracking

Production
x86
Mac OS X
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [kb=1630766] , URL)

Tracking web metrics for the Hello onboarding campaign

Elements and responsibilities:

- Snippet (GA 36): Osmose/sgarrity
- Product Page (GA 35): jpetto
- FTUE Tour (GA 35): agibson
- Google Analytics set up & confirmations: cmore

Matrix: https://docs.google.com/a/mozilla.com/spreadsheets/d/1crhDaU17YSz5IR3PSoqnU3naX5VrdDYRIkwTxjfFsoU/edit#gid=188348682
After meetings with the webdev and Fx engineering team, below is what will be possible:

== GA 35 (January 13th) ==

Product Page
# visits
# users who did not have Hello available but were updated
# users who had Hello available
# users who opened Hello panel
# users who clicked on the alternative CTA (sumo article)
# users who clicked "Start a conversation" straight from the product page (e.g. already took tour)
# users who clicked "Copy link" straight from the product page

FTUE tour
# users who started the tour
# users who clicked "Start a conversation"
# users who clicked "Copy link"
# conversations started

== GA 36 (February 24th) ==

Tie in events back to source (snippet ID, product page, etc)
Sharing links directly to the tour could be at risk because of locale configuration
# Better experience to bring user to the product page with unique campaign identifiers instead of forcing them into a tour
# We could track the source using this route.
Would be nice to differentiate Copy Link vs Email Link.
(In reply to Cory Price [:ckprice] from comment #2)
> Would be nice to differentiate Copy Link vs Email Link.

:agibson - does bug 1080948 supply this? Or do clicks on either button come through the same? No rush here.
Flags: needinfo?(agibson)
(In reply to Cory Price [:ckprice] from comment #3)
> (In reply to Cory Price [:ckprice] from comment #2)
> > Would be nice to differentiate Copy Link vs Email Link.
> 
> :agibson - does bug 1080948 supply this? Or do clicks on either button come
> through the same? No rush here.

No we can't differentiate on which one is clicked - the same event fires for both.
Flags: needinfo?(agibson)
Updated spreadsheet[0] separating GA 35 (complete) and GA 36 (WIP) tagging into different sheets.

For GA 35 we will not be able to track the source for Tour referrals. Bug 1115227 is the first step to capturing this information, and we're targeting GA 36.

(In reply to Cory Price [:ckprice] from comment #2)
> Would be nice to differentiate Copy Link vs Email Link.
Bug 1115225 is opened to facilitate this request; targeting GA 36.

[0] Hello onboarding tagging matrix - https://docs.google.com/a/mozilla.com/spreadsheets/d/1crhDaU17YSz5IR3PSoqnU3naX5VrdDYRIkwTxjfFsoU/edit#gid=0
Thanks Cory,

The FTE GA events for Firefox 35 are now implemented and up on demo2 for testing as part of Bug 1109132
Duplicate of this bug: 1108830
Updating to block GA 36 tracker
Blocks: 1113400
No longer blocks: 1084075
Whiteboard: [kb=1630766]
We investigated the possibility of just using the utm_content parameter to associate the TourConnectConversation event to a referrer. Unfortunately, the results[1] showed that the final event was not associated with the original Ad Content value.

To connect the TourConnectConversation event to a referrer, we'll be using the ideas in bug 1115227 and bug 1117898 comment 3.

The Hello tagging matrix[0] has been updated to reflect additions and changes for GA 36.

This is blocked by bug 1115227.

[0] Hello tagging matrix https://docs.google.com/a/mozilla.com/spreadsheets/d/1crhDaU17YSz5IR3PSoqnU3naX5VrdDYRIkwTxjfFsoU/edit#gid=408649200
[1] Testing results https://hello-onboarding.etherpad.mozilla.org/2015-01-22-ga-event-testing
Depends on: 1115227
Depends on: 1130194
Depends on: 1130198
Hi Cory, quick question on the custom variables for /whatsnew:

In the Google doc we have:

_setCustomVar(12, "Hello Available", "Yes", "2")
_setCustomVar(12, "Hello Available", "No", "2")

However looking at previous custom variable usage we have a slightly different syntax:

_gaq.push(['_setCustomVar', 5, 'First Time Taking Firefox Tour', 'true', 2]);

Can you confirm which one we should be using here Cory? Also note the last variable in the array is a string in the top example ("2"), but a number in the existing example (2).

Also just want to make sure the only other new GA event being added is when someone adds the hello icon e.g:

_gaq.push(['_trackEvent', 'Tour Interaction','Add it now','The Hello icon']);

I guess we're not as concerned here with collecting data for those that click "Later" and don't choose to add it? (or perhaps we can already gather this statistic without needing to another event for it).
Flags: needinfo?(cprice)
(In reply to Alex Gibson [:agibson] from comment #10)
> Hi Cory, quick question on the custom variables for /whatsnew:
> 
> In the Google doc we have:
> 
> _setCustomVar(12, "Hello Available", "Yes", "2")
> _setCustomVar(12, "Hello Available", "No", "2")
> 
> However looking at previous custom variable usage we have a slightly
> different syntax:
> 
> _gaq.push(['_setCustomVar', 5, 'First Time Taking Firefox Tour', 'true', 2]);
> 
> Can you confirm which one we should be using here Cory? Also note the last
> variable in the array is a string in the top example ("2"), but a number in
> the existing example (2).
> 
> Also just want to make sure the only other new GA event being added is when
> someone adds the hello icon e.g:
> 
> _gaq.push(['_trackEvent', 'Tour Interaction','Add it now','The Hello icon']);
> 
> I guess we're not as concerned here with collecting data for those that
> click "Later" and don't choose to add it? (or perhaps we can already gather
> this statistic without needing to another event for it).

Let's stick to convention. Google Doc updated to address both of your points. Thanks for them bringing up.
Flags: needinfo?(cprice)
No longer depends on: 1130194
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.