1128713 - Request: Sticky Behavior Tracking for Add-on Installs

Status: RESOLVED INCOMPLETE

(addons.mozilla.org Graveyard :: Statistics, defect, P2)

Description

[[e-mail]]

Hello,

I'm submitting a request to enable Google Analytics tracking for Add-on and Theme installs. 

Use case: 

Firefox's digital marketing channels (i.e. email, social media, and Snippet) regularly send users to install Add-ons and themes to promote sticky behavior and ultimately increase Firefox usage hours. We'd like to be able to track all the way from channel message to install (like we already can do for Desktop downloads, for example). This will ensure that we're testing and optimizing our content to generate the most impact. 

Note:

We link to add-on and theme collections significantly more than linking to specific Add-ons or themes. Our hope is that the Google Analytics tracking will also help us to see what Add-ons and Themes are being installed when we give our users choice of what to install. 

Please let me know if you have any questions! Would love to have this enabled by end of February, but it is an internal deadline. If not possible, just let me know. 

Thanks!
Michaela

Comment 1

[Gareth Cull [:garethc]]

Can we please call the following event on click of any +Add to Firefox button?

_gaq.push(['_trackEvent', 'AMO Addon / Theme Installs', type, name]);

where type is:
theme
addon

where name is:
name of the Addon or Theme. Thanks.

Gareth

Comment 2

[[e-mail]]

Hi Gareth -- I don't think others are on this bug except you, me, and Liz. Who else should be added to enable this?

Comment 3

[Gareth Cull [:garethc]]

Wil, is this something your team can help implement?

Comment 4

[Wil Clouser [:clouserw]]

Yes.  There is a lot of work in the queue already and <1 FTE developer on AMO so it may be a while.

Comment 5

[Liz Hull]

Hi Wil! Just following up on this. Has there been any progress in getting tracking attached to add-on and theme downloads? We are building a dashboard to track both acquisition and retention activities, and these are two of our big retention activities! 

Thanks so much, 

Liz

Comment 6

[Wil Clouser [:clouserw]]

The signing project which we've been working on is nearing conclusion and this bug is in the chunk of bugs to do next.  So...I'd say a few weeks?

Comment 7

[[e-mail]]

Hi team -- I changed the name of this bug to reflect the use case: Sticky Behavior Tracking (expanding beyond add-on and theme install tracking). 

We are blocked on understanding what marketing messages lead to sticky behavior for Firefox users given our lack of ability to track:

- Add-on installs
- Theme installs
- Hello calls
- Sync sign ups
- Pocket sign ups
- Accounts sign ups
- Bookmarks creation

Is this the right place within Bugzilla to restart a discussion of how to tackle all of these different fronts? Gareth, hoping you can help.

Comment 8

[Gareth Cull [:garethc]]

Hey Michaela,

I think the additions of the following really expand the scope of this bug to beyond just tracking installs on AMO, as they happen elsewhere.  

- Hello calls
- Sync sign ups
- Pocket sign ups
- Accounts sign ups
- Bookmarks creation
 
You may want to create a new tracking bug for all of these and then link back to this bug in its original form. Then wil could give an update on when we think we can get event tracking implemented on amo. 

:clouserw do you have an ETA on this?

Also, how do you plan on attributing marketing initiatives to what looks like actions that primarily happen in the browser. It may be a bit of a challenge unless you are driving traffic to specific product landing pages or using snippets to trigger these actions. Happy to hop on a call sometime next week to discuss further if you'd like.

Comment 9

[Wil Clouser [:clouserw]]

(In reply to Gareth Cull [:garethc] from comment #8)
> :clouserw do you have an ETA on this?


hmmm...I was expecting this to be done before now.  I will need to get back to you.

Comment 10

[Mathieu Agopian [:magopian]]

PR: https://github.com/mozilla/olympia/pull/639

Comment 11

[[:josephine]]

We already have a bug on account sign ups: Bug 1162192

Comment 12

[[github robot]]

Commits pushed to master at https://github.com/mozilla/olympia

https://github.com/mozilla/olympia/commit/9e665346f770d894b2c84d655795fc65f10f70dd
Track addons and themes install (bug 1128713)

https://github.com/mozilla/olympia/commit/e91b3d9902debcce594c795a310c903285393b43
Merge pull request #639 from magopian/1128713-addons-personas-install-tracking

Track addons and themes install (bug 1128713)

Comment 13

[Victor Carciu]

Can you please add some STRs to this bug or mark it as [qa-] ?

Comment 14

[Mathieu Agopian [:magopian]]

STR: whenever you install an add-on or a theme (from the listing page, from a search, from a collection, from the landing page, from the discovery pane...), you should see a GET request in the "network" tab of the dev tools.

YOU NEED TO DISABLE BLOCKERS like ublock/adblock/ghostery/disconnect or they will block the requests to google analytics.

The request should look something like that: 

https://ssl.google-analytics.com/__utm.gif?utmwv=5.6.5&utms=2&utmn=510375946&utmhn=addons-dev.allizom.org&utmt=event&utme=5(AMO%20Addon%20%2F%20Theme%20Installs*addon*ReminderFox)&utmcs=UTF-8&utmsr=1440x900&utmvp=1440x541&utmsc=24-bit&utmul=fr&utmje=1&utmfl=18.0%20r0&utmdt=Modules%20pour%20Firefox&utmhid=1743589317&utmr=-&utmp=%2Ffr%2Ffirefox%2F&utmht=1438858465601&utmac=UA-36116321-7&utmcc=__utma%3D128045276.477163730.1432216791.1432216791.1438858458.2%3B%2B__utmz%3D128045276.1432216791.1.1.utmcsr%3D(direct)%7Cutmccn%3D(direct)%7Cutmcmd%3D(none)%3B&utmjid=&utmu=6AAAAAAAAAAAAAAAAAAAAAAE~

The important part is (AMO%20Addon%20%2F%20Theme%20Installs*addon*ReminderFox): it needs to say:
AMO%20Addon%20%2F%20Theme%20Installs*<"addon" or "theme" depending on what is installed>*<Name of the addon or theme>

Comment 15

[Mathieu Agopian [:magopian]]

I just realized that this doesn't work in discovery pages (it's missing the google analytics script). It's the cause of bug 1191391, that I'll fix there.

Comment 16

[Madalin Cotetiu]

Everything appears to work well except one thing. When installing an addon from discovery pane after performing a search the request is not visible. After talking with :magopian we decided to log a new bug for this specific issue. 
Closing this bug, and :magopian will open a new bug for that specific scenario.

Comment 17

[Mathieu Agopian [:magopian]]

Bug for the discovery pane search: bug 1191887

Comment 18

[[e-mail]]

Just posted this in bug 1191887, but in case it is better here:

"Ultimately, we know that add-on installations are indicative of sticky behavior. We run multiple campaigns every month directing users to add-on collections, and even specific add-ons. If Google Analytics isn't possible per add-on, I would look to ya'll to see what is. 

This is a part of a larger effort to track behaviors that impact retention so that we can get better at user retention."

Comment 19

[[e-mail]]

Hi team -- I'm re-opening this because I'm not sure it is resolved. Is tracking now enabled for add-ons and themes? 

If yes:

(1) How should we -- as marketers -- tag our links to specific add-ons or add-ons collections to make sure that our efforts are measured?

(2) Where can we go to see the metrics, so that we can optimize our content accordingly?

Thanks!

Comment 20

[Gareth Cull [:garethc]]

(In reply to [e-mail] from comment #19)
> Hi team -- I'm re-opening this because I'm not sure it is resolved. Is
> tracking now enabled for add-ons and themes? 


I just checked GA and we are getting install data for themes and addons. Now that this is enabled on addons.mozilla.org, you should be able to attribute these events to a specific marketing channel.

> 
> If yes:
> 
> (1) How should we -- as marketers -- tag our links to specific add-ons or
> add-ons collections to make sure that our efforts are measured?

You should use the same source and medium parameters you are using when driving traffic to mozilla.org, and update the campaign and content parameters accordingly.

> (2) Where can we go to see the metrics, so that we can optimize our content
> accordingly?

Metrics are available in addons.mozilla.org. Can you file a bug for anyone that needs access to this account? Thanks.

Gareth

Comment 21

[[e-mail]]

Awesome! So glad to hear this. Is it retroactive? Or just based on what has come through since it was enabled?

Gareth -- which bug form should I use? The people who need access are:

Chelsea Novak
TJ Lee
Ben Niolet
Jess Osorio
Jessilyn Davis
Elizabeth Hull
Maura Tuohy
Jean Collings
Jessica Vondrak
Me

Comment 22

[Mathieu Agopian [:magopian]]

Just as a reminder, there's still the bug 1191887 that is opened, haven't had the time to work on that yet. It means that there's no analytics for add-ons installed from the discovery pages (the about:addons page) when doing a search.

Comment 23

[Gareth Cull [:garethc]]

Ben,

You can file a bug for access to that site here and I'll take a look at it: 
https://bugzilla.mozilla.org/enter_bug.cgi?product=Websites&component=Web%20Analytics

Thanks.

Gareth

Comment 24

[Wil Clouser [:clouserw]]

re-closing this verified bug...

Comment 25

[[e-mail]]

Hi team -- I'm re-opening. 

Gareth, the in-product communications team told me that the tracking that has been implemented actually doesn't track all the way to install. Instead, it tracks clicks on the install button (which trigger a doorhanger, which then gives a user the opportunity to continue or discontinue with the install process). 

Is there a way to truly track all the way to install?

Comment 26

[Gareth Cull [:garethc]]

Hey Michaela,

We don't put Google Analytics in-product so there is no way for me to measure this interaction as it stands right now. 

Also, The addons page doesn't seem to change after the user has installed an addon. I'm wondering if the site could detect and update after the user clicks the install doorhanger and we trigger another success event to GA?

We do this on marketplace and the click to successfully install rate is 72%. Wil, could you comment on this? Would this be feasable?

Comment 27

[Wil Clouser [:clouserw]]

Yes, there is a callback that we could hook into, docs at https://developer.mozilla.org/en-US/docs/Web/API/InstallTrigger/install (thanks mossop!).

Michaela - we try to avoid reopening bugs which are verified+fixed as it undoes our process with development and QA, as well as our flow for fixing bugs (all the bugs that are dependent/blocked on this bug have now been notified it has been reopened).

Can we file a new bug with the new request in it and re-close this one?  Thanks.