1025805 - Multiple referral tracking codes over time

Status: RESOLVED FIXED

(Webmaker Graveyard :: Metrics, defect, P2)

Description

[Adam Lofting (:adamlofting)]

Extend the Webmaker referrer ID... 
* https://wiki.mozilla.org/Webmaker/Maker_Party/referrer_api 

...to enable user profile history to record engagement with multiple campaigns and partnerships or events. Rather than just the new account creation stage. 

This will allow us to understand how existing/repeat Webmakers engage with campaigns like Maker Party over time, but also things like how many Makes were made at an event. This is useful for us, but also for partnership organizations and for mentors running events to see the impact of their work.

This is related to Bug 1021042 where we are planning some dashboards that will require this data.

Comment 1

[Adam Lofting (:adamlofting)]

The logic for this is something I can probably write myself. But it would depend on being able to setup a dev environment with the necessary databases etc. I will assign to myself for now to explore how feasible this is.

Comment 2

[David Ascher (:davida)]

jbuck, jp -- please make sure adam has what he needs to setup a dev environment of AllTheThings.

Comment 3

[JP Schneider :jp]

Most certainly!  Adam, which db's or sources would you need?

Comment 4

[Adam Lofting (:adamlofting)]

I'm getting my head around login.webmaker.org and the existing RIDs logic now, so I'll open a WIP pull request to check I'm taking the right approach to adding this functionality.

Comment 5

[Adam Lofting (:adamlofting)]

Attachment: https://github.com/mozilla/login.webmaker.org/pull/282

Is this initial approach for storing multiple RIDs over time suitable?

* I've added a new Sequelize model, that generates a new table.
* I'm saving the initial RID to here as well as the existing field in the User table
* This new table has a createdAt and updatedAt value so we can see longer term engagement with an existing campaign
* Where a referrer ID is being passed to login.w.o I'm updating this new table on login or authentication (so it applies to existing users as well as new users)

If this is OK so far, the next steps I think this needs are:
* Deleting a cookie after it's been used so they only get counted again if they actively visit the RID URL again (i.e. engage with that particular campaign/partner)
* Writing some tests to go with these new features
* Documentation

Comment 6

[Adam Lofting (:adamlofting)]

A general note re "AllTheThings": I'm expecting I'll get stuck when it comes to the park-warden and forestry themed bits of managing and reporting on this data. So I'll probably have some more specific setup questions then.

Comment 7

[Jon Buckley [:jbuck]]

Comment on attachment 8448777 [details] [review]
https://github.com/mozilla/login.webmaker.org/pull/282

Solid approach Adam! Before we iterate on this further, what is the method whereby a user hits a campaign URL, and then we update that information in the database? We should discuss this tomorrow

Comment 8

[Adam Lofting (:adamlofting)]

(In reply to Jon Buckley [:jbuck] from comment #7)
> Comment on attachment 8448777 [details] [review]
> https://github.com/mozilla/login.webmaker.org/pull/282
> 
> Solid approach Adam! Before we iterate on this further, what is the method
> whereby a user hits a campaign URL, and then we update that information in
> the database? We should discuss this tomorrow

This is based on the existing referrer_api model, where a URL can contain a string passed in via a parameter which gets stored in a cookie upon visiting that page. Currently when the user logs in, if they are creating a brand new account we store the RID against the account. But if they are an existing user we don't store anything. There are ways we can improve the referrer_api experience for partners using landing pages, but I think we can separate that from the implementation here.

The idea here is that existing users logging back into webmaker could also store multiple RIDs over time. If they engage in multiple events, or come to the site via multiple channels. (This will allow us to understand things like: How many users signs-up from the snippet and goes on to join a maker party 2 weeks later and six months later become a webmaker mentor).

(I'm in workshops for a couple of days now so won't get back into the details until the close of the week).

Comment 9

[Adam Lofting (:adamlofting)]

Thanks jbuck,

I've worked through the initial fixes:
https://github.com/mozilla/login.webmaker.org/pull/282/files

The next step will be adding code into webmaker-auth-client and the 'Angular log-in logic on webmaker.org' to pass the referrer cookie value through on log-in, as well as account creation. I'll work on this after Bug 1032863.

Comment 10

[Adam Lofting (:adamlofting)]

Attachment: https://github.com/mozilla/webmaker-auth/pull/14 https://github.com/mozilla/webmaker-auth-client/pull/33

Comment 11

[Jon Buckley [:jbuck]]

Attachment: https://github.com/mozilla/webmaker-auth/pull/14

Comment 12

[Jon Buckley [:jbuck]]

Attachment: https://github.com/mozilla/webmaker-auth-client/pull/33

Comment 13

[Jon Buckley [:jbuck]]

Comment on attachment 8461047 [details] [review]
https://github.com/mozilla/webmaker-auth-client/pull/33

r+ with one nit noted in the PR

Comment 14

[[github robot]]

Commits pushed to master at https://github.com/mozilla/webmaker-auth-client

https://github.com/mozilla/webmaker-auth-client/commit/cf10bf1b3ccd5b62d74dd22c33da1cfeefda3286
Part of Bug 1025805 multiple referrers over time, passes referrer id through on login and removes cookie after this has been saved.

https://github.com/mozilla/webmaker-auth-client/commit/fca1ee559d6a1850a362fb4c4d0701f58a1ee262
Merge pull request #33 from adamlofting/bug1025805

Part of Bug 1025805 - multiple referrers over time

Comment 15

[[github robot]]

Commits pushed to master at https://github.com/mozilla/webmaker-auth

https://github.com/mozilla/webmaker-auth/commit/a6c17ab44ecb1a18f20a4fcd7697385c6f0b7b17
WIP, part of Bug 1025805 - multiple referrers over time

https://github.com/mozilla/webmaker-auth/commit/48a2ceb4f1aab2710ebf5b4fd1a79e7d7b796021
Merge pull request #14 from adamlofting/bug1025805

Part of Bug 1025805 - multiple referrers over time

Comment 16

[[github robot]]

Commits pushed to master at https://github.com/mozilla/login.webmaker.org

https://github.com/mozilla/login.webmaker.org/commit/d2a26036799b14e7ab18fb9a2e02ab3f61d36c38
Part of Bug 1025805, multiple RIDs over time

https://github.com/mozilla/login.webmaker.org/commit/4cf7b2673c55e19ff00ca358f8b5799edbafa8c0
Merge pull request #282 from adamlofting/bug1025805

Part of Bug 1025805 multiple RIDs over time

Comment 17

[Adam Lofting (:adamlofting)]

These have all been merged and the version numbers bumped with mversion.

Comment 18

[[github robot]]

Commits pushed to master at https://github.com/mozilla/login.webmaker.org

https://github.com/mozilla/login.webmaker.org/commit/3304d9418359ae5bbc89c2718c7dcea20bafd46a
Revert "WIP do not merge Bug 1025805 multiple RIDs over time"

https://github.com/mozilla/login.webmaker.org/commit/147c97f6a72300e773001bab9ef8ea56071af27c
Merge pull request #289 from mozilla/revert-282-bug1025805

Revert "WIP do not merge Bug 1025805 multiple RIDs over time"

Comment 19

[Jon Buckley [:jbuck]]

Comment on attachment 8448777 [details] [review]
https://github.com/mozilla/login.webmaker.org/pull/282

:adam - This looks pretty solid, but I backed this out because there's some work we'll want to do before we push this up to production.

We'll need to add a migration for this new table, and moves the existing referral data from the User model to the ReferrerCode model.

Comment 20

[Matt Thompson (@OpenMatt :OpenMatt)]

* given that, shall we move to [aug8]?

Comment 21

[Adam Lofting (:adamlofting)]

Thanks jbuck,

1) Could you point me to an example of a previous migration script?

2) Re moving the existing referral data: there's potential value in keeping the field on the user model, but we might want to adjust naming and so on. My rationale here is:
 
* the referral data in the User Model exclusively reflects referral codes that brought *new* users into webmaker. 
* Whereas the multiple codes over time also applies to existing users. 
* In the current setup, it's easy to query for values on the user model to see which codes brought new people into webmaker. But if we remove this, we'd have to cross check the user creation date against the first referral code date in their records. 

I'm happy either way, so let me know which approach you'd prefer.

Comment 22

[Jon Buckley [:jbuck]]

We've got a bunch of migrations scripts here: https://github.com/mozilla/login.webmaker.org/tree/master/migrations

Essentially it's just some SQL to run before we update the code on each node.

I think it'd make sense to migrate the referral codes from the user model onto the Referral model. If we want to track whether it was for account creation or engagement with another campaign, lets add an additional enumerable field to the referral model that say what the referral was for.

See the sequelize docs for enums: http://sequelizejs.com/docs/1.7.8/models#block-2-line-18

Comment 23

[Adam Lofting (:adamlofting)]

Attachment: https://github.com/mozilla/login.webmaker.org/pull/291

* userStatus of 'new' or 'existing' is stored against each new RID
* I've made the naming more consistent now (just calling it a referrer code, and not sometimes a campaign code)
* Feedback from previous github annotations is included
* Migration script is in two steps 
** A to run before updating the app
** B to run after updating the app

Comment 24

[Adam Lofting (:adamlofting)]

Almost there... next train :)

Comment 25

[Jon Buckley [:jbuck]]

Comment on attachment 8466068 [details] [review]
https://github.com/mozilla/login.webmaker.org/pull/291

r+ with some nits noted in the PR

Comment 26

[Adam Lofting (:adamlofting)]

Thanks! 

Nits fixed, squashed and rebased: https://github.com/mozilla/login.webmaker.org/pull/291

What is the process for deploying to staging and prod when there are migration scripts involved?

Comment 27

[[github robot]]

Commit pushed to master at https://github.com/mozilla/login.webmaker.org

https://github.com/mozilla/login.webmaker.org/commit/b87b38ce308d84e347d1742584a0c7bd57413953
Bug 1025805 multiple RIDs over time

Comment 28

[Jon Buckley [:jbuck]]

I've deployed this to staging, but we'll need to update all of the apps with the newest version of webmaker-auth before it'll start working.

Comment 29

[Jon Buckley [:jbuck]]

:adam - I've run the create table migration on production, so feel free to push this to production on Monday. After it's been pushed to production, you'll need to push all of the other apps to production so that they start forwarding referral codes:

- events
- goggles
- login
- makeapi
- popcorn
- profile
- thimble
- webmaker.org

Comment 30

[Jon Buckley [:jbuck]]

Although, I still need to run the migration for referral codes on User to their own table.

Comment 31

[Jon Buckley [:jbuck]]

This is now deployed. Hooray!