1279291 - Construct and send attribution code from download buttons

Status: RESOLVED FIXED

(www.mozilla.org :: Pages & Content, defect)

Description

[Cory Price [:ckprice] (bugmail disabled, NI me!)]

In bug 1273940 we are building a service which will accept an `attribution_code` parameter in a query string.

In bug 1259612 we detail how this code should be built based on the values in the url string.

This bug will track modifying the download button to 1st authenticate the data (bug 1278981), then send over to the stub service.

:cmore is our marketing contact here. Questions should go through him to start.

Comment 1

[Chris More [:cmore]]

I wrote some pseudocode with help of Gareth on how how the h would need to be calculated:

https://docs.google.com/a/mozilla.com/document/d/1DzIg19kAdtYEzS_waQNCQBfi8CSGj4cl9N8T24WSiyc/edit?usp=sharing

Real www.mozilla.org Firefox download source, medium, campaign content data, totals and hashes that could be constructed:

https://docs.google.com/a/mozilla.com/spreadsheets/d/1U-0JHpc3INJnBwTFkdrPqpk7yqvpEn16DJtp6d6TRiY/edit?usp=sharing

Comment 2

[Cory Price [:ckprice] (bugmail disabled, NI me!)]

New target for this is Firefox 50 (November 8th).

Comment 3

[Paul [:pmac] McLanahan]

For the service, do you really want/need it to send "(not set)" or just leave out that parameter? The service can then do what it needs to with the data sent. It just feels like a waste of bytes.

Comment 4

[Cory Price [:ckprice] (bugmail disabled, NI me!)]

Per our meeting today (https://public.etherpad-mozilla.org/p/st2u5U0HyH)

Create an on/off switch for this to be able to bypass the service and send directly to bouncer.

Nice to have: an ability to throttle usage (e.g. send only 50% of requests to the service).

Comment 5

[Alex Gibson [:agibson]]

Chris, is this use case primarily for /firefox/new/, or is the hope to have this on every download button throughout bedrock? (the latter being a bit more complicated).

It would be great to get some form of final requirements in terms of how this is supposed to work so we can get the ball rolling. I'm kind of waiting on Bug 1273940 to hopefully answer the final pieces, like what the resulting download URL's should be.

Comment 6

[Chris More [:cmore]]

(In reply to Alex Gibson [:agibson] from comment #5)
> Chris, is this use case primarily for /firefox/new/, or is the hope to have
> this on every download button throughout bedrock? (the latter being a bit
> more complicated).
> 
:agibson: Yes, primary is /firefox/new/, but since other buttons point to the second scene of /new, can the attribution still work, but we attribute the download to another page on www.mozilla.org. For example, if someone clicks on the download button on the homepage, it would be medium=referral, source=www.mozilla.org, campaign=(none), content=(none).

> It would be great to get some form of final requirements in terms of how
> this is supposed to work so we can get the ball rolling. I'm kind of waiting
> on Bug 1273940 to hopefully answer the final pieces, like what the resulting
> download URL's should be.

Initial requirements were created a year ago:

https://docs.google.com/document/d/1fKO3wpGbbr19QFvxMFC2oZ1rIrG-eGuZerxZzwBmgNg/edit#heading=h.s25t9xt9u487

Logic:

https://docs.google.com/document/d/1DzIg19kAdtYEzS_waQNCQBfi8CSGj4cl9N8T24WSiyc/edit#heading=h.883gjcln63zt

User stories on uses:

https://docs.google.com/a/mozilla.com/document/d/102y4dThQJ2DPAxI6H7M8gyT8FoR8s8ao9XdGkvV_9sU/edit?usp=sharing

Let me know if that clarifies it. Thanks!

Comment 7

[Alex Gibson [:agibson]]

Thanks Chris,

The final URLs in these documents don't seem to be a valid domain:

https://stubservice/

What should the URL's be, if these docs are in fact final?

Comment 8

[Chris More [:cmore]]

(In reply to Alex Gibson [:agibson] from comment #7)
> Thanks Chris,
> 
> The final URLs in these documents don't seem to be a valid domain:
> 
> https://stubservice/
> 
> What should the URL's be, if these docs are in fact final?

:agibson:

I don't know what URL that is going to be. It was just an example in my code since it wad before anything was created. 

Also, here's an example tagging matrix with the most common tags: https://docs.google.com/a/mozilla.com/spreadsheets/d/1U-0JHpc3INJnBwTFkdrPqpk7yqvpEn16DJtp6d6TRiY/edit?usp=sharing

Just to note, if 95% of the time, there are no UTM tags, thus it needs to be figured out via the referrer, which is how GA does it.

:ckprice: do you know when the final URL of the stub service will be known?

Comment 9

[Alex Gibson [:agibson]]

(In reply to Chris More [:cmore] from comment #6)
> :agibson: Yes, primary is /firefox/new/, but since other buttons point to
> the second scene of /new, can the attribution still work, but we attribute
> the download to another page on www.mozilla.org. For example, if someone
> clicks on the download button on the homepage, it would be medium=referral,
> source=www.mozilla.org, campaign=(none), content=(none).

In order to accurately capture referral traffic, we're going to need to do the stub attribution logic on scene1 of /firefox/new/, before the user hits scene2, and pass this through to the URL on scene2 where it can be added to the download URL. This also applies to other buttons on mozorg that point to /firefox/new/?scene=2.

With the logic you suggest above though, this now implies that all downloads via mozorg pages should be attributed as a referral and not direct traffic. Is this correct? The logic document shows a final conditional for direct traffic, so I'd like to be clear here on what you're asking.

Thanks

Comment 10

[Chris More [:cmore]]

(In reply to Alex Gibson [:agibson] from comment #9)
> (In reply to Chris More [:cmore] from comment #6)
> > :agibson: Yes, primary is /firefox/new/, but since other buttons point to
> > the second scene of /new, can the attribution still work, but we attribute
> > the download to another page on www.mozilla.org. For example, if someone
> > clicks on the download button on the homepage, it would be medium=referral,
> > source=www.mozilla.org, campaign=(none), content=(none).
> 
> In order to accurately capture referral traffic, we're going to need to do
> the stub attribution logic on scene1 of /firefox/new/, before the user hits
> scene2, and pass this through to the URL on scene2 where it can be added to
> the download URL. This also applies to other buttons on mozorg that point to
> /firefox/new/?scene=2.

Hmmm. If we do it only on scene 1 and just pass it to scene 2, then anything pointing to scene 2 will not get attribution. Given that, what are your thoughts on only doing it on scene 2 and not passing it in the URL when clicking the button on scene 1?

> 
> With the logic you suggest above though, this now implies that all downloads
> via mozorg pages should be attributed as a referral and not direct traffic.
> Is this correct? The logic document shows a final conditional for direct
> traffic, so I'd like to be clear here on what you're asking.
> 
> Thanks

Good point. Most of the logic was assuming that /firefox/new/ was the first page hit on www.mozilla.org and I was ignoring other pages on www.mozilla.org directing traffic to /firefox/new/. Luckily most traffic to /firefox/new/ originates from external to the site, but all the other pages on www.mozilla.org do contribute some of the overall downloads. 

I just updated the comment here (https://docs.google.com/document/d/1DzIg19kAdtYEzS_waQNCQBfi8CSGj4cl9N8T24WSiyc/edit#) to mention in the referral section " If UTM parameters don't exist and they came from another website or another page on www.mozilla.org, use that referring website as the source"

Basically, if there UTM parameters available, use them, if not check to see if HTTP.referer is defined. If HTTP referer is defined, parse the strong to set the source and set the medium to referral. If HTTP.referer is not defined, then set medium=(direct), source=(none), campaign=(none), and content=(none).

So, if when a user go click "free download on the www.mozilla.org homepage and get directed to /firefox/new/ and if http.referer is not defined, then it is direct traffic. The http.referer may get stripped because of bug 1008390 not got resolved.

Comment 11

[Alex Gibson [:agibson]]

(In reply to Chris More [:cmore] from comment #10)
> Hmmm. If we do it only on scene 1 and just pass it to scene 2, then anything
> pointing to scene 2 will not get attribution. Given that, what are your
> thoughts on only doing it on scene 2 and not passing it in the URL when
> clicking the button on scene 1?

There are several reasons why we want to do this before the user hits scene2:

1.) Referral data.
2.) For some browsers (old IE) we trigger the download before the user hits scene 2.
3.) Putting the ajax call before the download initiates on scene2 would create a hard dependency on the ajax call succeeding, and also delays the download starting until the ajax call finishes.
4.) Triggering downloads automatically via an async ajax call could potentially cause issues in certain browsers blocking the download.

Putting the ajax call on scene1 (or any mozorg page with a dl button pointing to scene2) would cover mozorg pretty well. We don't really advise linking directly to scene2 from external sites, as it circumvents a lot of the logic we have in place.

> Good point. Most of the logic was assuming that /firefox/new/ was the first
> page hit on www.mozilla.org and I was ignoring other pages on
> www.mozilla.org directing traffic to /firefox/new/. Luckily most traffic to
> /firefox/new/ originates from external to the site, but all the other pages
> on www.mozilla.org do contribute some of the overall downloads. 
> 
> I just updated the comment here
> (https://docs.google.com/document/d/
> 1DzIg19kAdtYEzS_waQNCQBfi8CSGj4cl9N8T24WSiyc/edit#) to mention in the
> referral section " If UTM parameters don't exist and they came from another
> website or another page on www.mozilla.org, use that referring website as
> the source"
> 
> Basically, if there UTM parameters available, use them, if not check to see
> if HTTP.referer is defined. If HTTP referer is defined, parse the strong to
> set the source and set the medium to referral. If HTTP.referer is not
> defined, then set medium=(direct), source=(none), campaign=(none), and
> content=(none).
> 
> So, if when a user go click "free download on the www.mozilla.org homepage
> and get directed to /firefox/new/ and if http.referer is not defined, then
> it is direct traffic. The http.referer may get stripped because of bug
> 1008390 not got resolved.

When capturing the source for mozorg should it always just be the TDL, i.e. www.mozilla.org, or do you want to start making exceptions to the referrer logic and start capturing the actual page they came from?

Comment 12

[Chris More [:cmore]]

> When capturing the source for mozorg should it always just be the TDL, i.e. www.mozilla.org, or do you want to start making exceptions to the referrer logic and start capturing the actual page they came from?

:agibson: the source should always be the TLD as we don't need the specific pages. We already have the specific pages in GA and the root domain is suffice for understand product retention. We don't need to be any more granular than that.

Comment 13

[Alex Gibson [:agibson]]

(In reply to Chris More [:cmore] from comment #12)
> > When capturing the source for mozorg should it always just be the TDL, i.e. www.mozilla.org, or do you want to start making exceptions to the referrer logic and start capturing the actual page they came from?
> 
> :agibson: the source should always be the TLD as we don't need the specific
> pages. We already have the specific pages in GA and the root domain is
> suffice for understand product retention. We don't need to be any more
> granular than that.

Great, thanks for the clarification

Comment 14

[Paul [:pmac] McLanahan]

Alex: you don't need to worry about it though, you can just send the referrer info to the service and it will send back just the domain. It needs to do that regardless.

Comment 15

[Alex Gibson [:agibson]]

(In reply to Paul [:pmac] McLanahan from comment #14)
> Alex: you don't need to worry about it though, you can just send the
> referrer info to the service and it will send back just the domain. It needs
> to do that regardless.

Good to know, thanks.

Comment 16

[Alex Gibson [:agibson]]

Chris, :pmac mentions that we also no longer need to do the logic part that decides if the referrer was a search engine or not. If this is correct, can you please update your logic doc to match whatever the final requirements should be? Thanks.

Comment 17

[Alex Gibson [:agibson]]

Ok, after talking to :pmac I now understand the client needs to pretty much ignore most of the logic in the doc Chris links to, since most of it will be handled in the ajax service. All the client needs to do is pass 5 values (the UTM params and the referrer), and then construct the final URL based on the signed param returned.

I still need to find out what the final URLs should be to take this forward.

Comment 18

[Cory Price [:ckprice] (bugmail disabled, NI me!)]

Here is the suggestion from oremj bug1273940comment22

Any reason this wouldn't work?

Comment 19

[Alex Gibson [:agibson]]

(In reply to Cory Price [:ckprice] from comment #18)
> Here is the suggestion from oremj bug1273940comment22
> 
> Any reason this wouldn't work?

Does this imply that a final constructed URL would be:

https://stubdownloader.prod.mozaws.net/?os=win&lang=en-US&product=Firefox-stub-46.0&attribution_code=source%3Dgoogle.com%26medium%3Dorganic%26campaign%3D(not%20set)%26content%3D(not%20set)

Having to pull in all the os/lang/product info all over again isn't ideal, but if this what we're going to end up using we can make it work. It would also be great to be able to test these URLs in some way instead of hoping they are working correctly? Do we have any idea when bug 1273940 will be complete?

Comment 20

[Cory Price [:ckprice] (bugmail disabled, NI me!)]

I confirmed in bug1273940comment23 that this is the final URL.

(In reply to Alex Gibson [:agibson] from comment #19)
> Does this imply that a final constructed URL would be:
> 
> https://stubdownloader.prod.mozaws.net/?os=win&lang=en-US&product=Firefox-
> stub-46.0&attribution_code=source%3Dgoogle.
> com%26medium%3Dorganic%26campaign%3D(not%20set)%26content%3D(not%20set)
> 
> Having to pull in all the os/lang/product info all over again isn't ideal,
> but if this what we're going to end up using we can make it work. It would
> also be great to be able to test these URLs in some way instead of hoping
> they are working correctly? Do we have any idea when bug 1273940 will be
> complete?
NI :oremj to answer this bit.

Comment 21

[Paul [:pmac] McLanahan]

The proper solution is to have bouncer see that an "attribution_code" param has been set and do the appropriate thing, be that a redirect or just doing the thing. Was this considered and rejected for some reason? Like agibson said, replacing the entire URL will be a bit more squirly.

Also, the final URL above doesn't contain the HMAC signature. Can we just add it in a "signature" parameter, or would you prefer something else? And, like I said before and we decided in the meetings, we're not doing that logic doc, so the URL won't contain any "organic" or "(not set)" stuff. It'll just be any of the values that we were able to gather and nothing more.

Comment 22

[Alex Gibson [:agibson]]

+1 to what :pmac says here. I would very much prefer to let bouncer handle the redirect if it detects the attribution code, as opposed to mozorg needing to rewrite & replace all the URL's.

Comment 23

[Jeremy Orem [:oremj]]

(In reply to Alex Gibson [:agibson] from comment #19)
> (In reply to Cory Price [:ckprice] from comment #18)
> > Here is the suggestion from oremj bug1273940comment22
> > 
> > Any reason this wouldn't work?
> 
> Does this imply that a final constructed URL would be:
> 
> https://stubdownloader.prod.mozaws.net/?os=win&lang=en-US&product=Firefox-
> stub-46.0&attribution_code=source%3Dgoogle.
> com%26medium%3Dorganic%26campaign%3D(not%20set)%26content%3D(not%20set)
> 
> Having to pull in all the os/lang/product info all over again isn't ideal,
> but if this what we're going to end up using we can make it work. It would
> also be great to be able to test these URLs in some way instead of hoping
> they are working correctly? Do we have any idea when bug 1273940 will be
> complete?

We need the os/lang/product to know what to serve back to the user. Bug 1273940 is waiting for an available testable stub installer.

Comment 24

[Jeremy Orem [:oremj]]

Alternatively, we could pin the product to Firefox-Stub-Latest and the os to win.

Comment 25

[Jeremy Orem [:oremj]]

(In reply to Paul [:pmac] McLanahan from comment #21)
> The proper solution is to have bouncer see that an "attribution_code" param
> has been set and do the appropriate thing, be that a redirect or just doing
> the thing. Was this considered and rejected for some reason? Like agibson
> said, replacing the entire URL will be a bit more squirly.
> 
> Also, the final URL above doesn't contain the HMAC signature. Can we just
> add it in a "signature" parameter, or would you prefer something else? And,
> like I said before and we decided in the meetings, we're not doing that
> logic doc, so the URL won't contain any "organic" or "(not set)" stuff.
> It'll just be any of the values that we were able to gather and nothing more.

We were, originally, trying to avoid modifying bouncer. I suppose we could modify it to redirect to the stub service if attribution_code is set, otherwise, proceed as usual.

Comment 26

[Paul [:pmac] McLanahan]

(In reply to Jeremy Orem [:oremj] from comment #25)
> We were, originally, trying to avoid modifying bouncer. I suppose we could
> modify it to redirect to the stub service if attribution_code is set,
> otherwise, proceed as usual.

That would certainly make the code on bedrock a lot more simple and reliable. It would also potentially allow us to ensure that we don't send any users of winxp or other old OSes over to the new thing if they can't use it because of either a newer SSL cert or newer software signing cert.

Comment 27

[Alex Gibson [:agibson]]

Attachment: GitHub pull request

Comment 28

[Alex Gibson [:agibson]]

Some additional criteria the bedrock client side code will check for before making a request to authenticate data with the stub service added in Bug 1278981:

- User is on Windows (since only windows users get the stub installer).
- User's browser does not need sha-1 bouncer.
- User does not have DNT enabled.
- User falls within a predefined sample rate limit (set via an env variable). See https://bugzilla.mozilla.org/show_bug.cgi?id=1278981#c6

Comment 29

[Alex Gibson [:agibson]]

This is now up on demo4 for initial testing based on the criteria above in Comment 28:

https://www-demo4.allizom.org/en-US/

Once attribution data has been signed and encoded via the bedrock service, it is then stored in a session cookie and used to update the Bouncer links (for windows only) with the appropriate query params. Things seem to be working as expected on the bedrock side, but requires further testing & QA.

The attribution sample rate on demo4 is currently set to 1.0, meaning a request will be made to the service 100% of the time as long utm / referrer requirements are satisfied.

Comment 30

[Alex Gibson [:agibson]]

I've configured demo4 (linked above) to use https://bouncer-bouncer.stage.mozaws.net/.

If this is not the correct staging URL for Bouncer, please let me know.

Comment 31

[Stephen Donner [:stephend] Not actively reading bugmail]

:oremj / :pmac - will we need https://github.com/mozilla-services/go-bouncer/pull/36 merged before comment 30 can be tested, integration-wise, so the redirection from Mozilla.org -> Bouncer -> Stub Attribution server works?

Comment 32

[Paul [:pmac] McLanahan]

:oremj would know for sure, but it looks to me like the answer is "yes".

Comment 33

[Cory Price [:ckprice] (bugmail disabled, NI me!)]

Jeremy confirms that yes it is required, and it's merged (still has to deploy to stage).

Comment 34

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to Cory Price [:ckprice] from comment #33)
> Jeremy confirms that yes it is required, and it's merged (still has to
> deploy to stage).

Jeremy, can you confirm if this is deployed to https://stubattribution-default.stage.mozaws.net/ , and if not, mind doing so, so I can start testing?  Thanks!

Comment 35

[Jeremy Orem [:oremj]]

https://github.com/mozilla-services/go-bouncer/pull/36 has been deployed to stage bouncer: https://bouncer-bouncer.stage.mozaws.net/

Comment 36

[Alex Gibson [:agibson]]

Stephen,

Small update on the mozorg side for this - we're now using a cookie with an expiry time set to 24 hours (we we're previously using a session cookie, which could run on too long if the user doesn't restart their browser often enough). So to see new cookie data now you may need to clear cookies, as opposed to just restarting the browser.

This has been updated on demo4.

Thanks

Comment 37

[Stephen Donner [:stephend] Not actively reading bugmail]

Even though I'm hitting all of the conditions in https://bugzilla.mozilla.org/show_bug.cgi?id=1279291#c28, and I can see the stub-attribution service JS bundle being loaded/used, I'm still being returned "?product=firefox-stub" rather than the "test-stub" product set up for temporary testing, over in bug 1318456.

need-info?-ing :agibson just for extra visibility; in our IRC conversation, looks like a part of Bedrock wasn't aware of that specific product value/test file name.

Comment 38

[Paul [:pmac] McLanahan]

I'll briefly say what I said in the email chain: Hopefully we can use the real product names. Bedrock's button code is complex and was never setup to use different product names since a problem there could break downloads on the site. And if we have to customize the buttons just for the test, then we'll be testing this customization and not the real button code. This custom button code will need to be thrown out after the test, so it won't be a valid full test. I suggest that we do one of the following in order of preference:

1) Change the product for the test to match the names in prod bouncer; the names as they'll be for real.

2) Test by going to demo-4 as it is now, copying the bouncer-stage url, change the product name manually to "test-stub" then manually going to the modified bouncer URL.

Option #2 isn't optimal but I feel it's better than spending time on code that will never go to prod and which will make the tests of little value since it'll be very different when it's deployed to production.

Comment 39

[Cory Price [:ckprice] (bugmail disabled, NI me!)]

The `test-stub` name was used as a flag to be able to serve this test build (ref: bug 1318456 comment 12) since default build won't be ready until bug 1320773 is resolved.

As this is a project with several moving parts, I'm hopeful we can make some concessions along the way outside of an optimal real life testing environment (e.g. a variation of option 2 above).

Comment 40

[[github robot]]

Commits pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/4eb1c878777fb1a450151125154e55c74903d59d
[fix bug 1279291] Construct and send attribution code from download buttons

https://github.com/mozilla/bedrock/commit/04af7aea1ee8ec7018f41aab90d750d4d7315e69
Merge pull request #4456 from alexgibson/bug-1279291-stub-attribution-client-js

[fix bug 1279291] Construct and send attribution code from download buttons

Comment 41

[[github robot]]

Commits pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/4fd02108adc35506ce7859fe17e9fd23ba5bc491
[bug 1279291] Default to stub attribution rate of 1 for DEV only

https://github.com/mozilla/bedrock/commit/bf913b604c25d174e046d032fb3b7f95e3a77028
Merge pull request #4624 from alexgibson/stub-attribution-rate-default

[bug 1279291] Default to stub attribution rate of 1 for DEV only