649756 - A/B testing of Join Mozilla promos on Mozilla.org

Status: RESOLVED INCOMPLETE

(www.mozilla.org :: General, defect)

Description

[Chelsea Novak [:chelsea]]

Bug for discussion of the A/B testing of the three Join Mozilla promos.

1) Promo 1 - No price point
2) Promo 2 - Price point
3) Promo 3 - Community angle

Would like to track clicks and conversions on this page if possible: 
https://wiki.mozilla.org/JoinMozilla/Creative/Mozilla.orgPromos

Comment 1

[David Boswell]

The promos have been added to the home page and About page and promo 1 is now live.  Here are instructions for rotating the promos and performing the A/B test.

The three different images are at

http://www.mozilla.org/images/promos/join_promo_a.png
http://www.mozilla.org/images/promos/join_promo_b.png
http://www.mozilla.org/images/promos/join_promo_c.png

To swap them out on the pages simply reference a different image in the image link.

To let WebTrends know that it is now tracking a different promo we need to update the WT.ad (in the head) and WT.ac (in the promo link) tags on both pages.  The values for these need to be the same.

The tags should be structured like the following:

* join_h_a (says this is the join promo A on home page)
* join_h_b (says this is the join promo B on home page)
* join_h_c (says this is the join promo C on home page)

* join_a_a (says this is the join promo A on about page)
* join_a_b (says this is the join promo B on about page)
* join_a_c (says this is the join promo C on about page)

So when you update the image tag also update the two WebTrends tags on the page to correspond.

Getting the data out of WebTrends is going to be a little trickier than usual because of the l10n redirecting we're doing at mozilla.org/join.  Normally there is a clickthrough rate report that automatically generates the data.

To count a clickthrough though WebTrends is expecting the page someone lands on to be the page referenced in the link and in this case it isn't because of the redirect.  There may be a way to get around this in WebTrends or restructure the redirect so it registers that someone hit that page before sending them on.

As a short-term solution we can look at referrer reports for the transaction page and see what traffic is coming from the home page and About page and see how that changes as the promos themselves change.

I think there is a better solution for this but it might involve getting on the phone with WebTrends support and asking them about what to do in this case.  Happy to talk through more of the options or to answer questions about any of this.

Comment 2

[Chelsea Novak [:chelsea]]

Hey Ryan and Ben, 
I'm not sure where things stand with Join dev, but can you let me know if these bugs need to stay open?

Comment 3

[Ben Simon]

Did the A/B/C testing that was requested here ever actually get implemented?

Comment 4

[Ryan Merkley]

Closing this bug out. If the a/b testing happened, it's no longer relevant anyway.