1108769 - EOY A/B Test - PayPal only form option

Status: RESOLVED FIXED

(Mozilla Foundation :: Metrics, task)

Description

[Adam Lofting (:adamlofting)]

Test the current best performer (Sequential: https://sendto.mozilla.org/page/contribute/givenow-seq) against an even simpler, but PayPal only form like: https://sendto.mozilla.org/page/content/paypal-donate-usd/

---

Test write-up and link to live results to follow soon.

---

Usual caveat note: Variability of fundraising donation amounts can lead Optimizely to think it's found statistically significant winners or lose on a revenue per visitor basis earlier than it's really meaningful. We have to watch the revenue average lines until they stabilize, which given current traffic is likely 12-24hrs.

Comment 1

[Adam Lofting (:adamlofting)]

Just to note on this bug. We lose a % of PayPal conversion traffic who don't make it back to the thank you page (~20%), but 100% of Credit Card donations hit the thank you page.

This isn't usually a factor when comparing like-for-like donation experiences including CC and PayPal, but we will need to be considered when looking at these results, as it will bias one of the variants.

Comment 2

[Adam Lofting (:adamlofting)]

This test has been closed:
https://docs.google.com/a/mozillafoundation.org/document/d/1FAzyZvYhFkX5gSYnQQITECmelukdrHdX5VSfLfRWBs4/edit

The PayPal only form had a significant negative impact on conversion and revenue so we're sticking with the sequential form for the remainder of the campaign.