1108987 - EOY A/B Test - Firefox for Android Simple PayPal option

Status: RESOLVED FIXED

(Mozilla Foundation :: Metrics, task)

Description

[Adam Lofting (:adamlofting)]

Test write-up: 
** https://docs.google.com/a/mozillafoundation.org/document/d/1AMNsL42zXwli824Ib-Bi4wRqsTLFDvMdzw6ipLb05R8/edit

Live results:
** https://www.optimizely.com/results2?experiment_id=2204493974#view=2

---

Special note for this test:
  We lose 20% of PayPal conversion tracking, so we will need to adjust the test results to compensate for this.

Usual caveat note: Variability of fundraising donation amounts can lead Optimizely to think it's found statistically significant winners or lose on a revenue per visitor basis earlier than it's really meaningful. We have to watch the revenue average lines until they stabilize, which given current traffic is likely 12-24hrs.

Comment 1

[Adam Lofting (:adamlofting)]

The test variation on this has beaten the control, but it is not significant in relation to the overall campaign. We should roll-out the winner, but prioritize work the desktop snippet for the remainder of the campaign.

https://docs.google.com/a/mozillafoundation.org/document/d/1AMNsL42zXwli824Ib-Bi4wRqsTLFDvMdzw6ipLb05R8/edit