1109628 - EOY A/B Test: Removing Logos

Status: RESOLVED FIXED

(Mozilla Foundation :: Metrics, task)

Description

[Adam Lofting (:adamlofting)]

Test write-up is here:
- https://docs.google.com/a/mozillafoundation.org/document/d/1omo0-jm8h6TboenYV8rja6iFvgqIjuOJlvtO4nvgpqg/edit


Live results can be followed here:
- http://optimize.ly/~RTj6hF?token=c012ef6ee9dc8ab7f921#view=2


Usual caveat note: Variability of fundraising donation amounts can lead Optimizely to think it's found statistically significant winners or lose on a revenue per visitor basis earlier than it's really meaningful. We have to watch the revenue average lines until they stabilize, which given current traffic is likely 12-24hrs.

Comment 1

[Adam Lofting (:adamlofting)]

This test has finished:
https://docs.google.com/a/mozillafoundation.org/document/d/1omo0-jm8h6TboenYV8rja6iFvgqIjuOJlvtO4nvgpqg/edit

"Removing either the logo or Tabzilla has a positive effect, but the strongest effect is seen when removing Tabzilla. A 3.2% increase in conversion rate and a 3.4% increase in dollar per visit."

I'll file bugs to implement the winning changes now.