Closed Bug 966188 Opened 10 years ago Closed 10 years ago

Set up an A/B test for Webmaker.org

Categories

(Webmaker Graveyard :: Metrics, defect)

x86
macOS
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: adam, Unassigned)

References

Details

(Whiteboard: [workweek][workweek-metrics])

Generate ideas for things we could test.

Use page traffic and number of variations to estimate time to complete.

Setup a test in Optimizely (aim for something that can conclude this week).

Corresponding Etherpad:
https://etherpad.mozilla.org/webmaker-workweek-metrics-testing
Assignee: adam → nobody
@ Adam: let me know how I can help with that front page test you mentioned?
Flags: needinfo?(adam)
Here is an example of the preview link that comes out of Optimizely:

http://optimize.ly/~1VBjSR
Flags: needinfo?(adam)
* To be clear: this test is now running on the live site! We're testing three different variables, in 20 different combinations: headline text, image, and call to action buttons. 

* The optimize.ly link will show you all the different combinations now being tested. 
http://optimize.ly/~1VBjSR
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
* Looks like we may have a small bug with the way the call to action buttons are appearing:
https://dl.dropboxusercontent.com/spa/6c38yp3crbxni5b/k06s5_jq.png

* We also have better photos available for testing. Like for example:
https://webmaker.org/img/front-page/india-story.jpg
Or various others available here: http://www.flickr.com/photos/mozilladrumbeat/favorites
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
@ Adam: how do we see the results?
* The bug is only in the Optimizely preview, so we're going to keep an eye on it for now, but it's displaying OK on live.

* For images, the drumbeat stuff is great, but we didn't have capacity to build in image attribution for today (and the pictures we wanted were using a restricted license) - so we're using one of Brett's images to get the test up and running.

* Even if this photo does well in the test, it should be followed up with a longer test using multiple images. I think we leave what we have in place for the workweek, and iterate over the next few weeks.

* Results are locked down to authorized accounts in Optimizely, so we'll think about this in our processes. Also, all results are "top-secret" until work-week attendees have placed their bets on the winning combination ;)
Status: REOPENED → RESOLVED
Closed: 10 years ago10 years ago
Resolution: --- → FIXED
Nice. Thanks Adam.
You need to log in before you can comment on or make changes to this bug.