846756 - a/b test positioning of products on start page

Status: RESOLVED FIXED

(support.mozilla.org :: Knowledge Base Software, task, P3)

Description

[Kadir Topal [:atopal]]

We want to know if people can distinguish Firefox for Desktop, for Android, and Firefox OS even though they are using the same icon, and choose the right product on the start page. If they can, then changing the position of the products should have no effect on click throughs, if they can't there should be a pattern (like people clicking whatever is first in the list)

If it turns out that they can't distinguish those products, we need to work with marketing and creative to find a solution to the problem of similar names and identical icons.

For the first test, let's swap the position of Firefox for Desktop and Firefox for Android.

Comment 1

[Ricky Rosario [:rrosario, :r1cky]]

This seems like simple A/B test. See Bug 830753 for reference. That one was 2pts because we didn't know what we were doing back then.

For this, we need Ibai to create the content experiment in google analytics for us.

Comment 2

[Ibai Garcia [:ibai]]

We need to define what's the goal of this. Is the amount of clicks to desktop? 

After that, I need the URL to the variation/s and I can generate the experiment.

Comment 3

[Ricky Rosario [:rrosario, :r1cky]]

(In reply to Ibai Garcia [:ibai] from comment #2)
> After that, I need the URL to the variation/s and I can generate the
> experiment.

Currently, https://support.mozilla.org/en-US/products shows:
Fx - Android - FxOS - Thunderbird

How about we add https://support.mozilla.org/en-US/products?b that shows:
Android - Fx - FxOS - Thunderbird

Is that enough or do we need more combinations?

Comment 4

[Ibai Garcia [:ibai]]

We can make it work with it!

I'll put together the experiment and share the code this afternoon.

Comment 5

[Ibai Garcia [:ibai]]

Ricky, you have mail

https://www.google.com:443/analytics/web/siteopt/signed/minisite?experimentId=Ir4RfSQ3Q0yZ5QC6zUTJYg&webProperty=UA-36116321-2&signature=AFB3sia04TNwhNpBCLhaGZF-wcb8olBTB0dPqHFyYm2xJE89SMYzcd24hEHc2iHK2GDeacNdSb38zaUL5jkvzoEBFNavKK-yP6dSTnlVHm8MoDrjTeJ00_LR04WTax5Rxv3mdmlJbCi2&hl=en_US

Comment 6

[Ricky Rosario [:rrosario, :r1cky]]

In a pull requesto:
https://github.com/mozilla/kitsune/pull/1192

Comment 7

[Ricky Rosario [:rrosario, :r1cky]]

On master:
https://github.com/mozilla/kitsune/commit/19ad3927f55c64daa8a681aa31e352c5848a408e

Comment 8

[Ricky Rosario [:rrosario, :r1cky]]

Deployed to prod now.

Comment 9

[:ibai]

And the experiment is active.