1262004 - Create A/B test for new space theme /firstrun/ page

Status: RESOLVED FIXED

(www.mozilla.org :: Pages & Content, defect)

Description

[Chris More [:cmore]]

Hypothesis: Users who see the space theme sign up page will convert to signing for a Firefox Account at the same rate as the current first run page.
Goal: Fxa Sign up rate
Audience: Firefox desktop
 
Variations: 

Original: https://www.mozilla.org/en-US/firefox/45.0.1/firstrun/?v=0
Space theme: https://www.mozilla.org/en-US/firefox/45.0.1/firstrun/?v=1

Mana page and Optimizely experiment coming soon.

Comment 1

[Chris More [:cmore]]

Mana page: https://mana.mozilla.org/wiki/pages/viewpage.action?pageId=60588239

Comment 2

[Chris More [:cmore]]

Experiment: https://app.optimizely.com/edit?experiment_id=5503693130

Comment 3

[Chris More [:cmore]]

:jpetto:

I have the experiment created. I was getting a JS error in the experiment JS section, but then I realized the HTML changed for the iframe.

It was:

var fxaIframeSrc = $("#intro").data("fxa-iframe-src");

and I changed it to:

var fxaIframeSrc = $("#intro").data("fxa-iframe-host");

The experiment may seem like a weird setup, but it is what I figured out at Mozlando due to custom events not being able to override the target URL. There is a traffic cop page that is just a redirect to either v=0 or v=1 and then the other variation have no code so we don't create an infinite loop. The redirect only happens on the traffic cop page and the traffic cop page only matches the first run URL with no parameters.

Check it out on comment 2 and let me know if I have the green light. If so, flip on the /firstrun/ flag.

Comment 4

[Jonathan Petto [:jpetto]]

After being led through the circuitous logic necessary to make Optimizely correctly track the iframe event, the experiment looks good to me.

r+

Comment 5

[Chris More [:cmore]]

Test started at 11am PST on 4/5/2016. Conversions are coming in fine.

Comment 6

[Chris More [:cmore]]

Test stopped on 4/11/2011: Results here: https://mana.mozilla.org/wiki/pages/viewpage.action?pageId=60588239