Closed Bug 1317751 Opened 8 years ago Closed 6 years ago

Optimizely Test for Funnelcake 91/92 (Yahoo) & 93/94 (Onboarding v1.5) experiments

Categories

(www.mozilla.org :: Analytics, defect)

Production
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: cmore, Unassigned)

References

Details

Audience: US + English + Non-Firefox + Desktop

Goal: Engagement (not really needed)

Variations:

Onboarding v1.5 control (93)
Onboarding v1.5 variation (94)
Yahoo control (91)
Yahoo variation (92)

Traffic allocation: 80% without Yahoo running, 100% with Yahoo running.

Start: 11/22/2016
End: 12/12/2016

Test created here:

https://app.optimizely.com/edit?experiment_id=7858290010
:jpetto: r?
Flags: needinfo?(jon)
Code looks good.

Noting that Yahoo and Yahoo control currently have 0% traffic, and that funnelcake URLs are not yet active.
Flags: needinfo?(jon)
Funnelcake 93 and 94 are ready to be launched. Yahoo funnelcakes will not be started until Nov 30th.

agibson/jpetto/craigcook: I need to enable the funnelcake test by this Saturday the 26th. This has been the planned funnelcake with the known date of this week for the past 2 months and I just realized that Optimizely is not enabled on /firefox/new/.

Can one of you enable it on the page, so we can start the test?
Flags: needinfo?(jon)
Flags: needinfo?(craigcook.bugz)
Flags: needinfo?(agibson)
(In reply to Chris More [:cmore] from comment #4)
> Funnelcake 93 and 94 are ready to be launched. Yahoo funnelcakes will not be
> started until Nov 30th.
> 
> agibson/jpetto/craigcook: I need to enable the funnelcake test by this
> Saturday the 26th. This has been the planned funnelcake with the known date
> of this week for the past 2 months and I just realized that Optimizely is
> not enabled on /firefox/new/.
> 
> Can one of you enable it on the page, so we can start the test?

SWITCH_FIREFOX_NEW_OPTIMIZELY is now set to `on` in prod for both us and eu clusters. Double checked that Traffic Cop is disabled on the page.

Chris, please note the Yahoo search PR [1] has not yet been reviewed/pushed to production. I'll see if we can make this happen today, assuming we can cut through the many layers of red approval tape.

[1] https://github.com/mozilla/bedrock/pull/4466
Flags: needinfo?(jon)
Flags: needinfo?(craigcook.bugz)
Flags: needinfo?(agibson)
Hi Chris,

Please note that the Yahoo page has not been merged yet. We're holding off on this as it's not clear if we have Yahoo's final approval yet. Last I heard we (Mozilla) have signed off on it, but they (Yahoo) still needed to see it on a demo. I haven't heard anything back on this yet. I don't want to hold up your onboarding test, but it's doubtful that the Yahoo page will be ready until we have confirmation on Monday.
Flags: needinfo?(chrismore.bugzilla)
Depends on: 1313708
(In reply to Alex Gibson [:agibson] from comment #6)
> Hi Chris,
> 
> Please note that the Yahoo page has not been merged yet. We're holding off
> on this as it's not clear if we have Yahoo's final approval yet. Last I
> heard we (Mozilla) have signed off on it, but they (Yahoo) still needed to
> see it on a demo. I haven't heard anything back on this yet. I don't want to
> hold up your onboarding test, but it's doubtful that the Yahoo page will be
> ready until we have confirmation on Monday.

Thanks, Alex. The Yahoo variations are disabled in the Optimizely experiment. This request is only for the Onboarding funnelcakes, but we are using the same experiment because the Yahoo experiment should start next week.

Thanks
Flags: needinfo?(chrismore.bugzilla)
This is now live on production - Chris, please feel free to enable the experiment when ready. Thanks

https://www.mozilla.org/en-US/firefox/50.0/firstrun/?f=92
Flags: needinfo?(chrismore.bugzilla)
Launched on 11/30/2016 at 8:49am PST.
Flags: needinfo?(chrismore.bugzilla)
This looks complete.
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.