1348129 - Creating experiment to distribute the Tour Notification Funnelcake Builds

Status: RESOLVED FIXED

(Firefox :: General, enhancement)

Comment 1

[Nicole Yee (:nicoleyee)]

Hey Jcrawford and Cmore,

We are ready to create the optimizely test for the tour notification funnelcake. Would be good to  note that we still plan on shipping on the 20th.

Comment 2

[Chris More [:cmore]]

Ok, created the Optimizely experiment here:

https://app.optimizely.com/edit?experiment_id=8334713409

A few things we need to confirm:

1) is 25% split among all 4 variations correct?

2) What is the sample size that we should be targeting for non-Firefox + English + Windows visitors to /firefox/new/? I have it set to 20% now.

3) Right now, we are targeting Windows and the funnelcakes are win32, which means if a win64 user gets to the page, we will be giving them a win32 funnelcake unless they click the "firefox for other platforms" link that has a win64 option (/firefox/all/).

:pdol: can you help with these 3 questions?

:jpetto: while we are getting pdol to help answer these 3 questions above, you could do a sweep of the optimizely experiment for these 4 funnelcakes to see if you find anything wrong in the meantime.

Comment 3

[Peter Dolanjski [:pdol]]

1) Yes, I think that's fine
2) That also sounds fine
3) That's fine.  We're not shipping by default until v55.

Comment 4

[Chris More [:cmore]]

:jpetto: we are ready for a code review here. The typical funnelcake deployment test. The Firefox team is targeting May 20th to be abled if QA gets signed off. If you want to build this same thing with TrafficCop, you are free to do.

Target audience: non-Firefox + Desktop + English languages

Variations: 4 variations that redirect to their associated ?f=XXX funnelcake ID:

IDs and variations:

110	Control, vanilla Firefox 53 with just the configs to denote the funnelcake
111	Streamlined stub installer (bugs 1333873, 1350405) + Automigration (bugs 1271775, 1271799, + fixes under 1332225) + tour notifications variant A
112	Streamlined stub installer (bugs 1333873, 1350405) + Automigration (bugs 1271775, 1271799, + fixes under 1332225) + tour notifications variant B
113	Streamlined stub installer (bugs 1333873, 1350405) + without automigration + tour notifications variant A

Sample: 30% of target audience at a 25% split between each 4 variation.

Comment 5

[Jonathan Petto [:jpetto]]

If the scheduled start of the experiment is 5/20, we should have time to set this up using TrafficCop.

For the audience segmentation, what exactly does "Desktop" mean when applied in Optimizely? Is that screen size, not a known mobile browser, a combination, or something else?

Comment 6

[Chris More [:cmore]]

(In reply to Jon Petto [:jpetto] from comment #5)
> If the scheduled start of the experiment is 5/20, we should have time to set
> this up using TrafficCop.
> 
> For the audience segmentation, what exactly does "Desktop" mean when applied
> in Optimizely? Is that screen size, not a known mobile browser, a
> combination, or something else?

Desktop is determined via the Operating System via the user agent and it is fairly reliable given that the user agents are pretty distinct between desktop and mobile. 

Example:

var ua = navigator.userAgent.toLowerCase();
var isDesktop = !(/(android|bb\d+|meego).+mobile|avantgo|bada\/|blackberry|blazer|compal|elaine|fennec|hiptop|iemobile|ip(hone|od)|iris|kindle|lge |maemo|midp|mmp|netfront|opera m(ob|in)i|palm( os)?|phone|p(ixi|re)\/|plucker|pocket|psp|series(4|6)0|symbian|treo|up\.(browser|link)|vodafone|wap|windows (ce|phone)|xda|xiino/i.test(a)||/1207|6310|6590|3gso|4thp|50[1-6]i|770s|802s|a wa|abac|ac(er|oo|s\-)|ai(ko|rn)|al(av|ca|co)|amoi|an(ex|ny|yw)|aptu|ar(ch|go)|as(te|us)|attw|au(di|\-m|r |s )|avan|be(ck|ll|nq)|bi(lb|rd)|bl(ac|az)|br(e|v)w|bumb|bw\-(n|u)|c55\/|capi|ccwa|cdm\-|cell|chtm|cldc|cmd\-|co(mp|nd)|craw|da(it|ll|ng)|dbte|dc\-s|devi|dica|dmob|do(c|p)o|ds(12|\-d)|el(49|ai)|em(l2|ul)|er(ic|k0)|esl8|ez([4-7]0|os|wa|ze)|fetc|fly(\-|_)|g1 u|g560|gene|gf\-5|g\-mo|go(\.w|od)|gr(ad|un)|haie|hcit|hd\-(m|p|t)|hei\-|hi(pt|ta)|hp( i|ip)|hs\-c|ht(c(\-| |_|a|g|p|s|t)|tp)|hu(aw|tc)|i\-(20|go|ma)|i230|iac( |\-|\/)|ibro|idea|ig01|ikom|im1k|inno|ipaq|iris|ja(t|v)a|jbro|jemu|jigs|kddi|keji|kgt( |\/)|klon|kpt |kwc\-|kyo(c|k)|le(no|xi)|lg( g|\/(k|l|u)|50|54|\-[a-w])|libw|lynx|m1\-w|m3ga|m50\/|ma(te|ui|xo)|mc(01|21|ca)|m\-cr|me(rc|ri)|mi(o8|oa|ts)|mmef|mo(01|02|bi|de|do|t(\-| |o|v)|zz)|mt(50|p1|v )|mwbp|mywa|n10[0-2]|n20[2-3]|n30(0|2)|n50(0|2|5)|n7(0(0|1)|10)|ne((c|m)\-|on|tf|wf|wg|wt)|nok(6|i)|nzph|o2im|op(ti|wv)|oran|owg1|p800|pan(a|d|t)|pdxg|pg(13|\-([1-8]|c))|phil|pire|pl(ay|uc)|pn\-2|po(ck|rt|se)|prox|psio|pt\-g|qa\-a|qc(07|12|21|32|60|\-[2-7]|i\-)|qtek|r380|r600|raks|rim9|ro(ve|zo)|s55\/|sa(ge|ma|mm|ms|ny|va)|sc(01|h\-|oo|p\-)|sdk\/|se(c(\-|0|1)|47|mc|nd|ri)|sgh\-|shar|sie(\-|m)|sk\-0|sl(45|id)|sm(al|ar|b3|it|t5)|so(ft|ny)|sp(01|h\-|v\-|v )|sy(01|mb)|t2(18|50)|t6(00|10|18)|ta(gt|lk)|tcl\-|tdg\-|tel(i|m)|tim\-|t\-mo|to(pl|sh)|ts(70|m\-|m3|m5)|tx\-9|up(\.b|g1|si)|utst|v400|v750|veri|vi(rg|te)|vk(40|5[0-3]|\-v)|vm40|voda|vulc|vx(52|53|60|61|70|80|81|83|85|98)|w3c(\-| )|webc|whit|wi(g |nc|nw)|wmlb|wonu|x700|yas\-|your|zeto|zte\-/i.test(a.substr(0,4)));

We don't want to give the funnelcake download out to users who are coming from mobile devices.

Comment 7

[Chris More [:cmore]]

Another option:

var isTouchDevice = function() {  return 'ontouchstart' in window || 'onmsgesturechange' in window; };
var isDesktop = window.screenX != 0 && !isTouchDevice() ? true : false;

Comment 8

[Jonathan Petto [:jpetto]]

Cool, thanks. That should be plenty to write up the segmentation. We'll get time allocated for this during our next sprint.

Comment 9

[Chris More [:cmore]]

(In reply to Jon Petto [:jpetto] from comment #5)
> If the scheduled start of the experiment is 5/20, we should have time to set
> this up using TrafficCop.
> 
> For the audience segmentation, what exactly does "Desktop" mean when applied
> in Optimizely? Is that screen size, not a known mobile browser, a
> combination, or something else?

Opps. I realized I wrote May 20th. I meant April 20th, like in two days, but it is possible it will slip to next week because they are still having issues QA'ing the build. Sorry, I was thinking May for something else. I just created this experiment yesterday because they needed it done.

If it slips to next week, maybe for traffic cop?

Comment 10

[Jonathan Petto [:jpetto]]

Ah, okay. 4/20 isn't going to be doable for TrafficCop. We've got too much work to finish before the end of this sprint to take on any additional.

If this slips to next week, we can likely get it done in TrafficCop. When will we know the target date? I took a quick pass at the Optimizely code and it's pretty basic, so we can review that quickly tomorrow or Thursday morning if needs be.

Comment 11

[Chris More [:cmore]]

(In reply to Jon Petto [:jpetto] from comment #10)
> Ah, okay. 4/20 isn't going to be doable for TrafficCop. We've got too much
> work to finish before the end of this sprint to take on any additional.
> 
> If this slips to next week, we can likely get it done in TrafficCop. When
> will we know the target date? I took a quick pass at the Optimizely code and
> it's pretty basic, so we can review that quickly tomorrow or Thursday
> morning if needs be.

Hi Jon.

New go live for the funnelcake is 4/27. Is that enough time to do TrafficCop?

Thanks

Comment 12

[Jonathan Petto [:jpetto]]

(In reply to Chris More [:cmore] from comment #11)
> 
> Hi Jon.
> 
> New go live for the funnelcake is 4/27. Is that enough time to do TrafficCop?
> 
> Thanks

Yep, that should give us enough time! Keep the Optimizely code around just in case, but I should be able to start on a TC version tomorrow.

Comment 13

[Jonathan Petto [:jpetto]]

:cmore - Will the funnelcakes be available for Win32 + Win64, or just one of the two?

Comment 14

[Jonathan Petto [:jpetto]]

Attachment: GitHub PR

Comment 15

[Chris More [:cmore]]

(In reply to Jon Petto [:jpetto] from comment #13)
> :cmore - Will the funnelcakes be available for Win32 + Win64, or just one of
> the two?

Comment 2 and comment 3 had a question to @pdol about Win64. The funnelcake will only be win32.

Awesome jpetto!

Comment 16

[Jonathan Petto [:jpetto]]

The branch is up on a demo - would be great to get some QA:

https://bedrock-demo-jpetto-fc.us-west.moz.works/en-US/firefox/new/

Comment 17

[Verdi [:verdi] Best to slack me]

(In reply to Jon Petto [:jpetto] from comment #16)
> The branch is up on a demo - would be great to get some QA:
> 
> https://bedrock-demo-jpetto-fc.us-west.moz.works/en-US/firefox/new/

I'm not seeing the night time download page. If I change the parameter at the end of the url to f=111 or 112 or 113 I still see the regular download page. I got 110 to begin with. Do I need to delete cookies or something?

Comment 18

[Jonathan Petto [:jpetto]]

(In reply to Verdi [:verdi] from comment #17)
> (In reply to Jon Petto [:jpetto] from comment #16)
> > The branch is up on a demo - would be great to get some QA:
> > 
> > https://bedrock-demo-jpetto-fc.us-west.moz.works/en-US/firefox/new/
> 
> I'm not seeing the night time download page. If I change the parameter at
> the end of the url to f=111 or 112 or 113 I still see the regular download
> page. I got 110 to begin with. Do I need to delete cookies or something?

What is the expected view? I didn't see anything above or in the Optimizely code denoting a different UI...

Comment 19

[Verdi [:verdi] Best to slack me]

(In reply to Jon Petto [:jpetto] from comment #18)
 
> What is the expected view? I didn't see anything above or in the Optimizely
> code denoting a different UI...

Maybe it was an oversight but we should be using the same download page we used for the 2.0 funnelcake. I'm not sure how it all works but here's the bug for that funnelcake https://bugzilla.mozilla.org/show_bug.cgi?id=1326252 and a prototype with a near final version of the download page - https://dl.dropboxusercontent.com/u/105549/mvp2.framer/index.html

Comment 20

[Verdi [:verdi] Best to slack me]

Oh - here's the bug for the page is bug 1333435 and a live example https://www.mozilla.org/en-US/firefox/new/?f=99

Comment 21

[Jonathan Petto [:jpetto]]

The demo has been updated to use the template from ?f=99:

https://bedrock-demo-jpetto-fc.us-west.moz.works/en-US/firefox/new/?f=111

Any other requests before we ship this?

Comment 22

[Jonathan Petto [:jpetto]]

:cmore - Are we good to roll this code into prod? It wont be enabled until we flip the switch, but needs to go to prod first.

Comment 23

[Chris More [:cmore]]

(In reply to Jon Petto [:jpetto] from comment #22)
> :cmore - Are we good to roll this code into prod? It wont be enabled until
> we flip the switch, but needs to go to prod first.

:jpetto: are the sample size and everything the same as what is mentioned in comment 4? How did you test to confirm the JS is doing the expected results? (more curious than anything)

The funnelcake is planned to release on the 27th, but if they run into QA problems, it may slip. I've pinged Nicole to confirm.

Comment 24

[Chris More [:cmore]]

:jpetto: ^^

Comment 25

[Jonathan Petto [:jpetto]]

(In reply to Chris More [:cmore] from comment #23)
> (In reply to Jon Petto [:jpetto] from comment #22)
> > :cmore - Are we good to roll this code into prod? It wont be enabled until
> > we flip the switch, but needs to go to prod first.
> 
> :jpetto: are the sample size and everything the same as what is mentioned in
> comment 4? How did you test to confirm the JS is doing the expected results?
> (more curious than anything)
> 
> The funnelcake is planned to release on the 27th, but if they run into QA
> problems, it may slip. I've pinged Nicole to confirm.

Traffic Cop gives each user not already assigned a variation a random number [1]. If that number falls within a variation range, they get that variation. If not, they get the normal page. In either case (variation or no variation), the user will see the same result for the duration of the cookie (which I've set to 1 week for this experiment [2]).

en-US is specified by the switch [3], the rest of the conditions are specified in the JS [4], with each variation having a 7% chance of being chosen [5]. (Hopefully 28% of traffic will do the trick - if not we can update to 8% each/32% total. Let me know here.)

[1] https://github.com/mozilla/trafficcop/blob/master/src/mozilla-traffic-cop.js#L205
[2] https://github.com/mozilla/bedrock/pull/4778/files#diff-15333cb77711f467605f0f32a0cbc670R18
[3] https://github.com/mozilla/bedrock/pull/4778/files#diff-dfcc4dd366ddd6697c2ddf03b6996429R10
[4] https://github.com/mozilla/bedrock/pull/4778/files#diff-15333cb77711f467605f0f32a0cbc670R10
[5] https://github.com/mozilla/bedrock/pull/4778/files#diff-15333cb77711f467605f0f32a0cbc670R20

Comment 26

[Chris More [:cmore]]

:jpetto: it looks like the funnelcake may be ready tomorrow (27th) to launch. Is everything ready for Traffic Cop to turn it on tomorrow once we get the green light from QA?

Comment 27

[Jonathan Petto [:jpetto]]

As long as there are no more change requests, we can merge and push the code today. After that, we'll just need to flip a switch to turn the experiment on.

We should make sure no other experiments are running on scene 1 of /firefox/new/, as Traffic Cop and Optimizely may not play well together. Will the "Firefox Downloaders Survey" be complete today?

We can perhaps try having both experiments running at the same time on stage, but should be prepared for it to go poorly.

Comment 28

[Verdi [:verdi] Best to slack me]

(In reply to Jon Petto [:jpetto] from comment #27)
> We should make sure no other experiments are running on scene 1 of
> /firefox/new/, as Traffic Cop and Optimizely may not play well together.
> Will the "Firefox Downloaders Survey" be complete today?
> 
> We can perhaps try having both experiments running at the same time on
> stage, but should be prepared for it to go poorly.

Wait. Why was Traffic Cop's incompatibility with Optimizely not brought up before? The Firefox Downloaders survey needs to run through May 2nd. If we'd known these didn't work together we would have insisted on doing the funnelcake with Optimizely. How long will it take to switch from Traffic Cop to Optimizely?

Comment 29

[Chris More [:cmore]]

(In reply to Jon Petto [:jpetto] from comment #27)
> As long as there are no more change requests, we can merge and push the code
> today. After that, we'll just need to flip a switch to turn the experiment
> on.
> 
> We should make sure no other experiments are running on scene 1 of
> /firefox/new/, as Traffic Cop and Optimizely may not play well together.
> Will the "Firefox Downloaders Survey" be complete today?
> 
> We can perhaps try having both experiments running at the same time on
> stage, but should be prepared for it to go poorly.

The download survey experiment will continue to run for a while, which is running in Optimizely. Since it is just inserting a survey link on the first and second scene and not doing any redirects, it should be fine. I could exclude the f=XXX IDs in the URL from the survey Optimizely experiment if we want to ensure they don't overlap. Thoughts?

Comment 30

[Chris More [:cmore]]

(In reply to Verdi [:verdi] from comment #28)
> (In reply to Jon Petto [:jpetto] from comment #27)
> > We should make sure no other experiments are running on scene 1 of
> > /firefox/new/, as Traffic Cop and Optimizely may not play well together.
> > Will the "Firefox Downloaders Survey" be complete today?
> > 
> > We can perhaps try having both experiments running at the same time on
> > stage, but should be prepared for it to go poorly.
> 
> Wait. Why was Traffic Cop's incompatibility with Optimizely not brought up
> before? The Firefox Downloaders survey needs to run through May 2nd. If we'd
> known these didn't work together we would have insisted on doing the
> funnelcake with Optimizely. How long will it take to switch from Traffic Cop
> to Optimizely?

It is already set up in Optimizely as a backup plan if we think Optimizely+Traffic Cop won't play nice together. Since one is a redirect and the other is just inserting into the DOM, any overlap, in theory should be fine. :jpetto, can you think of a scenario where it would be a problem?

Comment 31

[Jonathan Petto [:jpetto]]

Since Optimizely would only be injecting a link (and not performing a redirect), I don't think there'd be a problem. However, I would want to test on a demo first to make sure.

Before I do that, however, I want to ask how long this experiment is slated to run. If it will run far beyond 5/2, then I'll test the two frameworks together on a demo today. If this experiment will be done before, on, or shortly after 5/2, then it doesn't really make sense to go through the trouble of using Traffic Cop, as Optimizely would already be running on the page (meaning using Traffic Cop in addition may be worse for the end user).

Comment 32

[Verdi [:verdi] Best to slack me]

(In reply to Jon Petto [:jpetto] from comment #31) 
> Before I do that, however, I want to ask how long this experiment is slated
> to run. If it will run far beyond 5/2, then I'll test the two frameworks
> together on a demo today. 

This would need to run at least though May 14.

Comment 33

[Jonathan Petto [:jpetto]]

Okay, then I think we should try to use Traffic Cop for the funnelcake tests.

Re-iterating for my own sanity:

An en-US visitor lands on scene 1 of /firefox/new/

# Path 1

TC chooses them for a funnelcake and redirects them to /firefox/new/?f=11X. Optimizely never runs, as TC is first in the template and takes precedence. After the redirect, Optimizely does nothing, as it ignores visitors with ?f=11X in the URL. Visitor clicks the download button and goes to scene 2, while retaining the ?f=11X in the URL, which means Optimizely ignores this visitor on scene 2 as well.

# Path 2

TC chooses no funnelcake for the visitor. No redirect happens. Optimizely JS is executed as normal, inserting a survey link. User clicks download button. TC is not running on scene 2, so Optimizely is free to do it's thing.

If all the above is correct, we shouldn't have an issue.

:cmore - Does this look sane/safe to you?

Comment 34

[[github robot]]

Commits pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/4aa6acde087b3db4ed7721ac4caf7f992846e6e3
[fix bug 1348129] Experiment for funnelcakes 110-113.

https://github.com/mozilla/bedrock/commit/2b84d5e79ceb9adc81b38ffad30a911baa5d785f
Merge pull request #4778 from jpetto/bug-1348129-funnelcake-experiment

[fix bug 1348129] Experiment for funnelcakes 110-113.

Comment 35

[Chris More [:cmore]]

Talked to jpetto over IRC and we're good to go here. He is pushing to prod and will enable the funnelcake soon. I will then watch real-time GA to make sure I can see the funnel. This won't impact the existing download survey running.

Comment 36

[Chris More [:cmore]]

:jpetto: what is the overall sample rate you have it set to now? It is an even split among all variations, right?

The low down on the experiment is that we cannot start this experiment bug 1360397. Basically this page they created for f=99 (https://www.mozilla.org/en-US/firefox/51.0.1/firstrun/?f=99) needs to work for 111, 112 and 113. 

Also, the Firefox team noticed the distribution.ini in the funnelcakes had the wrong configs. So, both the /firstrun/ change and the distribution.ini in the funnelcakes needs to be resolved. 

I am on PTO on Friday, so Michael Verdi will be the go/no go person for you. Once he confirms the full funnel is working as planned, he will ping you to launch the experiment. 

If the world blows up, have someone text me. Thanks!

Comment 37

[Chris More [:cmore]]

Going to re-open this and we'll keep it open until the experiment is launched. Verdi will give the green light after all of the pieces are in place.

Comment 38

[Chris More [:cmore]]

See bug 1360400 for the distribution.ini file issues.

Comment 39

[Verdi [:verdi] Best to slack me]

I'm sorry Jon. I wasn't clear on the download pages and you probably don't know the intention of the test. Now all the variations have the nighttime download page. FC100 is our control and it should have the regular daytime "Browse Freely" download page.

Comment 40

[Nicole Yee (:nicoleyee)]

(In reply to Verdi [:verdi] from comment #39)
> I'm sorry Jon. I wasn't clear on the download pages and you probably don't
> know the intention of the test. Now all the variations have the nighttime
> download page. FC100 is our control and it should have the regular daytime
> "Browse Freely" download page.

Filing separate bug for this. let's not lose track of bugs in comments.

Comment 41

[Jonathan Petto [:jpetto]]

(In reply to Chris More [:cmore] from comment #36)
> :jpetto: what is the overall sample rate you have it set to now? It is an
> even split among all variations, right?

Each funnelcake has a 7% chance of being chosen for an en-US visitor, so 28% total.

Comment 42

[Chris More [:cmore]]

:jpetto: if we want to go out the door with a 1% total to minimize the risk and then up to 28% a few days later, is that a big problem?

Comment 43

[Jonathan Petto [:jpetto]]

The minimum we could do with Traffic Cop is 1% per variation, so 4% total. Would require a PR, but a pretty simple code change.

Comment 44

[Chris More [:cmore]]

(In reply to Jon Petto [:jpetto] from comment #43)
> The minimum we could do with Traffic Cop is 1% per variation, so 4% total.
> Would require a PR, but a pretty simple code change.

1% per and 4% total is suffice for now. Can we set this experiment to that for now, so we can go out the door small first? We'll give the green light soon.

Comment 45

[[github robot]]

Commits pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/6c9e0532c5abde11b000ae29fe6d743bb18071e9
[fix bug 1348129] Adjust funnelcake experiment audience size

https://github.com/mozilla/bedrock/commit/3a466f96a47b90cff91391640a8884762b8c25fa
Merge pull request #4796 from jpetto/bug-1348129-adjust-funnelcake-experiment-audience

[fix bug 1348129] Adjust funnelcake experiment audience size

Comment 46

[Chris More [:cmore]]

Green light on funnelcake 110-113! Ship it. Please ping :cmore on IRC when live and I will look at real-time GA traffic.

Comment 47

[Chris More [:cmore]]

Launched with 4% sample on May 2nd 2017 @ 12:55pm PST.

Comment 48

[[github robot]]

Commits pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/8c12570cf3d347e3a8360b432fa9d567ddef4f5f
[fix bug 1348129] Increase experiment audience size.

https://github.com/mozilla/bedrock/commit/dbdb925e95c0fbdb53cab46eb73b774c5a5b0d5c
Merge pull request #4801 from jpetto/bug-1348129-increase-experiment-audience-size

[fix bug 1348129] Increase experiment audience size.

Comment 49

[Chris More [:cmore]]

:jpetto: can you shut off this traffic cop experiment on 5/22? Thanks!

Comment 50

[Jonathan Petto [:jpetto]]

Will do. Should we shoot for any specific time, or just start of day 5/22?

Comment 51

[Chris More [:cmore]]

(In reply to Jon Petto [:jpetto] from comment #50)
> Will do. Should we shoot for any specific time, or just start of day 5/22?

Any time will do. Thanks

Comment 52

[Jonathan Petto [:jpetto]]

The experiment has been turned off via the switch, and I'm now opening a PR to remove the experiment code.

:pdol - Can we remove the template logic around these funnelcakes? In other words, if a visitor now hits https://www.mozilla.org/en-US/firefox/new/?f=112, is it okay (or preferred) for them to see the canonical template (https://www.mozilla.org/en-US/firefox/new/)? I'm assuming we can remove this template logic, but want to make sure.

Comment 53

[Peter Dolanjski [:pdol]]

(In reply to Jon Petto [:jpetto] from comment #52)
> :pdol - Can we remove the template logic around these funnelcakes? In other
> words, if a visitor now hits
> https://www.mozilla.org/en-US/firefox/new/?f=112, is it okay (or preferred)
> for them to see the canonical template
> (https://www.mozilla.org/en-US/firefox/new/)? I'm assuming we can remove
> this template logic, but want to make sure.

Yes, the template logic can be removed.

Comment 54

[[github robot]]

Commits pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/9b3cda99b69ee0e0d9301e184d43952a0145fc44
[fix bug 1348129] Remove experiment files.

https://github.com/mozilla/bedrock/commit/5bf9097912d40dbcdfae7ff735dd01b0901d3f61
Merge pull request #4846 from jpetto/bug-1348129-disable-funnelcake-experiment

[fix bug 1348129] Remove experiment files.