1128708 - Enable waffle flag for Optimizely test on /firstrun/ template

Status: RESOLVED FIXED

(www.mozilla.org :: Bedrock, defect)

Description

[Chris More [:cmore]]

We are going to be running an A/B test on this URL start on Feb 12th:

https://www.mozilla.org/en-US/firefox/35.0.1/firstrun/

Can we get Optimizely enabled on this page?

Will enabling this for firstrun enable it for all firstrun URLs given that the URL changes every release?

Comment 1

[Chris More [:cmore]]

Actually, we are not going to need this. We'll just do an in-page javascript random function for the sub a/b tests.

Comment 2

[Chris More [:cmore]]

Josh: I have an urgent request from mmayo to do a quick test that requires me having optimizely on the firstrun template. Can you help me get the waffle flag enabled for optimizely to be on the /firstrun/ base templates?

I added a line to this spreadsheet, but didn't add a flag or status:

https://docs.google.com/a/mozilla.com/spreadsheets/d/1gakdALIVwPnG_1BvVaNTnv8iDYkXH4JCQ6XcbDo45cM/edit#gid=0

Comment 3

[[github robot]]

Commits pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/39b5bbddf2ba04bf902b8531c768c5b92734d50b
Add optimizely to 36+ firstrun: fix bug 1128708

Add waffle switch 'firefox-firstrun-optimizely'

https://github.com/mozilla/bedrock/commit/2cab4fed26afcc3364fe21434e77873d22d20ccb
Merge pull request #2902 from jgmize/firstrun-optimizely

Add optimizely to firstrun: fix bug 1128708

Comment 4

[Josh Mize [:jgmize]]

Verified optimizely is now enabled on https://www.mozilla.org/en-US/firefox/36.0/firstrun/

Comment 5

[Chris More [:cmore]]

:jgmize :jpetto: can you let me know why Optimizely is not enabled on 

https://www.mozilla.org/en-US/firefox/38.0.5/firstrun/

but it is on:

https://www.mozilla.org/en-US/firefox/37.0/firstrun/

Shouldn't the waffle flag be generic to all versions regardless of the template?

I need to do an A/B test ASAP. Thanks.

Comment 6

[Josh Mize [:jgmize]]

Attachment: bedrock pr 3047

Comment 7

[Josh Mize [:jgmize]]

(In reply to Chris More [:cmore] from comment #5)
> :jgmize :jpetto: can you let me know why Optimizely is not enabled on 
> 
> https://www.mozilla.org/en-US/firefox/38.0.5/firstrun/
> 
> but it is on:
> 
> https://www.mozilla.org/en-US/firefox/37.0/firstrun/
> 
> Shouldn't the waffle flag be generic to all versions regardless of the
> template?


The reason the 38.0.5 firstrun currently lacks optimizely is that template doesn't inherit from the same template that was used for 36 through 38, and no optimizely block was added. I've submitted a PR to explicitly add it to this template, and going forward we should ensure that the optimizely block is present in any new firstrun templates or in one of the parent templates.

Comment 8

[Chris More [:cmore]]

(In reply to Josh Mize [:jgmize] from comment #7)
> (In reply to Chris More [:cmore] from comment #5)
> > :jgmize :jpetto: can you let me know why Optimizely is not enabled on 
> > 
> > https://www.mozilla.org/en-US/firefox/38.0.5/firstrun/
> > 
> > but it is on:
> > 
> > https://www.mozilla.org/en-US/firefox/37.0/firstrun/
> > 
> > Shouldn't the waffle flag be generic to all versions regardless of the
> > template?
> 
> 
> The reason the 38.0.5 firstrun currently lacks optimizely is that template
> doesn't inherit from the same template that was used for 36 through 38, and
> no optimizely block was added. I've submitted a PR to explicitly add it to
> this template, and going forward we should ensure that the optimizely block
> is present in any new firstrun templates or in one of the parent templates.

Agree. I thought it would have been in the parent template already. Thanks for the PR! :)

Comment 9

[[github robot]]

Commits pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/cdf72be366bd6e6819087a6ad8a43369f13f29a8
Add optimizely to 38.0.5 firstrun: fix bug 1128708

https://github.com/mozilla/bedrock/commit/95f1904a6fd39ff366e3ba49e057e5386bbd8624
Merge pull request #3047 from jgmize/fx-38.0.5-firstrun-optimizely

Add optimizely to 38.0.5 firstrun: fix bug 1128708

Comment 10

[Josh Mize [:jgmize]]

verified optimizely is now on https://www.mozilla.org/en-US/firefox/38.0.5/firstrun/

Comment 11

[Chris More [:cmore]]

We're going to need this flag re-enabled for our Mozlando Project. We'll needinfo this bug when we're ready.

Comment 12

[Chris More [:cmore]]

jgmize/pmac: Can you please enable the waffle flag for the base first run templates before the end of this week so this is not a blocker next week for our Mozlando Project? We will still use the normal code reviews and QA of the /firstrun/ tests, but we want to ensure that this waffle flag is not on the critical path of getting the test launched given the tight time frame. No tests will be running until next week and only after the test gets a r+.

flag:

firefox-firstrun-optimizely 

Once you enable it, please verify the snippet is visible on this page:

https://www.mozilla.org/en-US/firefox/42.0/firstrun/

Thank you!

Comment 13

[Paul [:pmac] McLanahan]

I think the process is pretty clear that we only enable the waffle flag when a test is reviewed and ready. Enabling a flag is very quick and easy. Just let us know when you're ready and we'll do it then. I'm very uncomfortable having optimizely snippets enabled on pages when there isn't an active test running. We're going to be doing other server work (code pushes, etc.) for the other Mozlando projects, so it won't be a big deal to do this as well.

Comment 14

[Chris More [:cmore]]

(In reply to Paul [:pmac] McLanahan from comment #13)
> I think the process is pretty clear that we only enable the waffle flag when
> a test is reviewed and ready. Enabling a flag is very quick and easy. Just
> let us know when you're ready and we'll do it then. I'm very uncomfortable
> having optimizely snippets enabled on pages when there isn't an active test
> running. We're going to be doing other server work (code pushes, etc.) for
> the other Mozlando projects, so it won't be a big deal to do this as well.

Ok, we'll want to flip the flag Tuesday or Wednesday AM. I will needinfo or ping people when we are ready.

Comment 15

[Paul [:pmac] McLanahan]

(In reply to Chris More [:cmore] from comment #14)
> Ok, we'll want to flip the flag Tuesday or Wednesday AM. I will needinfo or
> ping people when we are ready.

Sounds good. Thanks for understanding Chris. If you cant find us via those methods my cell number is correct in the phonebook.

Comment 16

[Chris More [:cmore]]

:pmac :jgmize: need waffle enabled on /firstrun/ template now that we are r+ on bug 1217183. Need this enabled asap so we can get the test running. Thanks!

Comment 17

[Paul [:pmac] McLanahan]

firefox-firstrun-optimizely switch is now "on" for stage and prod.

Comment 18

[Josh Mize [:jgmize]]

thanks for taking care of this, :pmac