Closed Bug 1452807 Opened 6 years ago Closed 5 years ago

Funnelcakes 133 / 134 Facebook video traffic

Categories

(Release Engineering :: Release Requests, enhancement)

Product:
Release Engineering
Component:
Release Requests
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: frios, Unassigned)

References

Details

Facebook is prompting users that are unable to make video calls through their non-webrtc browser to download Chrome or Firefox. 

We'd like to improve the value prop for these visitors and make it easier for them to download Firefox and get back to Facebook quickly so that they can complete the video call.

#133 control: vanilla funnelcake
#134 variation 1: launch Facebook video URL upon install

Detailed test description to follow
What are our timeframes?
What platforms and locales are we targeting?

I'm not familiar with what needs to be different about #134. Does that require any browser/stub installer code? Is the video URL static, or will it be dynamic per download? (Is that even doable?) If it requires browser code, do we have developers assigned?

Do we want a variation 2 involving the Facebook Container?
(In reply to Aki Sasaki [:aki] from comment #2)
> What platforms and locales are we targeting?

Per the link, looks like en-US only.
My understanding is the only different here is showing a different firstrun URL on load:

https://www.mozilla.org/en-US/firefox/new/?xv=facebookvideo

That is going to show a video about FAcebook.
(In reply to Aki Sasaki [:aki] from comment #2)
> What are our timeframes?
> What platforms and locales are we targeting?
en-US and Windows
> I'm not familiar with what needs to be different about #134. Does that
> require any browser/stub installer code? Is the video URL static, or will it
> be dynamic per download? (Is that even doable?) If it requires browser code,
> do we have developers assigned?
Only change is change first run page from opening the accounts page, to launching facebook.com instead. If that's not possible, I can help create a custom first run page with a button that directs the user to facebook.com
> 
> Do we want a variation 2 involving the Facebook Container?
No container variation. Someone has to explicitly tell us they want it, and it's too much information to give at this moment.
Flags: needinfo?(aki)
(In reply to Mike Kaply [:mkaply] from comment #4)
> My understanding is the only different here is showing a different firstrun
> URL on load:
> 
> https://www.mozilla.org/en-US/firefox/new/?xv=facebookvideo
> 
> That is going to show a video about FAcebook.

Hey Mike, we want the funnelcake to launch facebook.com instead of our traditional first run page. Alternatively, I can create a custom first run page that points them to facebook.com.

I'm leaning toward the former because it gives the user a better experience by getting them back to Facebook asap.
Flags: needinfo?(mozilla)
Won't that mess up tracking if we go directly to Facebook? Because we won't be able to follow the user to the firstrun funnel?
Flags: needinfo?(mozilla)
(In reply to Mike Kaply [:mkaply] from comment #7)
> Won't that mess up tracking if we go directly to Facebook? Because we won't
> be able to follow the user to the firstrun funnel?

These people will miss out on the Sync/Account message. I think that's OK because we know that they only want one thing at this moment, to get on that video call they weren't able to complete on their other browser.
Would it also be helpful if we offer to do the first-run password etc syncing from their previous browser? In case they forgot their facebook login but the previous browser remembers it.

I think we need to decide on which URL we go to & any other behavioral changes before we can spin up the funnelcake.
Flags: needinfo?(aki)
Fabio and I had a call about this. We're going to show the facebook page as the firstrun page and the Firefox firstrun as an additional page (so tracking works).

I've pushed these funnecake configs.
I verified 134 work as expected - Facebook as welcome page, other pages in background.
Added the Firefox-stub-f133 and 134 locations for win and win64. I think the releng portion is done?
> Added the Firefox-stub-f133 and 134 locations for win and win64. I think the releng portion is done?

Yep.
(In reply to Mike Kaply [:mkaply] from comment #14)
> > Added the Firefox-stub-f133 and 134 locations for win and win64. I think the releng portion is done?
> 
> Yep.

Hey Mike, remind me, did we agree to go live with this in 60 so that we wouldn't have to recreate a new version on next release?
Flags: needinfo?(mozilla)
> Hey Mike, remind me, did we agree to go live with this in 60 so that we wouldn't have to recreate a new version on next release?

We did agree on 60, but it was mainly that it was easier with testing cohorts.

Typically funnelcakes don't go live this late in a release cycle. We like to start them a week or two after a release.
Flags: needinfo?(mozilla)
Just noting that we're working the the landing page that will host this funnelcake. ETA is May 9th
5/9 meeting led to the following outcomes:

QA of design yielded these decisions (and the Approver):
- Keep the image Carolyn presented - Fabio
- Keep the main copy - Fabio
- Content will exist in a push down structure, not an overlay to the page - Fabio

Other decisions/confirmations:
- Content is shown to en-US audience only - Fabio
- Content is shown on /firefox/ only (just the firefox page, none 'sub' to it) - Fabio

Action items and owners:
- Provide clarity in the button that the user is downloading - Carolyn
- Add the required Fx privacy notice / link below the download button - Carolyn

- Confirm product name (FB video chat or whatever it is)- Fabio
- Confirm w. legal that we may use the actual product name - Fabio
- Acquire Legal approval of all content - Fabio

- Secure screen shot from PC and remit to Carolyn asap (5/9) - Jennifer/Craig
Flags: needinfo?(jrouse)
Flags: needinfo?(frios)
Flags: needinfo?(craigcook.bugz)
Please ignore comment 19, wrong bug . . .
Flags: needinfo?(jrouse)
Flags: needinfo?(frios)
Flags: needinfo?(craigcook.bugz)
Blocks: 1458340
(In reply to Mike Kaply [:mkaply] from comment #16)
> > Hey Mike, remind me, did we agree to go live with this in 60 so that we wouldn't have to recreate a new version on next release?
> 
> We did agree on 60, but it was mainly that it was easier with testing
> cohorts.
> 
> Typically funnelcakes don't go live this late in a release cycle. We like to
> start them a week or two after a release.

Hey Mike, we're planning on going live this week

What are the links we should be using?
Flags: needinfo?(mozilla)
Responding with quotes from Slack conversation w. MKaply
--------
And the question is if it (FB container funnelcake) will be updated w. 60

Michael Kaply [9:18 AM]
Ah, yes, that happens automatically.

Jon Petto [9:19 AM]
but does the funnelcake id change?

Eric Renaud [9:19 AM]
automated, fantastic. So no required update to the URL?

Michael Kaply [9:20 AM]
No, ID doesn't change.

Jon Petto [9:20 AM]
excellent
--------
Flags: needinfo?(mozilla)
Fabio - 

Can you clarify the purpose of the vanilla funnelcake? Is the intent to determine if opening Firefox on first run to facebook.com matters in this scenario in regards to user retention?

I'm assuming we want to do a 50/50 split between the funnelcakes - is this correct?
Flags: needinfo?(frios)
(In reply to Jon Petto [:jpetto] from comment #22)
> Fabio - 
> 
> Can you clarify the purpose of the vanilla funnelcake? Is the intent to
> determine if opening Firefox on first run to facebook.com matters in this
> scenario in regards to user retention?
> 
> I'm assuming we want to do a 50/50 split between the funnelcakes - is this
> correct?

Hey Jon, let's leave off the vanilla funnelcake. 

Yes, that was my original intention here. But based on the changes we've had to make, I don't know that we'll have enough traffic if we split it up. If it's OK with you, let's just run the funnelcake that opens up facebook.com.
Flags: needinfo?(frios)
confirming that Michael Feldman from legal team reviewed and approved.
(In reply to Aki Sasaki [:aki] from comment #13)
> Added the Firefox-stub-f133 and 134 locations for win and win64. I think the
> releng portion is done?

I've refreshed those bouncer locations to point at 60.0 (the path also slightly changed with the new partner generation automation).

jlund & mtabara: we'll need to update the locations for each 60.0.x and so on, for whatever the lifetime of this is (not stated here AFAICT). See also bug 1450463 for funnelcake135.
(In reply to Nick Thomas [:nthomas] (UTC+12) from comment #25)
> jlund & mtabara: we'll need to update the locations for each 60.0.x and so
> on, for whatever the lifetime of this is (not stated here AFAICT). See also
> bug 1450463 for funnelcake135.

tracking for 60.0.1: https://github.com/mozilla-releng/releasewarrior-data/blob/master/inflight/firefox/firefox-release-60.0.1.md#status
(In reply to Fabio Rios [:frios] from comment #23)
> (In reply to Jon Petto [:jpetto] from comment #22)
> > Fabio - 
> > 
> > Can you clarify the purpose of the vanilla funnelcake? Is the intent to
> > determine if opening Firefox on first run to facebook.com matters in this
> > scenario in regards to user retention?
> > 
> > I'm assuming we want to do a 50/50 split between the funnelcakes - is this
> > correct?
> 
> Hey Jon, let's leave off the vanilla funnelcake. 
> 
> Yes, that was my original intention here. But based on the changes we've had
> to make, I don't know that we'll have enough traffic if we split it up. If
> it's OK with you, let's just run the funnelcake that opens up facebook.com.

Confirming that we will serve funnelcake 134 to all visitors seeing the banner - Windows, IE/Edge, en-US, facebook.com in referring URL.
(In reply to Jon Petto [:jpetto] from comment #27)
> (In reply to Fabio Rios [:frios] from comment #23)
> > (In reply to Jon Petto [:jpetto] from comment #22)
> > > Fabio - 
> > > 
> > > Can you clarify the purpose of the vanilla funnelcake? Is the intent to
> > > determine if opening Firefox on first run to facebook.com matters in this
> > > scenario in regards to user retention?
> > > 
> > > I'm assuming we want to do a 50/50 split between the funnelcakes - is this
> > > correct?
> > 
> > Hey Jon, let's leave off the vanilla funnelcake. 
> > 
> > Yes, that was my original intention here. But based on the changes we've had
> > to make, I don't know that we'll have enough traffic if we split it up. If
> > it's OK with you, let's just run the funnelcake that opens up facebook.com.
> 
> Confirming that we will serve funnelcake 134 to all visitors seeing the
> banner - Windows, IE/Edge, en-US, facebook.com in referring URL.

And also must be landing on /firefox <- that's important since there's also a large set of users from Facebook that are directed to /new.

Happy to chat through this.
Flags: needinfo?(jon)
Ah, yes, also landing on /firefox. (I knew I forgot one condition in comment 27 ;)
Flags: needinfo?(jon)
(In reply to Jordan Lund (:jlund) from comment #26)
> (In reply to Nick Thomas [:nthomas] (UTC+12) from comment #25)
> > jlund & mtabara: we'll need to update the locations for each 60.0.x and so
> > on, for whatever the lifetime of this is (not stated here AFAICT). See also
> > bug 1450463 for funnelcake135.
> 
> tracking for 60.0.1:
> https://github.com/mozilla-releng/releasewarrior-data/blob/master/inflight/
> firefox/firefox-release-60.0.1.md#status

done. updated win32 + win64 locations for both funnelcake 133 and 134
Updated bouncer locations to 61.0.1.
Updated bouncer locations to 62.0.2.

Fabio, have we revisited the effectiveness of this campaign and our future plans for it ?

From comment #23, and confirming with telemetry data, we're not distributing the 133 funnelcake (control) so we might as well remove that configuration.
Flags: needinfo?(frios)
Frios is now working on Pocket marketing.

Yes - let's please remove the control.  I'll find out what I can about any extant sources driving traffic to the 'video version'
Flags: needinfo?(frios) → needinfo?(nthomas)

Updated bouncer to point at 65.0 instead of 62.0.2, :/

Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Component: Custom Release Requests → Release Requests
You need to log in before you can comment on or make changes to this bug.