Remove spring campaign video from Fx 38+ /firstrun/ pages

RESOLVED FIXED

Status

www.mozilla.org
Pages & Content
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: cmore, Assigned: jpetto)

Tracking

Production

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [fxgrowth][kb=1825117])

Attachments

(1 attachment)

44 bytes, text/x-github-pull-request
Details | Review | Splinter Review
(Reporter)

Description

3 years ago
For the spring campaign, we added a brand video /firstrun/ page. We did a 50/50 split with a video overlay and video below to understand which one impacts the key conversions on the page.

We kept the video for the Fx40 release so that we could compare the new embedded FxA form to the previous version that was just a button. We will have enough data by 8/19/2015.

Let's get a PR done and have it staged after we get enough data to compare Fx38/39 to Fx40.
(Reporter)

Updated

3 years ago
Assignee: nobody → jon
(Reporter)

Updated

3 years ago
Whiteboard: [fxgrowth]
(Assignee)

Updated

3 years ago
Whiteboard: [fxgrowth] → [fxgrowth][kb=1825117]
(Assignee)

Comment 1

3 years ago
Created attachment 8648964 [details] [review]
GitHub PR

PR will also address displaying FxA error messages in the iframe. See https://github.com/mozilla/fxa-content-server/issues/2934
(Reporter)

Comment 3

3 years ago
Thanks :jpetto. We are ready to push this once we get this QA'd and code reviewed.

Comment 4

3 years ago
Commits pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/e31e7366850432b99f4021139e3210c6b1b84f46
[fix bug 1195391] Remove video from Fx38+ firstrun.

- Enable display of FxA frame error.

https://github.com/mozilla/bedrock/commit/e0724bd88d0739fa975c33ff26ada7389e03724a
Merge pull request #3215 from mozilla/bug-1195391-remove-video-fx38+-firstrun

[fix bug 1195391] Remove video from Fx38+ firstrun.

Updated

3 years ago
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
(Reporter)

Comment 5

3 years ago
Video was removed yesterday and we will monitor the conversion rates. From previous analysis, the video wasn't helping with conversion rates and only 5% of users were even watching the video.
(Reporter)

Comment 6

3 years ago
Here's the stats from the video analysis:

Video plays: 2-4% of all /firstrun/ users
Video complete: 20% of the video play users completed or about 0.06% of all /firstrun/ users.
Video impact to FxA conversion: The video lowered the conversion rate to engaging with FxA by 15-20%
Video impact to bounce rate: The presence of the video increased page bounce rate by 20%
Video impact to ADIs: The presence of the video decreased retention by 20% over 2.5 months

From these results, there is a new hypothesis:

Given that the FxA rate increased when no video was present and bounce rate decreased, users that signed up for an account and setup sync were less likely to churn over time. The video wasn't making them churn, but the video negatively impacted sign up rates and getting more people to sign up retained the no-video cohort longer.
You need to log in before you can comment on or make changes to this bug.