For the spring campaign, we added a brand video /firstrun/ page. We did a 50/50 split with a video overlay and video below to understand which one impacts the key conversions on the page. We kept the video for the Fx40 release so that we could compare the new embedded FxA form to the previous version that was just a button. We will have enough data by 8/19/2015. Let's get a PR done and have it staged after we get enough data to compare Fx38/39 to Fx40.
Whiteboard: [fxgrowth] → [fxgrowth][kb=1825117]
Created attachment 8648964 [details] [review] GitHub PR PR will also address displaying FxA error messages in the iframe. See https://github.com/mozilla/fxa-content-server/issues/2934
PR above has been pushed to demo4 for testing: https://www-demo4.allizom.org/firefox/40.0/firstrun/ https://www-demo4.allizom.org/firefox/38.0.5/firstrun/
Thanks :jpetto. We are ready to push this once we get this QA'd and code reviewed.
Commits pushed to master at https://github.com/mozilla/bedrock https://github.com/mozilla/bedrock/commit/e31e7366850432b99f4021139e3210c6b1b84f46 [fix bug 1195391] Remove video from Fx38+ firstrun. - Enable display of FxA frame error. https://github.com/mozilla/bedrock/commit/e0724bd88d0739fa975c33ff26ada7389e03724a Merge pull request #3215 from mozilla/bug-1195391-remove-video-fx38+-firstrun [fix bug 1195391] Remove video from Fx38+ firstrun.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
Video was removed yesterday and we will monitor the conversion rates. From previous analysis, the video wasn't helping with conversion rates and only 5% of users were even watching the video.
Here's the stats from the video analysis: Video plays: 2-4% of all /firstrun/ users Video complete: 20% of the video play users completed or about 0.06% of all /firstrun/ users. Video impact to FxA conversion: The video lowered the conversion rate to engaging with FxA by 15-20% Video impact to bounce rate: The presence of the video increased page bounce rate by 20% Video impact to ADIs: The presence of the video decreased retention by 20% over 2.5 months From these results, there is a new hypothesis: Given that the FxA rate increased when no video was present and bounce rate decreased, users that signed up for an account and setup sync were less likely to churn over time. The video wasn't making them churn, but the video negatively impacted sign up rates and getting more people to sign up retained the no-video cohort longer.
You need to log in before you can comment on or make changes to this bug.