Closed
Bug 979496
Opened 10 years ago
Closed 10 years ago
[Australis Onboarding] GA for tour copy test for Beta
Categories
(www.mozilla.org :: Analytics, defect)
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: malexis, Assigned: garethc)
References
Details
(Whiteboard: [kb=1293093] )
From Bug#944098 We will move forward with 2 versions of the tour copy: 1. feature driven (here are the new features and here is how you use them) vs 2. benefit driven (here is why they are new and what they do for you) With the goal of seeing which version has a higher tour completion rate, hypothesizing that a better educated user will generate more usage hours (despite our inability to fully test this). We are aiming for final copy on March 5, to be ready for Beta on March 18, analyze results, and begin localization of final text April 1 for General Audience release on April 29.
Updated•10 years ago
|
Whiteboard: [kb=1293093]
Comment 1•10 years ago
|
||
Hi Alex- Could you please give Gareth the URLs to test up the update test for Beta when they are ready? Thx, Jen
Flags: needinfo?(agibson)
Comment 2•10 years ago
|
||
Hi Gareth, URL's on the demo server are as follows: https://www-demo3.allizom.org/en-US/firefox/29.0/whatsnew/ https://www-demo3.allizom.org/en-US/firefox/29.0/whatsnew/b/ https://www-demo3.allizom.org/en-US/firefox/29.0/firstrun/ Thanks, Alex
Flags: needinfo?(agibson)
Assignee | ||
Comment 3•10 years ago
|
||
Hey Alex, As a follow-up to our conversation on vidyo, I noticed that when someone clicks on a bottom link within a step (version b), we fire an event for the link click, and then we trigger another event for when the next step loads. Unfortunately, this will make it difficult to determine whether someone is using link more vs. the arrows to navigate through the steps because the step event always fires. Instead, is it possible to track the link clicks and arrow clicks separately? For example, I just clicked 'Lets go' and am now on step 1. Can we set the link and arrow clicks to step 2 as: link: _gaq.push(['_trackEvent', 'Tour Interaction', 'link click to', 'Step 2']); arrow: _gaq.push(['_trackEvent', 'Tour Interaction', 'arrow click to', 'Step 2']); This way, we'll only need to fire one event, and we can keep our aggregate reporting the same because the event labels stay consistent. Then i'll be able to segment the steps by the event action to see what's getting used more. Does that work? Also, i noticed that we don't have any event tracking on the landing page after the user completes the tour. Can we set the following events on click of: Sync Button: link: _gaq.push(['_trackEvent', 'whatsnew Page Interactions - New Firefox Tour', 'button click', 'Get Started with Sync']); Learn more about the new Firefox links: _gaq.push(['_trackEvent', 'whatsnew Page Interactions - New Firefox Tour', 'link click', {url}]); Any questions, please feel free to ping me. Thanks for all your help! Gareth
Comment 4•10 years ago
|
||
Thanks, Gareth! Makes sense, I'll update on the demo server and you can test further.
Assignee | ||
Comment 5•10 years ago
|
||
Great! Here's the GA Experiments code for the demo3 test before we roll out to prod. Please paste this experiment code immediately after the opening head tag at the top of the original page only: https://www-demo3.allizom.org/en-US/firefox/29.0/whatsnew/ <!-- Google Analytics Content Experiment code --> <script>function utmx_section(){}function utmx(){}(function(){var k='66844022-36',d=document,l=d.location,c=d.cookie; if(l.search.indexOf('utm_expid='+k)>0)return; function f(n){if(c){var i=c.indexOf(n+'=');if(i>-1){var j=c. indexOf(';',i);return escape(c.substring(i+n.length+1,j<0?c. length:j))}}}var x=f('__utmx'),xx=f('__utmxx'),h=l.hash;d.write( '<sc'+'ript src="'+'http'+(l.protocol=='https:'?'s://ssl': '://www')+'.google-analytics.com/ga_exp.js?'+'utmxkey='+k+ '&utmx='+(x?x:'')+'&utmxx='+(xx?xx:'')+'&utmxtime='+new Date(). valueOf()+(h?'&utmxhash='+escape(h.substr(1)):'')+ '" type="text/javascript" charset="utf-8"><\/sc'+'ript>')})(); </script><script>utmx('url','A/B');</script> <!-- End of Google Analytics Content Experiment code --> I've set up the conversion goal as the 'Step 4' event. Let me know when its live and i'll turn on the experiment on demo3. Then we can test it. Thanks!
Comment 6•10 years ago
|
||
Thanks, Gareth! I've updated demo3 with the new tracking labels in comment 3 and the GA experiment snippet in comment 5. Also, the code for bug 981081 is in there and available for testing.
Assignee | ||
Comment 7•10 years ago
|
||
Looks like the demo experiment is populating with test data. I'm seeing conversion data flow into both variations based on our goal we set up. Test data: http://cl.ly/image/3c3t3p3s0A0y So, now I'll need to build another experiment for production. I'll post the updated snippet here shortly.
Assignee | ||
Comment 8•10 years ago
|
||
Alex, Here's the updated experiments snippet for when we push to prod. Please paste this experiment code immediately after the opening head tag at the top of the original page only: http://www.mozilla.org/en-US/firefox/29.0/whatsnew/ Thanks. <!-- Google Analytics Content Experiment code --> <script>function utmx_section(){}function utmx(){}(function(){var k='71153379-35',d=document,l=d.location,c=d.cookie; if(l.search.indexOf('utm_expid='+k)>0)return; function f(n){if(c){var i=c.indexOf(n+'=');if(i>-1){var j=c. indexOf(';',i);return escape(c.substring(i+n.length+1,j<0?c. length:j))}}}var x=f('__utmx'),xx=f('__utmxx'),h=l.hash;d.write( '<sc'+'ript src="'+'http'+(l.protocol=='https:'?'s://ssl': '://www')+'.google-analytics.com/ga_exp.js?'+'utmxkey='+k+ '&utmx='+(x?x:'')+'&utmxx='+(xx?xx:'')+'&utmxtime='+new Date(). valueOf()+(h?'&utmxhash='+escape(h.substr(1)):'')+ '" type="text/javascript" charset="utf-8"><\/sc'+'ript>')})(); </script><script>utmx('url','A/B');</script> <!-- End of Google Analytics Content Experiment code -->
Assignee | ||
Comment 9•10 years ago
|
||
Hey Alex, As a follow-up our conversation, the one thing I don't think we are capturing is the first time a user is seeing the tour. We need this so we can calculate the % that take the tour for the first time. So, after the page loads, can you please fire the following event: _gaq.push(['_trackEvent', 'Tour Interaction', 'First Time Seeing Tour', '[True OR False]',0,true]); The 0 is an event value that i need to assign, and the last value of true sets this event as being non-interactive. Please set the [TRUE or False] based on whether or not its their first time seeing the tour. Thanks. Gareth
Comment 10•10 years ago
|
||
We tested GA events and the A/B test on demo3 prior to prod push and Gareth confirmed all data was coming through ok. Marking this as fixed by bug 975099
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•