Closed
Bug 695473
Opened 14 years ago
Closed 14 years ago
[brand-Q4] Launch & Monitor new Firefox download page with stronger brand messaging (en-US)
Categories
(www.mozilla.org :: General, defect)
www.mozilla.org
General
Tracking
(Not tracked)
VERIFIED
FIXED
4.8
People
(Reporter: jslater, Assigned: sgarrity)
References
()
Details
One of Engagement's Q4 goals is to communicate our brand messaging more boldly and effectively during the Firefox download experience.
I've filed bug 695462 and bug 695465 to track the copy and design portion of the work, but also want to make sure this is on the web team's radar.
The assets are due on 11/3...Laura, what's the next release after that we can realistically get this into?
Reporter | ||
Updated•14 years ago
|
OS: Mac OS X → All
Hardware: x86 → All
Summary: [brand=Q4] update Firefox download page with stronger brand messaging → [brand-Q4] update Firefox download page with stronger brand messaging
Comment 1•14 years ago
|
||
Hey John - Thanks for filing. For next steps please enter the final copy - or copy variations we'd like to test, here and then Steven can help us build out a new page with this new copy.
In terms of testing: we'll definitely want to test this new homepage copy before pushing live to 100% of users, so let us know the copy the sooner the better. In the past we've found that modifying copy does not affect download conversion so I'd expect to see the same this time, although we still want to make sure we test.
Assignee: nobody → jslater
Target Milestone: --- → 4.5
Updated•14 years ago
|
Target Milestone: 4.5 → 4.6
Comment 2•14 years ago
|
||
Steven - Please create a separate URL for this so we can test how this converts. We'll want to direct 50% of the traffic here, 50% to /firefox/new. This can be called something like /firefox/new2.
Thanks!
John will update this bug with the new copy when it's ready.
Reporter | ||
Comment 3•14 years ago
|
||
Actually, can we test two here? If so, we'd have 50% going to the regular page and 25% going to both test pages.
The test copy has been finalized and is ready to go. Here is is:
headline 1: Different by Design
bullets 1: Proudly non-profit User-focused innovation Fast, flexible, secure
headline 2: The People's Browser
bullets 2: Proudly non-profit User-focused innovation Fast, flexible, secure
All other elements (button, etc) are the same.
Thanks!
Comment 4•14 years ago
|
||
Lets make sure we are clear on what we are testing for. Are we testing for downloads w/ the headlines?
Comment 5•14 years ago
|
||
(In reply to John Slater from comment #3)
> Actually, can we test two here? If so, we'd have 50% going to the regular
> page and 25% going to both test pages.
>
> The test copy has been finalized and is ready to go. Here is is:
>
> headline 1: Different by Design
> bullets 1: Proudly non-profit User-focused innovation Fast, flexible,
> secure
>
> headline 2: The People's Browser
> bullets 2: Proudly non-profit User-focused innovation Fast, flexible,
> secure
>
> All other elements (button, etc) are the same.
>
> Thanks!
Yes, that's fine.
Steven - can you create two new URLs to test this? Perhaps /firefox/new1 and /firefox/new2?
This is ready for Dev! Sooner the better on this one. HTML changes only (and, er, two new pages) so should be semi easy. Getting this into tomorrows batch would be great.
Assignee: jslater → steven
Comment 6•14 years ago
|
||
(In reply to Kristin Baird from comment #4)
> Lets make sure we are clear on what we are testing for. Are we testing for
> downloads w/ the headlines?
In this instance we're testing to make sure the new versions perform at a similar download conversion rate as our current version. We want to monitor any change to this high traffic, important page.
For any additional tests (retention rates, non-profit awareness) let's talk off-bug and create new bugs/talk to metrics team, etc.
Comment 7•14 years ago
|
||
Steven - Please do use "new1" and "new2" as the URLs - I have created custom reporting based on those URLs and we won't be able to track download conversion rate if any other URLs are used. Thanks!
Assignee | ||
Comment 8•14 years ago
|
||
Assignee | ||
Updated•14 years ago
|
Whiteboard: r=97874, b=trunk → r=97874,97875 b=trunk
Assignee | ||
Comment 9•14 years ago
|
||
Made a text update requested by Kristin Baird via email in trunk (replaced "User-focused innovation" with "Innovating for you").
Whiteboard: r=97874,97875 b=trunk → r=97874,97875,97901 b=trunk
Assignee | ||
Comment 10•14 years ago
|
||
Now that we have the text changes implemented and Laura has a stats/logging setup implemented via webtrends, we just need to get some actual traffic to these pages.
The simplest method I can thing of would be a javascript redirect, but this is ugly in a few respects:
A. It will involve a performance hit - the extra round-trip to the server with slow the pages down a bit. It may also have a brief flash of the old page before the new page loads.
B. Noise in the stats - if we're comparing pages and some redirect and others don't, then we're polluting the test results. We may learn that the page is less effective, but won't know if it's due to the text changes, or the redirect.
One possible (but ugly) solution would be to setup four distinct paths for visitors:
1. The default page - no redirect
2. The default page with a redirect
3. The /new1 page with a redirect
4. the /new2 page with a redirect
Most people can be sent to option 1., which don't have any performance impact. The test can be based on #2, #3, and #4, which will give the default, new1, and new2 an even playing field with the redirect performance hit.
As a side-effect of this method, we'll also be able to compare the default with with and without a redirect which would help determine how much the redirect is influencing our other results.
Any other suggestions? Something server-side would be ideal, but I'm not sure we can do that easily with our current setup.
Comment 11•14 years ago
|
||
adding Corey here too, since this bug depends on bug 700746
Summary: [brand-Q4] update Firefox download page with stronger brand messaging → [brand-Q4] A/B Test Firefox download page with stronger brand messaging (en-US)
Comment 12•14 years ago
|
||
(In reply to Steven Garrity from comment #10)
> Now that we have the text changes implemented and Laura has a stats/logging
> setup implemented via webtrends, we just need to get some actual traffic to
> these pages.
>
> The simplest method I can thing of would be a javascript redirect, but this
> is ugly in a few respects:
>
> A. It will involve a performance hit - the extra round-trip to the server
> with slow the pages down a bit. It may also have a brief flash of the old
> page before the new page loads.
> B. Noise in the stats - if we're comparing pages and some redirect and
> others don't, then we're polluting the test results. We may learn that the
> page is less effective, but won't know if it's due to the text changes, or
> the redirect.
>
> One possible (but ugly) solution would be to setup four distinct paths for
> visitors:
>
> 1. The default page - no redirect
> 2. The default page with a redirect
> 3. The /new1 page with a redirect
> 4. the /new2 page with a redirect
>
> Most people can be sent to option 1., which don't have any performance
> impact. The test can be based on #2, #3, and #4, which will give the
> default, new1, and new2 an even playing field with the redirect performance
> hit.
>
> As a side-effect of this method, we'll also be able to compare the default
> with with and without a redirect which would help determine how much the
> redirect is influencing our other results.
>
> Any other suggestions? Something server-side would be ideal, but I'm not
> sure we can do that easily with our current setup.
Steven, just added you to bug 700746 - question is over to Corey on when/if sitespect can be redeployed. If not, we'll need to move forward with your recommended solution in comment 10.
Assignee | ||
Updated•14 years ago
|
Target Milestone: 4.7 → 4.9
Reporter | ||
Comment 13•14 years ago
|
||
Resolving this per https://bugzilla.mozilla.org/show_bug.cgi?id=704645#c6. Thanks all.
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → WONTFIX
Assignee | ||
Comment 14•14 years ago
|
||
Reverted the .htaccess changes from this bug in trunk in r98082.
Whiteboard: r=97874,97875,97901 b=trunk → r=97874,97875,97901,98082 b=trunk
Comment 15•14 years ago
|
||
qa-verified trunk http://viewvc.svn.mozilla.org/vc/projects/mozilla.com/trunk/.htaccess?r1=98082&r2=98081&pathrev=98082
Status: RESOLVED → VERIFIED
Keywords: qawanted
Comment 16•14 years ago
|
||
Looped back with John and we don't want to launch the new landing page without measuring the affect it has on download. That's not the direct objective of this campaign, but it is the objective of the page, and knowing the affect this page has on download is important before diving straight in.
Plan: Launch the new page at the /new1 URL. Send 75% of traffic to this new page. Send 25% of traffic to the current /new page.
Dependencies: None - we can measure this manually through Webtrends and do not need Sitespect. Sitespect would be a lot easier, but we can do this manually for now, and I've setup the reporting to do so.
Status: VERIFIED → REOPENED
Resolution: WONTFIX → ---
Summary: [brand-Q4] A/B Test Firefox download page with stronger brand messaging (en-US) → [brand-Q4] Launch & Monitor new Firefox download page with stronger brand messaging (en-US)
Target Milestone: 4.9 → 4.8
Comment 17•14 years ago
|
||
(In reply to Laura Forrest from comment #16)
> Looped back with John and we don't want to launch the new landing page
> without measuring the affect it has on download. That's not the direct
> objective of this campaign, but it is the objective of the page, and knowing
> the affect this page has on download is important before diving straight in.
>
> Plan: Launch the new page at the /new1 URL. Send 75% of traffic to this new
> page. Send 25% of traffic to the current /new page.
>
> Dependencies: None - we can measure this manually through Webtrends and do
> not need Sitespect. Sitespect would be a lot easier, but we can do this
> manually for now, and I've setup the reporting to do so.
Looks like we aren't able to technically split traffic between two pages after-all. If we could, then we'd be able to use Webtrends to measure, but we can't without Zeus or SiteSpect.
That said, let's launch this to 100% of our users and then monitor the affect it has on downloads and non-profit awareness in real-time. Not ideal, but we will be able to monitor closely to see the affect this change has and compare it to conversion wk/wk.
For those results follow Bug 702414.
Status: REOPENED → RESOLVED
Closed: 14 years ago → 14 years ago
Resolution: --- → FIXED
Comment 19•14 years ago
|
||
I changed the large image on the page to be served from the CDN in r98241
Assignee | ||
Comment 20•14 years ago
|
||
(In reply to James Long (:jlongster) from comment #19)
> I changed the large image on the page to be served from the CDN in r98241
Thanks James. Since the CSS itself is usually served from the CDN, we use relative URLs for the images in CSS, so they come from the CDN too. I forgot that in this case we are including the CSS inline in the HTML and that doesn't work.
Comment 21•14 years ago
|
||
(In reply to Steven Garrity from comment #20)
> (In reply to James Long (:jlongster) from comment #19)
> > I changed the large image on the page to be served from the CDN in r98241
>
> Thanks James. Since the CSS itself is usually served from the CDN, we use
> relative URLs for the images in CSS, so they come from the CDN too. I forgot
> that in this case we are including the CSS inline in the HTML and that
> doesn't work.
Oh that's right, that's how that works. I'm not convinced that we need to serve everything inline; as long as we compress everything into one file, we should be able to serve it from the CDN without problems.
We'll fix that in bedrock though!
Comment 22•14 years ago
|
||
verified fixed http://www.mozilla.org/en-US/firefox/new/
Status: RESOLVED → VERIFIED
Updated•13 years ago
|
Component: www.mozilla.org/firefox → www.mozilla.org
Updated•13 years ago
|
Component: www.mozilla.org → General
Product: Websites → www.mozilla.org
You need to log in
before you can comment on or make changes to this bug.
Description
•