Closed Bug 596010 Opened 14 years ago Closed 14 years ago

Redesigned Download Page: Assets for Multivariate Testing

Categories

(Marketing :: Design, task)

Product:
Marketing
Component:
Design
Platform:
All
Other
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: lforrest, Assigned: jslater)

Details

(Whiteboard: [ms-11/12])

Attachments

(4 files)

homepage test - browser only version
869.46 KB, image/jpeg
Details
browser revised - 618x366
114.35 KB, image/png
Details
browser revised - 618x366 new shadow
115.20 KB, image/png
Details
Screenshot of possibly broken footer
40.48 KB, image/png
Details
We'd like to test different variations of the new download page to see what combinations result in the most downloads. To do this, various pieces of creative are needed - if would be great if these pieces can be created as this page nears completion: 1. Graphic Promo: A product shot without the animated cartoon character - just the product screenshot with its respective callouts/annotations. Hypothesis - a simple graphic will convert better than a complex one. 2. Download Button: A slightly larger version of the final download button. Hypothesis - download button may be getting overwhelmed by new animated graphic. A larger button will help draw back attention. 3. Headline: Two different variations, copy only needed since this isn't a graphic. Would be nice to test the final headline against a shorter one. Hypothesis - less copy converts better than more copy. 4. Download Counter: Another version of the download counter. Since we know that this is the #1 component that influences downloads, lets play with two different versions of the counter. Its been through a couple of iterations in the past mockups that we can perhaps pull from. Note: since we've been through so many iterations on this page these graphic pieces may already exist. My intent is to not create a lot of new work, it's to test slight alterations of almost final designs elements of this page - designs that we may have liked a lot in the past, but didn't make it in for some reason. Also lets try not to worry too much if the elements we're testing don't look perfect on the page. Remember that the spirit of testing is to try new things even if they feel a little uncomfortable at first ;) Timing - John, let me know what's reasonable. We'll want to run this test for about a week before we can get results.
(In reply to comment #0) > 2. Download Button: A slightly larger version of the final download button. > Hypothesis - download button may be getting overwhelmed by new animated > graphic. A larger button will help draw back attention. Lets change this from testing button size, to testing button colors. I think the button is as big as it can be for that space. The last mockup showed a green button. Lets also test a darker green and another color that works well with the page, for a total of 3 colors.
Update to this: Since the page has changed since this bug was filed here's an updated list of the assets we'll need to conduct testing: 1. Graphic Promo: A product shot without the animated cartoon character - just the product screenshot with its respective callouts/annotations. Hypothesis - a simple graphic will convert better than a complex one. Note: we may need to add in more product descriptions or the caption bubble to make this look right - right now there's no feature/product callouts. 2. Headline: Two different variations, copy only needed since this isn't a graphic. Would be nice to test the "Your Web" copy against new copy that's more Firefox/feature oriented. We've discussed something related to Speed - lets check with the PMMs on this one. 3. Language Dropdown: Not sure if we'll need another asset for this but Blake would like to test putting the dropdown at the top of the page instead of the bottom. The theory is that this will help our international users easily change to their default language. Reminder: in the near future we'd like to add back the download counter and possibly test the effect of putting it in different locations. If we're ready to do that now, then great. If not, lets keep this in mind.
I talked to Steven earlier...he said the basic version of the page should be ready for testing as early as Monday. There are still some tweaks he'll need to make, but nothing that should have a huge impact on the performance one way or the other. Blake, will you be implementing the versions here, or do we need Steven to do anything? I'm also copying Lee as a general heads up for any potential design work (Lee, it'll be minor). (In reply to comment #2) > 1. Graphic Promo: A product shot without the animated cartoon character - just > the product screenshot with its respective callouts/annotations. Hypothesis - a > simple graphic will convert better than a complex one. Note: we may need to add > in more product descriptions or the caption bubble to make this look right - > right now there's no feature/product callouts. How important are the product callouts? Simply showing off a screenshot is easy. Adding the callouts will take more work - both with the design and figuring out with the PMMs what we want to highlight. I'd lean towards the simple screenshot for the sake of testing. > 2. Headline: Two different variations, copy only needed since this isn't a > graphic. Would be nice to test the "Your Web" copy against new copy that's more > Firefox/feature oriented. We've discussed something related to Speed - lets > check with the PMMs on this one. Laura, will you start a thread with Mayumi & LauraM? > 3. Language Dropdown: Not sure if we'll need another asset for this but Blake > would like to test putting the dropdown at the top of the page instead of the > bottom. The theory is that this will help our international users easily change > to their default language. Seems like this wouldn't need a new asset just to test it. Blake, what do you think? > Reminder: in the near future we'd like to add back the download counter and > possibly test the effect of putting it in different locations. If we're ready > to do that now, then great. If not, lets keep this in mind. As noted yesterday, the problem is that the current counter is broken. We could always create a new landing page with static results, but I'm reluctant to do that given all the other existing projects. What do you think?
I shouldn't need any additional help. One additional request. Can we change the button text ("Firefox 4 free download") from an image to text?
> (In reply to comment #2) > > 1. Graphic Promo: A product shot without the animated cartoon character - just > > the product screenshot with its respective callouts/annotations. Hypothesis - a > > simple graphic will convert better than a complex one. Note: we may need to add > > in more product descriptions or the caption bubble to make this look right - > > right now there's no feature/product callouts. > How important are the product callouts? Simply showing off a screenshot is > easy. Adding the callouts will take more work - both with the design and > figuring out with the PMMs what we want to highlight. I'd lean towards the > simple screenshot for the sake of testing. A screenshot without callouts is fine - the main aspect of interest is the effect of removing the character. > > > 2. Headline: Two different variations, copy only needed since this isn't a > > graphic. Would be nice to test the "Your Web" copy against new copy that's more > > Firefox/feature oriented. We've discussed something related to Speed - lets > > check with the PMMs on this one. > Laura, will you start a thread with Mayumi & LauraM? Will do! > > Reminder: in the near future we'd like to add back the download counter and > > possibly test the effect of putting it in different locations. If we're ready > > to do that now, then great. If not, lets keep this in mind. > As noted yesterday, the problem is that the current counter is broken. We could > always create a new landing page with static results, but I'm reluctant to do > that given all the other existing projects. What do you think? Agree for now - but lets make sure to add this in once that's fixed. I'll monitor that bug, and update it to reflect this need. Since we know this affects download I don't want it to be forgotten.
(In reply to comment #4) > One additional request. Can we change the button text ("Firefox 4 free > download") from an image to text? I've been considering this anyhow. Unfortunately, we have Meta Black as a WOFF font that we can use to render real text (like the "YOUR WEB"), but not Meta Bold -the font used for the actual Firefox logo/wordmark. We can use MetaBlack text for the "Firefox 4 - Free Download" text, or Georgia, or some combination. Not using an image would be a big with though in terms of how quickly the button shows up. That said, file size difference only a few Kb, and there's no extra HTTP request as the text is part of the same image as the Firefox logo image. Should I try some font options? Perhaps Lee or Sean should have a go at it with the limitation of real-fonts only?
It will definitely change the look of the FF logo since it's stems from Meta Bold. John, would this be considered a no-no?
When I designed the original mockup, I had assumed the button would be pure live text/CSS. We don't use Meta Bold on the current buttons, so I don't think we're tied to it. Perhaps it could be built out and we could look at Meta Black / Georgia options?
Is it possible to test introducing the second WOFF font (Meta BOld) and seeing how that compares to having the image on there? Maybe it will represent an improvement even with the extra WOFF. I'd prefer not to render the Firefox wordmark in Meta Black or Georgia if we can avoid it. We'll certainly do that if need be - having the button load quickly is a big priority - but I'd like to exhaust all other options first.
Adding the second WOFF font (Meta Bold) will add another HTTP request and an additional 43Kb to the total page size. That said, it would be nice to have it to use elsewhere on the page/site. I've got the nova-branch stage site updated with the MetaBlack font in the download button in the mean time to give everyone an idea of what it looks like: http://nova.stage.mozilla.com/en-US/
Oh, and I should have mentioned - if we go this route (real WOFF text for download button), regardless of which weight of Meta, Safari users will see our fall-back font (Trebuchet MS).
Just remembered another requirement. Until all of the pages are finished, we're going to need to all links to point to the current mozilla.com site.
Quick update: Lee is going to mock up a version of what the button would look like with no images (and using a font besides MetaBlack). If that looks good, the plan will be to load the imageless version right away, but have the logo files loading in the background...when they're ready, they can pop in just like the WOFF fonts will. The desired end result of that is that we'll have a button that loads much more quickly, but we'll also have the fully-styled button for people on faster connections.
Here's some input on the copy test variations per PMMs request to test Speed and Customization/Personalization too. Copy Test Variations 1. Current copy - stays as-is. "Your web the way you want it" 2. Personalization centric messaging - right now the "Your web the way you want it" copy is more centered on the web than it is the actual Firefox product itself. Lets try to shift that more directly to the browser - the thing you can touch and customize that's bringing you the web to begin with. 3. Speed centric messaging - without saying we're the "fastest" lets communicate that Firefox is super quick, speedy, fast, etc. In both new instances above lets try to have the visitor leave the page feeling like Firefox equates to either personalization or speed. As always, copy should be direct and clear.
Hi all, Here are 3 versions of the download button in the 3 various sizes they've been appearing on the new pages. John and I seem to agree that it feels better having these appear (before the WOFF load) consistent w/ the general styles of our pages vs. a stylized (but 'not quite right') version of our ideal button, for that split second. http://intothefuzz.com/leetom/Mozilla.com/JPG/01_button_lrg.jpg http://intothefuzz.com/leetom/Mozilla.com/JPG/02_button_med.jpg http://intothefuzz.com/leetom/Mozilla.com/JPG/03_button_sml.jpg Here are 2 links to symbols that I found. It would be great if we could find a downward pointing arrow dingbat that would work with Georgia, although I haven't seen one...maybe you guys know of the command? http://www.mactraining.com/pdf/ZapfDingbatsChart.pdf http://unicode.org/charts/PDF/U2700.pdf
ˇ for Georgia is: http://grab.by/6VzP maybe we could try that? or... ˘ for Meta Black is: http://grab.by/6VA2 or.. > in MB is: http://grab.by/6VzW (maybe rotated? not ideal though...)
I really like the buttons in comment #15...let's roll with those. The Georgia downward carat in comment #16 should be ok, too. Thanks!
Download buttons have been updated with Georgia. It's nice when the solution happens to be the fastest AND easiest. The down-arrow glyphs can't be reliably produced on Windows, so that arrow is an image for now - I can probably sprite it in with the logo image file.
Hey Lee-- All we need to start initial tests on Mozilla.com is a version of the browser graphic with Fx3 instead of 4 - doesn't matter if there's a character holding it or not, either will due to get started. Is it possible to pass this over first once they're done? Thanks!
Hey Laura, I think these are the two versions you seek: http://intothefuzz.com/leetom/Mozilla.com/JPG/home_newusers_02_ff3illo.jpg http://intothefuzz.com/leetom/Mozilla.com/JPG/home_newusers_02_ff3brows.jpg let me know if there's a problem w/ either, Lee
Excellent, thanks Lee! Steven - so we can start our initial testing, can you post this graphic to the Nova staging site? http://intothefuzz.com/leetom/Mozilla.com/JPG/home_newusers_02_ff3brows.jpg That will enable Blake to start testing. Thanks everyone! Laura
Responding to comment #14, here are three variations each for personalization and speed. Laura M/Laura F/Mayumi...please pick a favorite for each and Lee can lay them out into headlines we can test. Personalization (JS note: I don't think any of these are better than "Your web, the way you want it" for this topic, but there's no harm in testing): FIREFOX IS YOURS Customize it how you like. BROWSE YOUR WAY Easy to personalize! IT'S YOUR WEB Personalize Firefox just for you. Speed: YOUR WEB As fast as you need it MORE SPEED Get where you're going quickly. BROWSE BETTER Firefox is faster than ever.
Steven, in terms of the page load improvements and other optimizations you've been working on, is this page now ready for testing? If you're going to be landing any big improvements in the next couple of days it would probably better to wait, but if you feel good about where things are then we should go ahead and get started.
I've committed the screenshot-only version of the feature image. I went with a Windows 7 screenshot instead - assuming that's more common for the test? John, we do hope to do more optimization, but what is there now should be solid enough to test. The download button JS is still a bottle-neck, but that's the same issue we have on the current production site.
(In reply to comment #24) Thanks Steven, Windows 7 vs. Mac shouldn't matter for this test, this is enough to get us going. Thanks! (In reply to comment #22) My copy votes: Agree with John that the personalized copy isn't too different from current 'Your Web...' copy, but to pick one from each these would be my choices: BROWSE YOUR WAY Easy to personalize! -While not my favorite copy, this one feels the most "personalize" to me so I think it's the strongest for testings sake. & MORE SPEED Get where you're going quickly. -Of the three this most clearly communicates speed
(In reply to comment #24) > I've committed the screenshot-only version of the feature image. I went with a > Windows 7 screenshot instead - assuming that's more common for the test? Blake, I have a question about this...right now the alternate version (screenshot only) is on the main staging server, which doesn't feel right. I'd like that to always contain the "regular" content, and the tests should be hosted elsewhere. What's the best way for Steven to deliver you the files you need in a way that doesn't confuse matters on the main staging server? Also, I have a bit of an issue with the way the test version looks...for one, I think it's odd to have "the browser with a mission" overlaid on top of the browser window, b/c it's more of a general statement rather than tied to a particular product feature. Also, that line seems particularly out of place when placed on top of Facebook. Does this bother anyone else? PMMs, let us know about comment #22 or LF's votes in comment #25.
Can we get all test versions deployed somewhere on mozilla.com (like we did with the First Run experiments)? At minimum, I need the images, CSS, and JS hosted.
(In reply to comment #26) > Also, I have a bit of an issue with the way the test version looks...for one, I > think it's odd to have "the browser with a mission" overlaid on top of the > browser window, b/c it's more of a general statement rather than tied to a > particular product feature. Also, that line seems particularly out of place > when placed on top of Facebook. Does this bother anyone else? It doesn't bother me in particular but we want to make sure that whatever we're testing accurately enough reflects the new design we're moving towards. In the other product shots the "Browser with a mission" callout is on top of the character/screenshot, not overlaid. Lets move it up a bit here too. To give it some room, perhaps move the product shot down (since it is rather large now). Lee - can you make this change? Details: -On facebook no monster version move the "Browser with a mission" bubble above the facebook screenshot instead of how it's currently overlaid on top of the screenshot -To make space for it, consider showing less of the screenshot by making it a bit shorter
(In reply to comment #27) > Can we get all test versions deployed somewhere on mozilla.com (like we did > with the First Run experiments)? At minimum, I need the images, CSS, and JS > hosted. Alex, what do you think is the best way to handle this? How can we host files & images without interfering with the work going on at the staging site?
updated version attached here.
After a discussion with John and Alex today, we're recommending we create a static (or mostly static?) copy of this page and put it in mozilla.com trunk (and then stage/production) at an alternate URL (mozilla.com/hometest or something?). This will keep it from getting mixed up with further development of the new design. Also, Alex was going to see if Anthony could take care of that.
Cool. Blake, does comment #31 work for you? Alex, let us know if Anthony can help.
I'm gonna work on this.
Thanks Anthony!
Steven, now that we have the new version set up, can you revert the basic nova site back the way it was (with the illustrations)?
(In reply to comment #37) > Steven, now that we have the new version set up, can you revert the basic nova > site back the way it was (with the illustrations)? Yup - illustrations are back.
Now that the import is done, what alternatives should I do? (Laura or John I think) And how do I set them up for the testing tool? (Blake?)
Thanks Anthony! At this point we may need to build pages that show the alternate headlines (3 in all), but I'm not sure if that's something Blake can do through SiteSpect. Blake - please advise.
Also Blake/Anthony - after chatting with John sounds like the page headlines will need to be built by Anthony. Blake - Is there a specific URL structure we'll need in place for testings sake?
To check things, we are talking about headings described in comment #25 and comment #33 ? Plus the YOUR WEB already in place.
(In reply to comment #42) The three different headlines would be: 1. Current "Your Web" in place 2. BROWSE YOUR WAY Easy to personalize! 3. MORE SPEED Get where you're going quickly.
r76111 to prepare the page for testing from any URL. The page should now be ready to go into SiteSpect. After chatting with Blake, it seems he will do the variations within SiteSpect. We'll need this to go in production so asking for QA. Test URL: http://www-trunk.stage.mozilla.com/en-US/nova-test-596010/
Keywords: qawanted
(In reply to comment #44) > We'll need this to go in production so asking for QA. > Test URL: http://www-trunk.stage.mozilla.com/en-US/nova-test-596010/ Hey Anthony - I think the graphic on staging is using a older version of the screenshot. Can you change it to the ones Lee revised in comment #33? (the callout bubble is on top of the product screenshot instead of overlaping it) Here they are: http://intothefuzz.com/leetom/Mozilla.com/JPG/home_newusers_02_header1.jpg http://intothefuzz.com/leetom/Mozilla.com/JPG/home_newusers_02_header2.jpg (Easy to miss in this long bug) Thanks!
(In reply to comment #44) Hi Anthony - Any progress here? We should be good to start testing after the latest product shot in published here: http://www-trunk.stage.mozilla.com/en-US/nova-test-596010/ Thanks!
This is a two minute change but we'll need a smaller (height) version of the screenshot. Something around 135px less. With the blue light at the bottom. http://www-trunk.stage.mozilla.com/en-US/nova-test-596010/screenshot-browser.png
(In reply to comment #47) > This is a two minute change but we'll need a smaller (height) version of the > screenshot. Something around 135px less. With the blue light at the bottom. > > http://www-trunk.stage.mozilla.com/en-US/nova-test-596010/screenshot-browser.png Lee, can you take this one?
Try this: - reduced 135px from the bottom - shadow is a little hard to match since the png crops the left side of shadow...
The 135px are ok. The shadow is too small I think. If you put the following image next to the screenshot, maybe you can adjust the shadow? http://www-trunk.stage.mozilla.com/en-US/nova-test-596010/creature-browser.png
Thanks. I think this one might work...
Yes, this looks great to me. Blake, is this everything you need to start the tests? Thanks Anthony and Lee!
Looks good to me. Can we push to production?
(In reply to comment #54) > qa-verified trunk http://www-trunk.stage.mozilla.com/en-US/nova-test-596010/ This page has broken links. Under the "Features" navigation, the "Tour" link is dead. The "More ways to connect" link in the footer is dead. We shouldn't be linking to /firefox/stats/, it's broken currently http://www-trunk.stage.mozilla.com/en-US/mobile/Sync/ is dead in the navigation At least those need to be fixed, and this needs to be QA'd again, more carefully.
I talked with Alex and am going to fix/remove the broken links from SiteSpect. Would still be good to fix these problems for future experiments.
I pushed the assets to production
Oops, sorry for that, I also only focused on layout. Links to remove: /firefox/customization/ /firefox/tour/ /firefox/participate/ /firefox/stats/ Links to fix: /mobile/Sync/ (lowercase) /firefox/connect/ Is that correct? Maybe we can find alternatives for the "links to remove".
what's the best way to update this test for Firefox 3.6.12?
Commited r76470. This fixes the dead links and the download button is now pointing to the latest Firefox available.
Keywords: qawanted
Commited r76688. This should improve load time a bit on IE and first render time for every browser. I only have IE6 to test here but it should be tested on a bunch of different IE before going live.
Whiteboard: [ms-11/12]
Forgot one… Fixed in r77456.
OS: All → Other
Fixed with r77459. Also fixing validation issues in other test pages.
Commited r77461. Use absolute URLs to the CDN so that it works with inline CSS.
r77462 on production
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
Is the content intended to overlap with the snow-monster in the footer like this (see attached screenshot)? If not, the culprit is probably a missing "padding-bottom: 80px;" on the "#home #sub-firefox.sub-feature" element". Maybe that was added to the nova branch after this page was branched off.
Commited r77538. Fixed the padding-bottom (and gradient). Thanks for spotting that Steven, I deleted too much CSS and forgot the original design would still run.
Status: RESOLVED → REOPENED
Keywords: push-needed
Resolution: FIXED → ---
I noticed that this test version still includes both rows of the footer - will a version without either/both be tested? We do have a simplified footer for pages that span locales that could be used: http://nova.stage.mozilla.com/en-US/firefox/geolocation/
(In reply to comment #72) > I noticed that this test version still includes both rows of the footer - will > a version without either/both be tested? We do have a simplified footer for > pages that span locales that could be used: > http://nova.stage.mozilla.com/en-US/firefox/geolocation/ Yes, check out this page w/o the footer: http://www-trunk.stage.mozilla.com/en-US/nova-test-596010/curly.html The two versions currently being tested should have that footer (or lack of footer...). Thanks for the geolocation example - we should keep this in mind as an option.
merged to production with r77667
Status: REOPENED → RESOLVED
Closed: 14 years ago → 14 years ago
Keywords: push-needed
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: