1111009 - Add Y! specific conditional content to /firefox/new/ page when referred from Y!

Status: RESOLVED FIXED

(www.mozilla.org :: Pages & Content, defect)

Description

[Chris More [:cmore]]

Attachment: Copy/image winner with 6% improvement in conversion rate for non-Firefox non-mobile traffic from Y!'s universal header promotion.

As of this week, Yahoo! is now promoting Firefox to non-Firefox users in the universal header on many of their primary websites (finance.yahoo.com, weather.yahoo.com, sports.yahoo.com, etc.) and they are directing traffic to:

https://www.mozilla.org/firefox/new/?utm_source=yahoo&utm_medium=referral&utm_content=uh&utm_campaign=y-upgrade-new-firefox

The opportunity:

The Yahoo! experience when using Firefox (their new enhanced search) is pretty darn cool and is available with Firefox 34+. I ran an A/B content test the past few days and found a 6% improvement in download conversion rate when saying "Try Firefox with the new enhanced Yahoo!" (screenshot attached) and showing an example search result.

We should test out some other examples of headlines and body content/images and see what resonates best for Yahoo users. Then after we find out what works, we should make that content conditional on /firefox/new/ when those UTM parameters exist above. Very similar to how we show conditional content based on platform, we can also make adjustments based off of source/medium as value propositions for Yahoo! may be unique enough to talk directly to them.

Let's discuss ideas and get these moving along.

Comment 1

[Jennifer Bertsch [:jbertsch]]

Hey Chris-

This sounds super interesting!  What sort of timeline are you looking at?  

+Matej and a few others

Thx,
Jen

Comment 2

[Jennifer Bertsch [:jbertsch]]

Want to set up time to get started?

Comment 3

[Chris More [:cmore]]

ASAP. Why? We are getting a ton of traffic from Y! now and they are converting much lower than organic search. Any improvements that we can make will help. I am going to continue to run the Optimizely experiment at 100% to keep the conversion rate up by 6% for now.

Comment 4

[Jennifer Bertsch [:jbertsch]]

Let's regroup with me, you, Matej, and Holly on Monday.  I'll find time.

Comment 5

[Chris More [:cmore]]

We can also discuss if we would be interested in adding subject matter content on the page if they come from say sports.yahoo.com vs weather.yahoo.com.

Comment 6

[Chris More [:cmore]]

Let's make sure Winston is in the meeting too as we'll need some strategy on how to talk to these unique users.

Comment 7

[Chris More [:cmore]]

Hi everyone. Here's the plan:

1) We are going to continue with using optimizely to vary the content for these Y! visitors until sometime in January where we have the bandwidth to make changes to the page. This saves us dev time.

2) Matej and team will come up with a few headline adjusts to the page that we can test conversion rates over the next few days.

The default page is: "Choose Independent. Choose Firefox."

The current Y! variation is: "Get Firefox with the new enhanced Yahoo!"

One suggestion was: "Firefox with the enhanced Yahoo! search experience."

3) Habber will find a better screenshot to use on the page.

4) We are going to focus on en-US only for now.

Thanks!

Comment 8

[Chris More [:cmore]]

Habber: Would you have a better Yahoo! screenshot that shows the full browser window and example result? I can photoshop it to the right size after.

Comment 9

[Holly Habstritt Gaal [:Habber]]

(In reply to Chris More [:cmore] from comment #8)
> Habber: Would you have a better Yahoo! screenshot that shows the full
> browser window and example result? I can photoshop it to the right size
> after.

Hi Chris, 
When you say "better", do you just mean better resolution? Is there something else that would make the screenshot better?

Also, when you say the "full browser", do you mean that you want the full page - top to bottom of the browser chrome? A full browser window scaled to fit this page will make elements in the screenshot very small. I think we should keep the top 3rd of the browser as the screenshot, as you have it. Do you agree? I think we want the elements at the top of the Yahoo! search results page and the top Firefox browser to be as large and clear as we can. Are there any other reasons to take another screenshot?

Comment 10

[Chris More [:cmore]]

Hi Holly.

There was some concern yesterday with the German restaurant search and result in the meeting yesterday. I just meant a full browser that shows the chrome as it does now and not just results. I am fine with the current screenshot, but others in the meeting yesterday felt that the German name could confuse some of the en-US visitors. Thoughts on that?

Comment 11

[Chris More [:cmore]]

I am currently running 3 variations of the page:

* original: "Choose Independent. Choose Firefox."
* variation 1: "Now with enhanced Yahoo search experience." w/ y! image (+4.2%)
* variation 2: "Enhanced Yahoo search experience included." w/ y! image (+3.9%)

Since this is an experiment, I am fine with testing more heading to see if any of them resonate better with the audience.

Comment 12

[Chris More [:cmore]]

I am now running a 3rd variation: "Faster Firefox. Better Yahoo.". Laura at Y! is going to come up with a few suggestions to test too.

Comment 13

[Holly Habstritt Gaal [:Habber]]

(In reply to Chris More [:cmore] from comment #10)
> Hi Holly.
> 
> There was some concern yesterday with the German restaurant search and
> result in the meeting yesterday. I just meant a full browser that shows the
> chrome as it does now and not just results. I am fine with the current
> screenshot, but others in the meeting yesterday felt that the German name
> could confuse some of the en-US visitors. Thoughts on that?

Since we are doing variations of copy that are generic to all users entering from Yahoo! (not copy unique to those entering via interest-specific sub-domains), perhaps a simple search term like 'Mozilla' for now.

Comment 14

[Jennifer Bertsch [:jbertsch]]

Chinese red panda is an oldie-but-goodie that also shows off the new SERP 

https://search.yahoo.com/yhs/search?p=chinese+red+panda&ei=UTF-8&hspart=mozilla&hsimp=yhs-001

Comment 15

[Jennifer Bertsch [:jbertsch]]

I thought the German-ish name of the restaurant made the existing image confusing.

Comment 16

[Holly Habstritt Gaal [:Habber]]

'Mozilla' or 'Firefox' don't yield very visual results, like 'suppenkuche' or 'red panda' do. 

Any 'red panda' fans? It's pretty adorable.

Comment 17

[Holly Habstritt Gaal [:Habber]]

I like that searching for a particular business can return a map in the right column, but also Yelp rating, address, phone number. Finding one with these additional data points may make it more clear that it is a location/business than the Suppenkuche example does (even for non-en-us users). I realize that more than en-us can receive this screenshot, but the reasons for using the restaurant example may be more recognizable than the red panda and it also uses the right column nicely. 

Search for a restaurant in NYC http://cl.ly/image/361v3l1E2N1W
Red Panda http://cl.ly/image/0W1V0R3Q0J1E

Travel info could also be recognizable across locales despite the en-us screenshot http://cl.ly/image/0h2I292w0d2m (though the height that the visual takes up is probably too much for the space we have on /new)

Other considerations that work across locales:
- currency http://cl.ly/image/191q0U3J0e2L
- calculation http://cl.ly/image/2X0R2B010N1I

I'm leaning towards my restaurant screenshot that has more datapoints shown than the Suppenkuche example. Does anyone disagree? My version of Parallels is acting up, does anyone else have the means to create a Windows screenshot at the moment?

Comment 18

[Verdi [:verdi] Best to slack me]

(In reply to Holly Habstritt Gaal [:Habber] from comment #17)
> does anyone else have the means to create a Windows screenshot
> at the moment?

I can make some in a little bit.

Comment 19

[Holly Habstritt Gaal [:Habber]]

(In reply to Verdi [:verdi] from comment #18)
> (In reply to Holly Habstritt Gaal [:Habber] from comment #17)
> > does anyone else have the means to create a Windows screenshot
> > at the moment?
> 
> I can make some in a little bit.

Thanks! Let's first see if :cmore or :jbertsch have a preference other than my restaurant/business example.

Comment 20

[Jennifer Bertsch [:jbertsch]]

Thanks, Holly!

I think the NYC restaurant example is great (hopefully I'm not biased since lunch at Mamoun's really saw me through graduate school).  But I like the fact that the name of the restaurant includes the word "restaurant."

Comment 21

[Jennifer Bertsch [:jbertsch]]

PS  Now I want felafel for lunch.

Comment 22

[Verdi [:verdi] Best to slack me]

Attachment: yahoo-windows7.png

Comment 23

[Verdi [:verdi] Best to slack me]

Attachment: yahoo-windows8.png

Comment 24

[Verdi [:verdi] Best to slack me]

Just added some windows screenshots. Let me know if these work or something else is needed.

Comment 25

[Holly Habstritt Gaal [:Habber]]

(In reply to Verdi [:verdi] from comment #24)
> Just added some windows screenshots. Let me know if these work or something
> else is needed.

Thanks, Michael! Looks like the Windows 8 version has a dotted line around the tab title. Any way to remove that?

:Cmore, if you need me to resize the mac version, just let me know. Not sure if the width is ok for you.

Comment 26

[Verdi [:verdi] Best to slack me]

Attachment: yahoo-windows8-2.png

(In reply to Holly Habstritt Gaal [:Habber] from comment #25)
> (In reply to Verdi [:verdi] from comment #24)
> Thanks, Michael! Looks like the Windows 8 version has a dotted line around
> the tab title. Any way to remove that?

Here you go.

Comment 27

[Matej Novak [:matej]]

Hi all,

Sorry for the delayed response. Here are some headline ideas:

Upgrade your browser and the way you search.

A smarter way to browse. A smarter way to search.

More choices from Firefox. Better results from Yahoo.

Get the best of the Web with Firefox and Yahoo.

Now with added Yahoo.

Comment 28

[Holly Habstritt Gaal [:Habber]]

Attachment: yahoo-mac.png

Attached image for mac, so it's easier to find than my cl.ly link. 

:mverdi and :cmore, do you notice how my first result isn't an ad and for the win version it is? Curious why that might be the case. Perhaps because I say "NYC" instead of "New York City"? 

I think it looks nicer without the ad as the first result, though I don't know how open we are to altering the look of the results in photoshop. I don't think we should skew the actual experience. Although, mine did not return an ad.

Comment 29

[Verdi [:verdi] Best to slack me]

Attachment: yahoo-windows7.png

(In reply to Holly Habstritt Gaal [:Habber] from comment #28)
> 
> :mverdi and :cmore, do you notice how my first result isn't an ad and for
> the win version it is? Curious why that might be the case. Perhaps because I
> say "NYC" instead of "New York City"? 

That was it! Here are some new screenshots.

Comment 30

[Verdi [:verdi] Best to slack me]

Attachment: yahoo-win7.png

Comment 31

[Verdi [:verdi] Best to slack me]

Attachment: yahoo-win8.png

Comment 32

[Chris More [:cmore]]

I've posted the Win 7 screenshot. Conversions are looking good. I will be testing Matej's headline tests in comment 27.

Comment 33

[Chris More [:cmore]]

I am now running 8 variations to see what wins:

1) original 
2) Faster Firefox. Better Yahoo. w/new image
3) Enhanced Yahoo Search w/new image
4) The 5 listed in comment 27. 

Thanks!

Comment 34

[Chris More [:cmore]]

I have the results of the A/B test in and one of the original variations we created a few weeks ago is still the most successful by 3%. The "Faster Firefox. Better Yahoo" is the winner and increased the download conversion rate by over 11% compared to the original page. I am going to keep this variation on 100% and the rest of the other variations off.

If you segment by browser, IE is up 14% for the winner variations as compared to Chrome that is only up 7%. 

Thanks everyone!

Comment 35

[Matej Novak [:matej]]

(In reply to Chris More [:cmore] from comment #34)
> I have the results of the A/B test in and one of the original variations we
> created a few weeks ago is still the most successful by 3%. The "Faster
> Firefox. Better Yahoo" is the winner and increased the download conversion
> rate by over 11% compared to the original page. I am going to keep this
> variation on 100% and the rest of the other variations off.

Has Winston or Eric seen this line? I just want to make sure they're comfortable with the messaging since we're not saying "Faster Firefox" anywhere else (I want to make sure we're able to back it up).

Comment 36

[Chris More [:cmore]]

I am confirming with Winston/Eric that they are comfortable with the "Faster Firefox. Better Yahoo" only for English visitors who are coming from the Y! network. Everyone else in the world will continue to get the Independent message.

Comment 37

[Matej Novak [:matej]]

(In reply to Chris More [:cmore] from comment #36)
> I am confirming with Winston/Eric that they are comfortable with the "Faster
> Firefox. Better Yahoo" only for English visitors who are coming from the Y!
> network. Everyone else in the world will continue to get the Independent
> message.

Thanks for letting me know. I have to admit I'm not a huge fan of this line and think it could be stronger. Hopefully there's still room to tweak and revise it at some point.

Comment 38

[Chris More [:cmore]]

(In reply to Matej Novak [:matej] from comment #37)
> (In reply to Chris More [:cmore] from comment #36)
> > I am confirming with Winston/Eric that they are comfortable with the "Faster
> > Firefox. Better Yahoo" only for English visitors who are coming from the Y!
> > network. Everyone else in the world will continue to get the Independent
> > message.
> 
> Thanks for letting me know. I have to admit I'm not a huge fan of this line
> and think it could be stronger. Hopefully there's still room to tweak and
> revise it at some point.

Do you have a suggestion that communicates a similar message, but is stronger?  We tested 10 different variations in 4 separate tests and the "Faster Firefox. Better Yahoo" continues to win by a wide margin. It appears to really resonate with IE users whereas Chrome users is a much smaller margin.

Comment 39

[Chris More [:cmore]]

There is totally room to revise this. I just wanted to capture as many downloads in the short-term until we have a collective strategy and program with Y! and Mozilla.

Comment 40

[Matej Novak [:matej]]

(In reply to Chris More [:cmore] from comment #39)
> There is totally room to revise this. I just wanted to capture as many
> downloads in the short-term until we have a collective strategy and program
> with Y! and Mozilla.

I totally understand that, but I want to make sure our team is at least reviewing any copy like this. I didn't see this line until it was already part of the test.

My main concern is that it isn't really saying much and therefore not working as hard as it could be. Even saying "The fastest Firefox ever. The best Yahoo yet." would be better. That at least gives some context ("faster" is pretty vague), I'm just not sure if we can make the "best Yahoo" claim.