Closed Bug 594885 Opened 14 years ago Closed 14 years ago

WebDev implementation of Mozilla.com Survey

Categories

(www.mozilla.org :: General, defect, P3)

Product:
www.mozilla.org
Component:
General
Type:
defect

Tracking

(Not tracked)

Status:
VERIFIED FIXED

People

(Reporter: lforrest, Assigned: sgarrity)

Details

Background:
In the online world, we think we know what website visitors are coming to our sites for, but we are normally wrong. We want to get a better idea of why people come to mozilla.com (beyond download).

That's why we'd like to launch a survey on mozilla.com that asks three simple questions: 1. why they're visiting, 2. if their visit is successful, 3. If not, why not.

WebDev Direction:
-Publish the survey graphic (link below) to the lower right hand corner of all the English language mozilla.com site pages
-Publish only for .05 (5%) of visitors.
-Upon clicking the promo graphic (link to embed: http://www.surveygizmo.com/s3/363180/2c6e18353b96) open survey as a smaller pop-up in front of the user's session on Mozilla.com. 
-If it won't add substantial dev time, please delay the first promo load time by 3 seconds into a user's session instead of right on page load. Giving a user time to navigate through the site will better match the survey questions since one asks "if you've found what you came to the site for". 

Since we only need ~1,400 responses, this graphic only needs to be live for about 2-5 days (will depend on clickthrough). Should I file a separate bug to have this taken down? 

Besides the desired 3 second load delay and the fact that it will be on every page, it should behave like this live Kampyle version here:
http://www.mozilla.com/en-US/products/download.html?product=firefox-3.6.9&os=win&lang=en-US

Graphic Asset: http://intothefuzz.com/leetom/Survey%20Graphic/
Survey Link: http://www.surveygizmo.com/s3/363180/2c6e18353b96
Desired Live Date: ASAP
Duration Live: 2-5 days
Alex, can you help with this?
Hi Alex, I'd like to get this survey up and running this week if possible. Do you have any time in the near future? The webdev work basically consists of publishing a survey promo to 5% of visitors - more of a small task than a project. 

Let me know. Thanks!
(In reply to comment #2)
> Hi Alex, I'd like to get this survey up and running this week if possible. Do
> you have any time in the near future? The webdev work basically consists of
> publishing a survey promo to 5% of visitors - more of a small task than a
> project. 
> 
> Let me know. Thanks!

I don't.  Please work with Mike to find a resource
I bet SilverOrange could whip this up really quick.  CC'ing Steven Garrity.  Steven, anybody have time for this?
We'll work on this. What about the page that already as the survey link (http://www.mozilla.com/en-US/products/download.html?product=firefox-3.6.9&os=win&lang=en-US) - is there a list of pages we should avoid so we don't have two survey links?
Hi Steven, thanks. 

Good question - is it possible to override that current survey (it's a kampyle survey) on download page with the survey here for only those 5% of people this survey will be served to?
(In reply to comment #5)
> We'll work on this. What about the page that already as the survey link
> (http://www.mozilla.com/en-US/products/download.html?product=firefox-3.6.9&os=win&lang=en-US)
> - is there a list of pages we should avoid so we don't have two survey links?

Just talked to the metrics team and it sounds like that other Kampyle survey (on the download page) may be done. Cc-ing Andres who is running that survey to check.
Hi Steven, thanks. 

Good question - is it possible to override that current survey (it's a kampyle survey) on download page with the survey here for only those 5% of people this survey will be served to?
Steven: I confirmed with Andres that the current Kampyle survey on /download is over and can be removed. This survey can take its place, and be published to the other .com site pages as well.
Just wanted to make sure we're all aware that bug 582349 is also slated to go on the bottom right, though not in the precise place.
(In reply to comment #10)
> Just wanted to make sure we're all aware that bug 582349 is also slated to go
> on the bottom right, though not in the precise place.

Good point Stephen - it can overlap a bit, but I it's fine, especially the way the survey graphic moves with the scollbar and fades in.

(In reply to comment #9)
> Steven: I confirmed with Andres that the current Kampyle survey on /download is
> over and can be removed. This survey can take its place, and be published to
> the other .com site pages as well.

Laura, should I remove the Kampyle survey myself, or is someone else doing that? Thanks.
(In reply to comment #11)
> (In reply to comment #10)
> > Just wanted to make sure we're all aware that bug 582349 is also slated to go
> > on the bottom right, though not in the precise place.
> 
> Good point Stephen - it can overlap a bit, but I it's fine, especially the way
> the survey graphic moves with the scollbar and fades in.
> 
> (In reply to comment #9)
> > Steven: I confirmed with Andres that the current Kampyle survey on /download is
> > over and can be removed. This survey can take its place, and be published to
> > the other .com site pages as well.
> 
> Laura, should I remove the Kampyle survey myself, or is someone else doing
> that? Thanks.

Please remove if you're under the hood.
Kampyle survey removed in trunk in r74510.
(In reply to comment #14)
> (In reply to comment #13)
> > Kampyle survey removed in trunk in r74510.
> 
> Verified removed on
> http://www-trunk.stage.mozilla.com/en-US/products/download.html?product=firefox-3.6.9&os=win&lang=en-US,
> etc.

Great, thanks for taking down the old survey. 

Was the new survey published as well?
The survey is implemented in trunk in r74618 and r74619. It currently shows up 100% of the time (for testing). It was set to show up only 5% of the time, but it's changed to 100% of the time (on all en-US pages) in trunk in r74620. for testing.

Over to QA for testing.
Keywords: qawanted
(In reply to comment #16)
> The survey is implemented in trunk in r74618 and r74619. It currently shows up
> 100% of the time (for testing). It was set to show up only 5% of the time, but
> it's changed to 100% of the time (on all en-US pages) in trunk in r74620. for
> testing.
> 
> Over to QA for testing.

It's not 100% for me on trunk, but I haven't spent the time to count and figure out the % (hard-refreshing a bunch of times, it's clear it's pretty low); while it's fine and completes properly when it appears, we should figure out what's going on with that.
Stephen, I should have mentioned - the survey sets a cookie to avoid hitting you over and over. If you delete the cookie you should get the survey again.
Please make sure during QA that this survey does not appear on the First Run or What's New product pages - those are technically mozilla.com pages, but since they're automatically shown to people we don't want to expose that group of users to this survey.
(In reply to comment #19)
> Please make sure during QA that this survey does not appear on the First Run or
> What's New product pages - those are technically mozilla.com pages, but since
> they're automatically shown to people we don't want to expose that group of
> users to this survey.

Thanks for the heads-up, Laura -- I just tested manually on:

http://www-trunk.stage.mozilla.com/en-US/firefox/3.6/firstrun/
http://www-trunk.stage.mozilla.com/en-US/firefox/4.0b5/firstrun/
http://www-trunk.stage.mozilla.com/en-US/firefox/3.6/whatsnew
http://www-trunk.stage.mozilla.com/en-US/firefox/3.6.9/firstrun/

We look to be fine -- the survey pop-up is an include from the footer, which none of those pages have.

Note, though, that because of that, it also doesn't appear on pages like Plugin Check: http://www-trunk.stage.mozilla.com/en-US/plugincheck/; does that matter?
Assignee: nobody → steven
Keywords: qawanted
(In reply to comment #19)
> Please make sure during QA that this survey does not appear on the First Run or
> What's New product pages - those are technically mozilla.com pages, but since
> they're automatically shown to people we don't want to expose that group of
> users to this survey.

Laura, I can confirm that the survey won't appear on the "in-product" pages like firstrun or whatsnew. The code lives in with the main footer part of the site template, so any page that doesn't have that (firstrun, whatsnew, etc) doesn't have it.
(In reply to comment #20)
> (In reply to comment #19)
> > Please make sure during QA that this survey does not appear on the First Run or
> > What's New product pages - those are technically mozilla.com pages, but since
> > they're automatically shown to people we don't want to expose that group of
> > users to this survey.
> 
> Thanks for the heads-up, Laura -- I just tested manually on:
> 
> http://www-trunk.stage.mozilla.com/en-US/firefox/3.6/firstrun/
> http://www-trunk.stage.mozilla.com/en-US/firefox/4.0b5/firstrun/
> http://www-trunk.stage.mozilla.com/en-US/firefox/3.6/whatsnew
> http://www-trunk.stage.mozilla.com/en-US/firefox/3.6.9/firstrun/
> 
> We look to be fine -- the survey pop-up is an include from the footer, which
> none of those pages have.
> 
> Note, though, that because of that, it also doesn't appear on pages like Plugin
> Check: http://www-trunk.stage.mozilla.com/en-US/plugincheck/; does that matter?

No, that does not matter.
This has been merged from trunk to stage in r74752. Alex, please push.
r74756 on tags/production
Status: NEW → RESOLVED
Closed: 14 years ago
Keywords: push-needed
Resolution: --- → FIXED
I or someone else can verify this when they see the survey appear, since it's set to 5% :-)
Laura, maybe you could just confirm that we've seen responses?  (Instead of any of us waiting/refreshing for that 5% chance of seeing one.)

Thanks!
I've got a SurveyGizmo account and the responses are indeed accumulating for this survey. I think we're all good.
(In reply to comment #27)
> I've got a SurveyGizmo account and the responses are indeed accumulating for
> this survey. I think we're all good.

Thanks, Steven.  Verified FIXED.
Status: RESOLVED → VERIFIED
(In reply to comment #26)
> Laura, maybe you could just confirm that we've seen responses?  (Instead of any
> of us waiting/refreshing for that 5% chance of seeing one.)
> 
> Thanks!

Yes, I can verify that we are seeing over 100 responses a day, which is within the targeted range we were aiming for. At this rate we should be done with the survey mid next week and able to remove it from the site. Should I file a separate bug for that, or update this bug once I know more?
Status: VERIFIED → RESOLVED
Closed: 14 years ago → 14 years ago
(In reply to comment #29)
> (In reply to comment #26)
> > Laura, maybe you could just confirm that we've seen responses?  (Instead of any
> > of us waiting/refreshing for that 5% chance of seeing one.)
> > 
> > Thanks!
> 
> Yes, I can verify that we are seeing over 100 responses a day, which is within
> the targeted range we were aiming for. At this rate we should be done with the
> survey mid next week and able to remove it from the site. Should I file a
> separate bug for that, or update this bug once I know more?

Separate bug, please; thanks!
Status: RESOLVED → VERIFIED
A question about how this was implemented: In staging I see the survey here http://www-trunk.stage.mozilla.com/en-US/about/participate/ but not here: https://www-trunk.stage.mozilla.com/en-US/ (even when I clear cookies). Is it showing to visitors to mozilla.com as well, as intended? 

This will effect how we interpret survey results. Thanks!
(In reply to comment #31)
> Is it showing to visitors to mozilla.com as well, as intended?

Yes it is. Good question though. The trunk staging site has some partially implemented optimization changes that are breaking some javascript. These changes don't affect Stage or Production though.
Component: www.mozilla.org/firefox → www.mozilla.org
Component: www.mozilla.org → General
Product: Websites → www.mozilla.org
You need to log in before you can comment on or make changes to this bug.