720870 - Survey WMP article visitors

Status: VERIFIED FIXED

(support.mozilla.org :: General, defect, P2)

Description

[:ibai]

We need to survey the visitors of the following article:

https://support.mozilla.org/en-US/kb/Using the Windows Media Player plugin with Firefox

The requirements are:
- Survey for a day to get a sample of a 10%. We have 25k+ visits a day. 
- The survey should be triggered at the beginning of the visit.
- Include an entry text that says: "Please, help us improve your experience with Firefox by answering these 2 simple questions".
- Questions:
* "What do you need help with?"
* "Why did you installed a toolbar in Firefox?"
- If the data collected after one day is not enough or doesn't have enough quality we will need to run the survey again the following day.

For this, we can use Ethnio, but if there's any other solution we are not married to it.

Comment 1

[James Socol [:jsocol, :james]]

From the technical side, I think the cheapest way to do this is use ethnio for the survey and the 10% sampling, and put a dirty hack somewhere to check a waffle switch && this is the right KB article to include the <script> tag.

We should file a bug to remove the hack immediately.

Comment 2

[Kadir Topal [:atopal]]

Ibai, can you tell me more about the background of this? What is the time frame? It was not on our backlog for the current sprint, so depending on the scope we might have to wait for the next sprint. And about the survey, do you already know what tool should be used for that? As far as I know ethnio doesn't offer its own survey tool, but I might be wrong.

Comment 3

[:ibai]

Kadir: https://support.mozilla.org/en-US/forums/contributors/708110
Regarding Ethnio i'm not 100% familiar with it but: http://ethn.io/tour. There's a forma that we can trigger. That's enough.

Comment 4

[Kadir Topal [:atopal]]

Ibai, that's for recruiting, I'm not sure if it does the kind of reporting you are looking for.

Crystal, do you know if Ethnio allows reporting with the tool that is actually supposed to be used for recruiting?

Comment 5

[:ibai]

I'm aware that Ethnio in general is for recruiting but for recruiting it offers a survey that you can customize, it seems, as much as you want and then you have this view where you can see all the answers and export the data:

http://cl.ly/0h130h1Z1B193A0R1u10

To me it seems like a simple workaround. Developers know how to integrate it and it's a tool that has been already approved. Even if it's not as flexible as other tools designed for this, it's good enough for a quick analysis.

Comment 6

[:ibai]

Survey questions revisited:

1- Briefly describe what you where looking for on this page. (Limit answer to 140 characters if possible).
2- Do you have a toolbar that looks somewhat like this http://cl.ly/3V2V1V2v3j3l0c1X2C2k? (Yes/No answer. It will be ideal if we can include the image in the form)
If #2 is yes show the next question.
3- Why did you install that toolbar? (Radio buttons/Single answer)
* Translation features
* Radio features
* Other features (games, social, video)
* I didn't intentionally install the toolbar

If the answer to #2 is NO: Finish the survey by saying thank you and show the article.

If the answer to #3 is "I didn't intentionally install the toolbar": Finish the survey by saying thank you and offer a link to https://support.mozilla.org/en-US/questions/746530

If the answer to #3 is any other: Finish the survey by saying thank you.

-----

Let me know if you need anything else from us.

Comment 7

[Crystal Beasley [:skinny, :crystal]]

Ethnio does have screener questions, but isn't intended to be used as a survey. It doesn't have branching logic that would allow you to show other questions conditionally. That said, we could easily direct them to surveymonkey or some other such provider.

Comment 8

[:ibai]

Do we need to approve SurveyMonkey for internal use? That was the reason why we tried to keep in simple with Ethnio.

If it's going to be a hassle to get SurveyMonkey to be approved we can simplify the survey. We are looking for basic information here, the proposal in Comment #6 is the ideal scenario.

Comment 9

[Ricky Rosario [:rrosario, :r1cky]]

This isn't making it in 2012.3

Comment 10

[:ibai]

Please, let me know if I can help this to move forward. 

The blockers I see today are: 
- Usage of SurveyMonkey or any other surveying tool. Do we have any of this approved by Security/Privacy and such?
- Implementation of Ethnio and it's connection to the Survey. Ricky, do you need anything from me/someone else to make it happen?

Comment 11

[James Socol [:jsocol, :james]]

(In reply to Ibai from comment #10)
> - Usage of SurveyMonkey or any other surveying tool. Do we have any of this
> approved by Security/Privacy and such?

https://bugzilla.mozilla.org/enter_bug.cgi?product=Legal&component=Privacy

You should file a privacy bug. They'll need links to the ToU and Privacy Policy, but also ask if SurveyMonkey, or anything else you could use, is already OKed.

Comment 12

[Ricky Rosario [:rrosario, :r1cky]]

(In reply to Ibai from comment #10)
> - Implementation of Ethnio and it's connection to the Survey. Ricky, do you
> need anything from me/someone else to make it happen?

What do you need from as far as code changes? If I understood correctly, we just need to add a new flag that, when enabled, puts the ethnio script into this one specific article (en-US only?). Is there anything else you need from me? I haven't done that because it seems there were doubts of being able to use ethnio for what you wanted.

Comment 13

[:ibai]

Ricky, that's ok. You mentioned that it was easy but I didn't know it was straight forward. Yes, it's only en-US visitors.

I found out that we have a license for SurveyGizmo so there's no need to ask Privacy. I asked for access and will update the bug as soon as it happens. It will be great if Crystal helps me connect Ethnio with it.

Comment 14

[:ibai]

Ricky, please embed the following code in the following article:
https://support.mozilla.org/en-US/kb/Using the Windows Media Player plugin with Firefox

-------
<script type="text/javascript" >document.write('<script src="http' + ( ("https:" == document.location.protocol) ? "s" : "") + '://www.surveygizmo.com/s3/js/816956/4a06d820b568?__ref=' + escape(document.location) + '" type="text/javascript" ></scr'  + 'ipt>');</script><noscript>This survey is powered by SurveyGizmo's <a href="http://www.surveygizmo.com">online survey software</a>. <a href="http://www.surveygizmo.com/s3/jsfallback/816956/4a06d820b568">Please take my survey now</a></noscript>
-----

This is not an Ethnio code. We will be using SurveyGizmo directly.

Comment 15

[:ibai]

The survey is programmed to run the 29th of February. If the code is not going to be implemented by them, please let me know so I can modify it.

Comment 16

[James Socol [:jsocol, :james]]

Is it possible to get JS for this that isn't synchronous and using document.write?

Comment 17

[:ibai]

James, I couldn't find anything on their documentation so I opened a ticket. I will update the bug as soon as I heard anything.

Is this a blocker?

Comment 18

[James Socol [:jsocol, :james]]

Not right now but it's 2 different worst practices, and if we want to use their code in the future (once we've flipped on CSP) then it will be.

Comment 19

[:ibai]

So regarding the asynchronous code, it can be used but "we need to generate it ourselves" or pay SurveyGizmo to do it for us. They don't offer it out of the box. If we can live without it for now...I guess that is a better option.

The rep was unable to give an answer about the second part without better specifics (she is not technical), so my suggestion is to do it the same way. Let's use this cheap and dirty version for this test and explore a cleaner one if we ever want to run this in a regular way.

Works?

Comment 20

[James Socol [:jsocol, :james]]

(In reply to Ibai from comment #19)
> Works?

Yes, this isn't a hill we need to make a stand on.

But document.write is death to front-end performance (causes painting to restart) and running code inline will only be acceptable until we roll out CSP, and even then it's not _good_ (causes painting to block even if it isn't using document.write).

Comment 21

[Ricky Rosario [:rrosario, :r1cky]]

We can land this on 2/28. I will put it behind a waffle flag so we enable it only the day we need it.

Any preference to placement of this? On the side bar below the YES/NO vote? Bottom of article? Somewhere else?

Comment 22

[:ibai]

Ricky, it should be an overlay on the article. Is that feasible?

Comment 23

[Ricky Rosario [:rrosario, :r1cky]]

I put the script into the page and this is what it displays right now: http://cl.ly/2s1k0U3w2h0Y2G0I3k17

We probably need to open the survey to see how it will really work for users. I am afraid it won't open up in a modal as desired, but there is only one way to find out.

Comment 24

[Ricky Rosario [:rrosario, :r1cky]]

https://github.com/mozilla/kitsune/commit/e812a8a28a4c7d1b5875f10654ef5e45bc2b3815

Waffle flag ("wmp-article-survey") needs to be created before the test begins and removed after. I will file a bug to remove this code afterwards.

QA: Create and enable that waffle flag and go to:
https://support.allizom.org/en-US/kb/Using%20the%20Windows%20Media%20Player%20plugin%20with%20Firefox

Under the title, you should see "This survey is currently closed." or something like that.

Comment 25

[Rebecca Billings [:rbillings]]

Verified able to add waffle flag, kb article displays with 'survey is now closed' message.