811266 - Add article survey to sidebar

Status: RESOLVED FIXED

(support.mozilla.org :: Knowledge Base Software, task, P3)

Description

[Kadir Topal [:atopal]]

Before the redesign we had the article survey in the sidebar in addition to having it at the bottom of the page. The reason was to cover users who didn't scroll down till the end of the article.

The reasoning behind that decision seems apt, so we should reinstate that survey in the sidebar, unless there is good reason not to.

Comment 1

[Bram Pitoyo [:bram]]

My argument for putting the survey in the sidebar is, we’ll get more quantity of feedbacks about the article,.

The argument against the sidebar survey is, the feedback quality might not be as high. People who doesn’t read the article all the way through might not know if the article solves his problem or not.

Before we make the change, I suggest we do a little comparison between:
* The number of article survey when there’s a sidebar
* And the number of survey when there’s no sidebar

From that, I think we’ll know:
* If the number doesn’t differ all that much, then we know that most people don’t use the sidebar
* If the average article rating raises or lowers drastically, then we know whether the sidebar produces a positive or negative net towards the article helpfulness number

Comment 2

[Verdi [:verdi] Best to slack me]

Can't we add a survey in the sidebar and then track the feedback for that survey vs. the one at the bottom?

Comment 3

[Michael Cooper [:mythmon]]

Well, having the survey in the sidebar might promote people leaving feedback without reading the article, as they will see it and use it before scrolling down. We would also need to tweak the survey to be able to differentiate between the different sources. I think it is a good thing to be able to distinguish between the two locations of the survey, but this isn't equivalent to testing with and without the survey in the sidebar.

I think it would be good to do all at  once. Show some people the sidebar, and others not (controlled by a waffle flag set to 50%) and record which form the survey came from. Then we can track if having no sidebar changes the feedback from the bottom survey, and if the feedback left in the sidebar is significantly different than the feedback at the bottom.

Comment 4

[Kadir Topal [:atopal]]

Distinguishing the two sources makes sense to me. 

Here are things we'd be looking for:

1. If the two produce identical results and we don't get significantly more reports by adding the survey to the sidebar, let's remove it from the sidebar again to reduce clutter.

2. If the two produce identical results in aggregate (compared to today), and we don't get significantly more reports by adding the survey to the sidebar, let's remove it from the sidebar again to reduce clutter. 

3. If the two produce different results even in aggregate (compared to today), let's leave the sidebar survey.

4. If we get significantly more reports by adding the sidebar survey, let's leave it.

Is there anything else we should be looking for?

Getting more reports won't do much for English (hello central limit theorem), but many languages have very few votes and increasing them would decrease the confidence interval of the helpfulness chart for those languages quite a bit.

Comment 5

[Bram Pitoyo [:bram]]

Attachment: article survey - sidebar

The article survey sidebar will have a white background color, similar to the Related Articles list.

The thing that makes this sidebar different from the one found on the bottom of the article is that the .ajax-vote-box div will be positioned below the “Yes” and “No” buttons, not on top of them.

Comment 6

[Bram Pitoyo [:bram]]

Forgot to note that I agree with Kadir’s suggestions and measurements on comment 4. Let’s go with it, and decide if it’s helpful or beneficial later.

Comment 7

[Ricky Rosario [:rrosario, :r1cky]]

We are reusing the survey styles, just might need some tweaks. ~1pt

Comment 8

[Michael Cooper [:mythmon]]

Pull: https://github.com/mozilla/kitsune/pull/983
Landed in: https://github.com/mozilla/kitsune/commit/5c957f32c7995a1de78e08a62d1e508cfd449758
Deployed to prod.

Comment 9

[Kadir Topal [:atopal]]

Mike, can you explain how we can distinguish the two sources when counting submissions?

Comment 10

[Ricky Rosario [:rrosario, :r1cky]]

(In reply to Kadir Topal [:atopal] from comment #9)
> Mike, can you explain how we can distinguish the two sources when counting
> submissions?

There is an entry in the vote metadata table with key="source". The possible values are {footer, sidebar, mobile}. Or something like that