[k] Don't show "x out of y people" if not moderator/admin

RESOLVED INCOMPLETE

Status

support.mozilla.org
Questions
RESOLVED INCOMPLETE
8 years ago
5 years ago

People

(Reporter: djst, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

8 years ago
Created attachment 460813 [details]
Helpfulness TMI

In the normal case, people will likely use the Helpful button more often than the Not Helpful button. This means that the general case for replies will either be no votes at all, or one or more helpful votes.

To reduce the visual noise in the forum, let's not show the full "x out of y people" unless x!=y. If x=y, just show "x people found this reply helpful" instead.
It doesn't seem like noise, but helpful information to me. It says that 100% of people found the answer helpful. "X" without "out of Y" doesn't really tell you anything.

Chowse: thoughts? Too much info-noise?
Target Milestone: 2.2 → ---

Comment 2

8 years ago
I was in favor of getting rid of "out of Y" entirely for non-moderators.  Yes, it's more information but all negative information.  We want the "this was helpful" to feel like a thank you not a rating system for community members.
If, for the majority of posts, there are only helpful votes (or no votes at all), then the "X out of X people" would start to feel redundant.

That said, it's a mostly harmless redundancy, and we've yet to see how our users will make use of it. I'd defer this choice until the feature's had some use in the wild.

(In reply to comment #2)
> I was in favor of getting rid of "out of Y" entirely for non-moderators.  Yes,
> it's more information but all negative information.  We want the "this was
> helpful" to feel like a thank you not a rating system for community members.

If that's the case, then we would probably eliminate the "Not Helpful" option altogether-- using it would not result in any perceptible change. My impression was that this would eventually be the foundation of a 'karma' system, not merely kudos. We are already using (helpful - not helpful) votes to identify most helpful replies.
(Reporter)

Comment 4

8 years ago
(In reply to comment #2)
> I was in favor of getting rid of "out of Y" entirely for non-moderators.  Yes,
> it's more information but all negative information.  We want the "this was
> helpful" to feel like a thank you not a rating system for community members.

I agree with Cheng that the "out of Y" part should only be shown to moderators and above -- for that group, this info is definitely relevant. However, it's not relevant to most users and contributors and can instead have a discouraging effect on contributors trying to help.

The idea is to promote the good answers, not necessarily point out the unhelpful posts.
Summary: [k] Don't show "x out of y people" if x=y → [k] Don't show "x out of y people" if not moderator/admin
(Reporter)

Comment 5

8 years ago
(In reply to comment #3)
> If that's the case, then we would probably eliminate the "Not Helpful" option
> altogether-- using it would not result in any perceptible change. My impression
> was that this would eventually be the foundation of a 'karma' system, not
> merely kudos. We are already using (helpful - not helpful) votes to identify
> most helpful replies.

The benefit of still having the "Not Helpful" option is that moderators can identify threads that are getting a lot of down votes. In a future version, we could even surface that in the moderator's dashboard, listing the most poorly performing replies so they can see if there's anything that needs to be done about them.

It could also be used in a future karma system, but exactly how that data would be used we'll have to figure out, as it's a non-trivial task to find the balance between an encouraging and fair karma system.
(In reply to comment #4)
> (In reply to comment #2)
> > We want the "this was
> > helpful" to feel like a thank you not a rating system for community members.

(In reply to comment #5)
> It could also be used in a future karma system, 

These two comments seem to disagree.


> I agree with Cheng that the "out of Y" part should only be shown to moderators
> and above -- for that group, this info is definitely relevant. However, it's
> not relevant to most users and contributors and can instead have a discouraging
> effect on contributors trying to help.

I am not convinced it's not relevant. I'm much more likely to follow advice that's found helpful by "4" people than by "4 out of 100", and following advice with a hidden 96% negative rating may well harm me.

That said, Cheng's pointed out that people are much happier to click on the negative buttons than the positive buttons. That's a much more compelling argument for hiding it.

But given that this has been built based on mockups that were available for months, I think a "wait and see" approach is best right now.
(Reporter)

Comment 7

8 years ago
(In reply to comment #6)
> (In reply to comment #4)
> > (In reply to comment #2)
> > > We want the "this was
> > > helpful" to feel like a thank you not a rating system for community members.
> 
> (In reply to comment #5)
> > It could also be used in a future karma system, 
> 
> These two comments seem to disagree.

Maybe I could have been clearer about how this could be used: one example is to allow admins to see the average Helpful/Not Helpful ratio of each contributor to determine which contributors may need help with improving their support techniques. 

What Cheng points out is that seeing, in public, that your reply got a lot of "Not Helpful" votes may be discouraging. That doesn't necessarily mean that we couldn't use those votes when determining the overall performance of a contributor. Consider this example: which  contributor is most helpful -- the one who has 100 helpful and 1000 unhelpful votes, or the one who has 99 helpful votes and 5 unhelpful votes? As an admin, I'd like to know that, but I don't think it helps foster a healthy community spirit to show this in public.
 
> I am not convinced it's not relevant. I'm much more likely to follow advice
> that's found helpful by "4" people than by "4 out of 100", and following advice
> with a hidden 96% negative rating may well harm me.

Maybe there are cases where this info would be relevant to most people, but what you're describing (4 out of 100) sounds like a pretty extreme edge case. I wouldn't want to design the UX around something that may happen in 1/1000 support threads.

> But given that this has been built based on mockups that were available for
> months, I think a "wait and see" approach is best right now.

Regarding "wait and see", I marked it as P5, so I'm definitely ok with the "wait" part. However, I don't think there's an easy way to measure how contributors feel about having certain unflattering stats exposed to fellow community members, so I don't think we'll be able to "see" much here. I suppose we could survey contributors in the discussion forum about it, though...
We've been testing this in the wild now for almost two years. Time to make a decision. The facts:

Reasons against showing not helpful votes:
* People still click on "not helpful" if they don't like an answer.
* Contributors might be demotivated by the negative votes. I haven't personally heard about this, but it might be worth a quick survey.


In favor of showing the not helpful votes:
* We are using the helpful/not helpful votes in the karma dashboard and need the "not helpful" button to stay for that purpose at least.
* Looking at large numbers the "not helpful" button seems to be used correctly.
* showing the "x of y" helpfulness rating can help contributors understand why their raking is so low.
* Not showing the negative votes might lead contributors to believe that people like the way they post (for example 10 solutions in one post, or only one liners) even though that is not the case, and they get no chance to improve.

My vote is to keep showing the not helpful votes as before. But in either case, this bug is two years old now and it's time to make a decision.
Priority: P5 → --

Comment 9

6 years ago
The way is presented this is more for the end-users than for contributors (users visiting an old thread will see if the answer was helpful in general or not). Contributors may come back to a resolved/old thread once or twice but no more than that (I may be wrong here though).

If we are trying to motivate contributors to improve the ratio, we need a better, more aggregated mechanism (i.e. show the ratio in their profile..for all the answers they have provided). 

For users, the main goal is to show them if an answer has been helpful or not for other users. It feels that the main concern when we opened the bug was that there was too much information to represent this. We could modify it with a simple graph (Thumbs up if the majority of votes are helpful or thumbs down in the majority of votes are negative...perhaps with a threshold where nothing happens).

Just my 2 cents.

Comment 10

6 years ago
I agree with Ibai, I think this is very useful for the end-users as in this way they are able to filter through the "quality" and "less quality" answers.
I also think this is helpful for the contributors to understand their ranking and a good incentive to improve.

If this is demotivating for contributors the only thing I would change is how this is presented in the Karma dashboard or anything that is visible to them. maybe have something like a "helpfulness" ratio an presented in a more positive way. 

Either way I think the way to go is to first do a quick survey and find out how does the community feel about this: do they feel demotivated by the unhelpful votes, do they use it as a way to improve their answers, would they have it presented in any other way. If they are ok with it I say leave it as it is.
I just realized that with the redesign we removed this feature, instead we are only counting the upvotes, we don't show the downvotes at all anymore. That means: even if an answer got 10 up, but 500 down votes, we'll only show the upvotes. 

I can file a bug to go back to the previous behavior, but wanted to check back. Madalina, what do you think?
Flags: needinfo?(madalina.ana)

Comment 12

5 years ago
Reopening this as I just saw it now it was flagged on the wrong email address. I say we leave it as it is for now and try to get the community's pulse on this.
Flags: needinfo?(madalina.ana)
If we need to change anything, file a new bug. This one is a little old.
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → INCOMPLETE
madalina, what did the community say? Are they in favor of hiding all downvotes?
You need to log in before you can comment on or make changes to this bug.