Closed Bug 985140 Opened 10 years ago Closed 8 years ago

[tracker] Create gradients of happy/sad for the feedback form

Categories

(Input Graveyard :: Submission, defect, P2)

Product:
Input Graveyard
Component:
Submission
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: Matt_G, Unassigned)

References

Details

Attachments

(4 files, 4 obsolete files)

Feedback.jpg
94.00 KB, image/jpeg
Details
Feedback_mobile.jpg
72.97 KB, image/jpeg
Details
Feedback sample.jpg
48.44 KB, image/jpeg
Details
colorful vertical.png
48.03 KB, image/png
Details 
User Story: 

1) As a user leaving feedback on Input, I would like to be able to indicate how strong my feelings are about the feedback.

2) As an Analyst reading feedback, I would like to be able to prioritize issues based on how frustrating they are to users.


In our current feedback system you can only be happy or sad. There is no in between. A user might be mildly upset about one issue and EXTREMELY upset about something else.

Adding some UX folks to get some ideas flowing. I think we would like to tackle this in Q2.
Changing this into a tracker. It'll host the discussion to figure out what we actually want to implement and then we can spin off bugs that block this one to implement those things.
Summary: Create gradients of happy/sad for the feedback form → [tracker] Create gradients of happy/sad for the feedback form
Putting this in 2014q2. Waiting on a plan, still. We had a meeting on friday that is kicking off the relevant work which should yield some options or something like that. Anyhow, it'll get figured out in 2014q2.
Priority: -- → P2
Target Milestone: --- → 2014Q2
Matt: I'm missing a lot of details before I can proceed here.

1. What is this going to look like? 5 stars? 7 starts?

2. Do the stars have equivalent English descriptions? If so, what are they?

3. What should the interface look like? 5/7 stars across? Where does the text go? How does this work on small screens like phones as well as large screens like desktops?

A mock-up here for mobile and one for desktop would be super.

4. Do we want to store the star-rating in the db in a new column (probably yes)? If so, do we want to "summarize" the rating as a happy/sad, too? For example, 1, 2, 3 -> Sad. 4, 5 -> Happy?
Oops--meant to needsinfo? Matt for comment #4.
Flags: needinfo?(mgrimes)
(In reply to Will Kahn-Greene [:willkg] from comment #4)
> Matt: I'm missing a lot of details before I can proceed here.
> 
> 1. What is this going to look like? 5 stars? 7 starts?
> 

We decided 5 is the number since it's a familiar scale to lots of people and allows for a "middle of the road" response.

We can use something like: http://www.ahutton.com/cgw/clipart-hosp/pain%20scale-03-faces-text.jpg 

We will probably want to keep the 2 images on each end of the scale and then come up with a nice "neutral" face.


> 2. Do the stars have equivalent English descriptions? If so, what are they?
> 

I think for now we can skip descriptions. If the icons are done properly, I think it will be intuitive and we won't have to worry about localization. I'm picturing something like "Firefox made me..." with the icons below it? We can play around with descriptions later.

> 3. What should the interface look like? 5/7 stars across? Where does the
> text go? How does this work on small screens like phones as well as large
> screens like desktops?
> 
> A mock-up here for mobile and one for desktop would be super.
> 

I'll get a mock for you today.

> 4. Do we want to store the star-rating in the db in a new column (probably
> yes)? If so, do we want to "summarize" the rating as a happy/sad, too? For
> example, 1, 2, 3 -> Sad. 4, 5 -> Happy?

Yes for a new column. I think summarizing would be cool too actually so we could do quick numbers around pos/neg for things. I think we might want to do:

1 and 2 = Sad
3 = Neutral
4 and 5 = Happy

I could be convinced otherwise though
Flags: needinfo?(mgrimes)
Attached image Desktop and Mobile Landscape.png (obsolete) — 
Desktop and Landscape Mobile mock
Attached image Mobile - Portrait.png (obsolete) — 
Mobile Portrait mock
Terrible mocks attached. Thinking green for positive, blue for neutral, and red for negative. Colors, icons, and layouts all open for feedback.
Awesome--this helps a lot.

One thing is that the current happy/sad is a boolean. It's either true or false and we don't have a neutral value. I'm not sure what to do with that. One way to deal with this is change it to an integer and change all the existing values to a new set of -1, 0 and 1. That's probably a couple of days of work since I need to follow the happy field through the code to make sure it gets the new values correctly. Then Cheng and company have to change their scripts.

Is neutral interesting? Generally, we're tracking mostly sad things. Is it ok to do 1-2 = happy, 3-5 = Sad?
Talked with Cheng and Matt and everyone about what to do with api users.

1. We need to update the API to add a 1-5 column.

2. If there's a happy/sad and a 1-5, then we'll save both to the db.

   If there's a happy/sad, but no 1-5, then we'll save happy=2 and sad=4 
   (i.e. not the extremes).

   If there's a 1-5 but no happy/sad, then we'll save 1-2=happy, 3-5=sad.


After all this lands and we have what the 1-5 scale should look like, we'll create bugs for Firefox for Android and Firefox OS to update the forms they have.
Probably want to use this for faces:

http://bluesock.org/~willkg/images/link/1400180948.png

This comes from the PDF we got from Zhenshuo.
Just spoke to Zhenshuo who was kind enough to offer to do some REAL mockups for us instead of my terrible terrible ones attached here. She says she can have these to us in the next couple of weeks. Adding her to  this bug!
That'd help a ton. Thank you!
Attached image Feedback.jpg — 
Hi all, sorry about the delay here, I was on PTO last week.

Anyway, here's my update, let me know if there's any feedback. I think we should also test whether there's a difference by putting the positive face first vs. putting the sad face first.

Also, background and future plan can be found here: http://people.mozilla.org/~zfang/Others/Feedback0328.pdf
Attached image Feedback_mobile.jpg — 
Mobile version
Awesome work! Thanks so much. How do you feel about removing the text so that we don't have to localize? I that would be easier from an implementation standpoint, but wanted to get your expert opinion.
(In reply to Matt Grimes [:Matt_G] from comment #17)
> Awesome work! Thanks so much. How do you feel about removing the text so
> that we don't have to localize? I that would be easier from an
> implementation standpoint, but wanted to get your expert opinion.

Even if we remove the text, we should still show the description of each "face" on hover, is it about the same amount of work localizing text on the page vs. text on hover?
I think it is the same amount of work. If you think it is important though, it is not a big deal. Thanks again for the help.
Yeah I think showing the description will be really helpful, and we should talk to Matej about the copy or even test different versions of the copy.
I'm on PTO today and tomorrow, but wanted to chime in so the ball keeps moving.

First off, these look great! I'm excited about this.

Second, regarding whether this adds additional translation work, I'm doing the 1-5 gradient changes alongside a bunch of other changes to the feedback forms which will require l10n translation work. So if this requires translation work, that's ok--it'll get done with all the other translation work.

Third, I'll talk to the l10n folks to see what they think about the text in regards to the layout. Some languages use a lot more space to say small things. Need to make sure the layout can work with that.

Fourth, mobile devices can't do "hover". I'm not sure offhand how to accurately distinguish between mobile devices which don't have a pointing device and desktop devices which do these days. It used to be not so hard. If I had my druthers, we'd always show the text and then not have to worry about hover. Is that possible? Where would the text show up on the horizontal mockup? If that's not possible, I'll ask around to see what our options are.

Fifth, we can definitely A/B test the order of the faces (sad to happy vs. happy to sad), but it's harder to A/B test different text because we need to translate the text. If we have all the variations we're testing up front, that'd work fine. But every time we want to try a different set of texts, we have to go through a translation round which takes a while. We could reduce the scope of testing to just en-US users which gets us around translation rounds. Would en-US users work as a A/B test group and can we extrapolate those results to all locales?
Thanks Will for your feedback during vacation!

Regarding your forth point, I agree it's better to show the description text all time instead of using hover over texts. Ideally the text show up below the faces on desktop and on the right of the faces on mobile if we can differentiate different platforms.

A/B testing the order is awesome! And we can A/B test the copy only if we are not sure about them. If people agree the copy is good enough then we don't have to test everything.
I'm fine with the copy and also with showing it at all times since the translation load is the same either way. Looking forward to seeing this in action!
Since we're planning to A/B test here, we need A/B testing infrastructure. So I'm making this block on bug #987209.
Depends on: 987209
Comment on attachment 8414081 [details]
Desktop and Mobile Landscape.png

Marking this obsolete so we don't get confused with this set and the other set of mockups.
Attachment #8414081 - Attachment is obsolete: true
Comment on attachment 8414094 [details]
Mobile - Portrait.png

Marking this obsolete so we don't get confused with this set and the other set of mockups.
Attachment #8414094 - Attachment is obsolete: true
Attached image Input Feedback form idea.png (obsolete) — 
Hey Will,

I've created a mockup of the Input feedback form (please find attached). It is a little different than what has been discussed thus far in this ticket. 

Please let me know what you think.
Hi Krystal! This looks a lot like the in-product Android feedback form. I like the simplicity, but it's missing some of the questions we ask in the generic form on Input. The thing that puzzles me is that it doesn't use the 5-smile rating changes we're making in this bug.

What aspect of your mockups do you want thoughts on?
Attached image Market Research - Inspiration.png (obsolete) — 
I've compiled some of my market research (please find attached). I've also included notes and considerations within the attachment. Hopefully it can help inspire some creative solutions :)
Hi Krystal! It sounds like you want to revisit the research and decisions we've made so far. We're not at that stage anymore. We're pretty settled with a 5-level rating and moderately settled with using smiles like the pain scale.

I'd like to proceed with the decisions we've made so far through the first phase of this project which allows us to see how the changes affect our sentiment numbers. Depending on how that goes, we can revisit some of the decisions as appropriate.

I'm sorry that I wasn't clear enough about where this project was status-wise last week.
Attached image Feedback sample.jpg — 
Hi Will,

Thank you for the clarification (I've made my 2 previous attachments obsolete to help avoid any confusion). 

In light of what you mentioned, I did a quick search online and came across a form (attached above)that could be helpful. I remember it being mentioned that we are trying to also include an accompanying description to the image. The attached form includes large uniform buttons that include both the smiley and description. Of course, imagine the attached form with 5 inputs instead and each button having an associated color gradient. 

Not sure if this is helpful but please let me know if this is the direction of what the ticket is trying to resolve for.
Attachment #8512198 - Attachment is obsolete: true
Attachment #8512233 - Attachment is obsolete: true
Interesting! I like how that looks and feels.

Off the top of my head, we'll probably need to flip the face and text between for rtl languages. Plus we'll have to figure out text for descriptions and make sure that gets translated correctly.

I'm working on redoing the layout for the generic form now to make way for 5-<thing> ratings. I'll try a layout like the one you suggested first. I want to see whether a vertical layout for all screen sizes works well. If so, that'd be great.
Attached image colorful vertical.png — 
Sweet! Oh and I've attached (refer to 'colorful vertical') example. It includes a color gradient per button and has a vertical layout.
Bumping this to 2015q1 since I'm hoping to do the work now.
Target Milestone: 2014Q2 → 2015q1
Grabbing this since I'm working on all the sub-bugs.
Assignee: nobody → willkg
Status: NEW → ASSIGNED
This project has been kicked down the road for a bunch of quarters now. I've gotten small parts of it done, but there's still probably a month of work to do before we can get the first phase done.

There are a number of complexities with this bug:

1. We need a layout for 5 faces that's accessible (screen readers, color blindness, etc), has the same meaning across cultures (faces, colors, wording, etc) and works on large screens and small (desktop, phone, etc). That's a lot of work. It'll involve multiple groups to do and several iterations of mockups etc.

2. Input hosts two feedback forms, but this work is only going to happen on the generic feedback form. We get most of our feedback for Firefox OS, Firefox for Android and Loop through in-product feedback forms. It's a major project to update their forms with this work even assuming they want to do that. I think that suggests we need to have a toggle for showing the multiple faces version or the happy/sad version of the feedback form on a product-by-product basis. That has implementation and maintenance costs.

3. Given that, we'll have feedback that was rated with happy/sad and feedback that was rated with 5 faces. We can probably convert 5-face answers to happy/sad answers, but we probably can't responsibly convert the other way. Given that, our analysis tools need to be implemented/changed to account for this curiosity in the data.

4. There are a lot of outstanding questions like "what do we do on the front page dashboard?", "how does this affect search and the Input API?", etc, that haven't been addressed and would need to be.


The original goals for this project were:

1. As a user leaving feedback on Input, I would like to be able to indicate how strong my feelings are about the feedback.

2. As an Analyst reading feedback, I would like to be able to prioritize issues based on how frustrating they are to users.


The more fine-tuned version of those goals were:

1. We need a better understanding of the severity of feedback. When a user leaves feedback, is the user so angry/sad that they're going to stop using Firefox because of this issue? Which issues change in severity over time? Which issues are "annoying, but no big deal" issues and which issues are "BIG DEAL--STOP THE TRAINS" issues? Maybe it's sufficient to just have a "I'm VERY angry about this" checkbox?

2. We need a better understanding of user sentiment over time which happy/sad doesn't have enough granularity to give us.

3. We want users to feel more confident that they're accurately communicating how they feel.


Heartbeat addresses item 2 much better than we could address it here.

Items 1 and 3 could be solved in other ways that are easier to implement and easier to interpret.


Given all that, we're putting implementation work on this project on hold until we've reassessed things.

(Matt, Cheng, others: Feel free to augment/correct anything I've written here.)
Assignee: willkg → nobody
Target Milestone: 2015q1 → ---
We haven't restarted this project, ergo I'm going to WONTFIX it.
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Resolution: --- → WONTFIX
Product: Input → Input Graveyard
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: