truncate response descriptions before saving

RESOLVED FIXED

Status

Input Graveyard
Submission
P2
normal
RESOLVED FIXED
5 years ago
2 years ago

People

(Reporter: willkg, Assigned: willkg)

Tracking

Details

(Whiteboard: u=user c=feedback p=1 s=input.2013q4)

One of the spam feedback responses had a description that was like 2.5 million bytes long. That's beyond goofy.

We should limit response size. I think the best thing to do is truncate the number of characters to some number before saving to the db.

Questions:

1. What's the maximum number of characters we want to deal with in a feedback response? Does this adequately handle people copy-and-pasting long information like addons lists and technical things and any other valid use cases?

2. When we truncate the number of characters, should we tell the user we truncated his/her response?
Bringing this up because that 2.5 million byte long response prevents me from loading the db dump. Figured I'd just nip that in the bud.

Maybe possible answers for the questions are:

1. 5000 characters?

According to my data dump from yesterday, there are 303 responses that have descriptions over 5000 characters. Over half of those are spam. The rest look like they're including HTML either for spam purposes or as examples of problems they're having. So... Let's say 50 in the entire db are over 5000 characters and are valid.

2. Don't tell the user--just truncate it and move on.

Does that sound ok?

If that's ok, do you want me to do a data migration and fix the existing feedback responses so none of them are over 5000 characters?
Flags: needinfo?(mgrimes)
Flags: needinfo?(cwwmozilla)
Priority: -- → P2
Whiteboard: u=user c=feedback p=1 s=input.2013q4
I like option 1. At least we let the user know there is a limit. I'm sure it's a small use case, but if we truncate there's a chance we could lose something useful at the end of the feedback.

For existing feedback over the limit, let's go ahead and truncate so you can move forward with your work. How's that sound?
Flags: needinfo?(mgrimes)
Flags: needinfo?(cwwmozilla)
(In reply to Matt Grimes [:Matt_G] from comment #2)
> I like option 1. At least we let the user know there is a limit. I'm sure ...

I don't know what "option 1" refers to. There are two questions in the description and comment #1 answers each question.

I essentially need to know what the maximum size should be and if the user's feedback exceeds it, what to do ui/ux-wise. If we want to do something more sophisticated than "just truncate it and move on", then we need to do a bunch of ux work which will add new strings to the form that need to be translated.

Also, pretty sure the 2.5 million byte response will affect you folks, too.
Oooops. Sorry about that. I thought those were two different approaches. I think we should limit to 5000 characters, BUT we should inform the user when they hit the max so they don't continue to type. Ie, box turns red or you just can't type anymore.

If you think there is a significant investment in creating that interaction, then let's "truncate and move on" since this will really only impact a very small number of users.
I think we need to redo the web form. The current one is pretty horrible ux-wise. But that's a big project and requires non-trivial efforts and probably a ux person. Saying that now so we can assume, "at some point in the future, we're going to redo the web form because it's horrible and a huge disservice to users."

Now that that's out of the way, I'll think about whether I can implement an interface for this that's effective and doesn't involve new strings. Maybe a counter like what Twitter does coupled with turning the box red like you suggest is sufficient.

Comment 6

5 years ago
5000 is more than sufficient. Feedback over 5000 chars long represent less than 0.1% of recent feedback (since January because that was easy to query). Limiting to 1000 would cut off 1%.

If we're truncating without warning, I'd cut off at 10K for now to be super conservative but if we do have a way to give feedback to the user, 5000 is a fine limit going forward.
Given all that, I'm going to:

1. add some code to truncate new responses to 10k characters when saving to the db.

2. write up a bug to fix this in the ui so that users have a better idea that we're limiting the number of characters--we can do this when we redo the ui for the feedback form


Does that sound like a good plan?

Comment 9

5 years ago
Plan sounds awesome.
Landed in master in https://github.com/mozilla/fjord/commit/015c82a

I'll push it to -stage and -prod on Monday.
Status: NEW → ASSIGNED
Assignee: nobody → willkg
Pushed it to -stage and verified it works. Then pushed it to -prod.

Marking as FIXED.
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
Duplicate of this bug: 870441
Product: Input → Input Graveyard
You need to log in before you can comment on or make changes to this bug.