537105 - The War on Review Spam

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Public Pages, enhancement, P3)

Description

[Jorge Villalobos [:jorgev] (he/him)]

We're being hit hard by spammers posting malware links in user reviews. Users are being created every day and posting dozens of reviews in the most popular add-on pages. We're dealing with this manually and it's very time consuming. We need better controls.

I suggest limiting the amount of reviews a user can post in a given time frame. I think 3 per day is reasonable.

This can be bypassed by creating multiple accounts, but maybe with bug 448721 we can also implement IP address tracking and blocking, which I think should be enough.

Comment 1

[Justin Scott [:fligtar]]

I think it's going to be hard to set a limit that legitimate users won't hit. I imagine a pretty common use case is someone writing reviews for all of the add-ons they use at once, which could easily be more than 3.

How about instead if a user has 2 or more reviews that are currently flagged for review, they can't post any more reviews.

Comment 2

[Jorge Villalobos [:jorgev] (he/him)]

It can be an inconvenience for a few users, yes.

The alternative you're proposing won't work because we're trying to prevent spammers that post dozens of reviews within a few minutes. They're most likely done posting with a given account before any reviews even appear on the site.

Comment 3

[Justin Scott [:fligtar]]

Good point. How about:

1. disallow a user from posting the exact same review that they've already posted
2. disallow a user from posting more than 2 reviews per day that contain URLs

Comment 4

[Jorge Villalobos [:jorgev] (he/him)]

Those I like much better. I'd be great if (1) was a little smarter and disallowed posting reviews that are similar within a threshold.

Comment 5

[Andrei Hajdukewycz [:sancus]]

"similar within a threshold" is a pretty vague and potentially complicated statement. Sounds like an e-mail spam filter -- most likely we'd want to look at ready-made alternatives/open source projects/etc. I think that's overly complex to get into when we can try the simple solution. If it fails, we can always revisit the issue, but either way, doing something like that would add a great deal of implementation time to this patch.

Comment 6

[Jorge Villalobos [:jorgev] (he/him)]

You're right, let's keep it simple. How about we implement the 2 points in comment #3, and we add this one:

3) Limit the amount of reviews that can be done in any given day to 10.

Comment 7

[Wil Clouser [:clouserw]]

ha, so now we're tracking total reviews, total reviews with URLs, exact text, and checking whether the current review text has URLs.  Yeah, that's keeping it simple. ;)

Comment 8

[Andrei Hajdukewycz [:sancus]]

1) Query for # of reviews posted in the last 24 hours.
2) Filter current review for URL.
3) Filter $reviews_today[] for URLs.
4) Check if you're breaking URL-reviews limit.
5) Check if current review text matches (last review text? any review text from the last 24hr? -- unclear, but it's pretty easy to defeat last-review-only).

This is the logic that would be running every time someone posts a review. It's very, very far from simple but it seems doable.

Comment 9

[Andrei Hajdukewycz [:sancus]]

I guess you'd want to match text against any review the user has ever posted, that could be a lot of checking for some users though.

Comment 10

[Jorge Villalobos [:jorgev] (he/him)]

You could place a hard limit, like last 20 reviews. That should be quick to check, and most spammers won't worry with that many variations for a message.

Comment 11

[Ngamer01]

I'm just chiming in with an opinion/suggestion, but wouldn't it be better to fight spam bots by modding the registration page with a question a human can answer?  Like "Who develops Firefox?"

A human question will hands-down stop all the bots from getting in without reprogramming.  Though human spammers will still be a concern, but that's nothing a IP ban hammer can't fix.

But if a spam bot manages to be reprogrammed and get an AMO account, an AMO admin could simply change the registration question to bork the bots again.

I do realize a captcha is already present on the registration page, but captchas are useless.  Only registration questions can stop most bots.

If this suggestion needs to go to it's own bug, then I'm fine with that if this bug is only a temporary fix until something more permanent can be done in the future.

Comment 12

[Jorge Villalobos [:jorgev] (he/him)]

I don't think our current problem is being caused by bots. We should also avoid user annoyance as much as possible, and asking a question every time you post a comment is a very annoying.

Comment 13

[Andrei Hajdukewycz [:sancus]]

The suggestion was referring to the registration page -- I assume that means user registration. This bug is about restricting the amount of spam damage an existing user can do, whether they're legitimate(a phished account, for example) or not.

User registration human-verification suggestions should go elsewhere, yes, not in this bug.

Comment 14

[Wil Clouser [:clouserw]]

Kicking to 5.7 because sancus says this adds a bunch of queries and slows stuff down and f that.

Comment 15

[Andrei Hajdukewycz [:sancus]]

I also used up almost all the time I allocated for stuff today helping with security bugs this morning -- that said I'll have a patch for this shortly so there will be lots of time to test and discuss alterations to the rather complex set of new conditions for posting a review.

Comment 16

[Wil Clouser [:clouserw]]

I'm pretty sure wenzel said he wanted to review this!

Comment 17

[Andrei Hajdukewycz [:sancus]]

I'll be happy to grant his request then! :)

Comment 18

[Justin Scott [:fligtar]]

The things mentioned in comment #8 seem sufficient for this bug. Given that this is only taking place when the user submits a review, it shouldn't be a huge deal if the performance isn't the best.

Comment 20

[Jeff Balogh (:jbalogh)]

Please figure out what measures you want to take, and list them in one place.  I'm going to bring reviews over to zamboni soon.

Comment 21

[Jorge Villalobos [:jorgev] (he/him)]

Here's what I want:

1) No user can post more than 10 reviews per day.
2) Reviews can't have URLs (see blocking bug).
3) No user can post a review identical to any of his last 20 reviews (or whatever amount is not too expensive to fetch).

Comment 22

[Wil Clouser [:clouserw]]

(In reply to comment #21)
> Here's what I want:
> 
> 1) No user can post more than 10 reviews per day.

How about once they hit 10 reviews we start asking them to solve a captcha?  If our captchas are broken too easily, that sounds like something else we should be solving anyway.

Comment 23

[Jorge Villalobos [:jorgev] (he/him)]

(In reply to comment #22)
> How about once they hit 10 reviews we start asking them to solve a captcha?

Sounds good to me.

Comment 24

[Justin Scott [:fligtar]]

As mentioned in bug 568458, I think we should hold off on blocking URLs entirely until we can make sure there aren't a ton of legitimate uses for them. I think limiting the user to only posting 2 reviews with URLs in a single day (as summarized in comment #8) is a better safeguard that won't affect legitimate uses.

I don't think captchas will help here -- I think the spam is done by humans who can read captchas.

Comment 25

[Varun]

Let developers flag users as spammers to send all their (site-wide) reviews for moderation. And let them go public only when an editor approves them.

Comment 26

[Jeff Balogh (:jbalogh)]

http://github.com/jbalogh/zamboni/commit/189c397

Adding a page that picks up on 
1) More than 10 reviews in the past 30 days
2) Any urls in a review
3) Exact match between review text

Comment 28

[Jorge Villalobos [:jorgev] (he/him)]

Reclassifying editor bugs and changing to a new whiteboard flag. Spam, spam, spam, spam...