1248476 - Train Akismet using historical spam

Status: RESOLVED FIXED

(developer.mozilla.org :: Security, enhancement)

Description

[Will Bamberg [:wbamberg]]

Akismet likes us to train it by feeding it known spam. Since we've been deleting spammy pages for over a year, and have (mostly, I hope) been writing "spam" in the revision comment, perhaps we could use this corpus of known-spam.

To do this we'd need to:
* somehow retrieve all pages deleted over, say, the last year
* that were marked as "spam" in the deletion comment
* somehow feed this to Akismet

Comment 1

[Florian Scholz (Open Web Docs)]

Good thing to use those that are marked with "spam" in the deletion comment, because as templates/macros are currently wiki pages as well, and as we have deleted many of them, be sure to not submit deleted pages that have "Template:" in their slugs to Akismet.

Comment 2

[Will Bamberg [:wbamberg]]

Yes, there are definitely other reasons we delete things, and it would be bad to feed it ham by mistake. I don't know about anyone else, but I'm pretty consistent in using "spam" as the deletion comment for spam pages.

If we were to get a list of deleted pages whose comment was "spam", we could manually scan the titles to confirm that it is all spam. Most of our spammy pages have very spammy titles.

Comment 3

[Jean-Yves Perrier [:teoli]]

I also use spam most of the time and 'spammer' when banning an account.

Comment 4

[Jannis Leidel [:jezdez]]

PR with a management command we can use: https://github.com/mozilla/kuma/pull/3790

Comment 5

[Jannis Leidel [:jezdez]]

This is now merged, waiting for another dev to be around to do a deploy.

Comment 6

[Jean-Yves Perrier [:teoli]]

I think this should be marked as RESOLVED/FIXED

Comment 7

[Will Kahn-Greene [:willkg] ET needinfo? me]

For bugs that are resolved, we remove the security flag. These haven't had their flag removed, so I'm removing it now.