1194729 - Project Review Bug: Implement Akismet for spam blocking on MDN

Status: RESOLVED FIXED

(mozilla.org :: Project Review, task)

Description

[Justin Crawford [:hoosteeno] [:jcrawford]]

Initial Questions:

Project/Feature Name: Implement Akismet for spam blocking on MDN
Tracking  ID:1188029
Description:
We would like to integrate with the Akismet API[0] to scan revisions on MDN for spammy content. 


[0] http://akismet.com/development/api/
Additional Information:
Here is the justification and specification for this work: https://docs.google.com/document/d/1RrUvVmISM1GHW81su3idAH92q4bTR_rSeyhKNwXWRxc/edit#
Key Initiative: Business Support
Release Date: 2015-10-01
Project Status: development
Mozilla Data: Yes
Mozilla Related: Mozilla Developer Network (MDN)
Separate Party: Yes
Type of Relationship: Vendor/Services
Data Access: Yes
Privacy Policy: https://automattic.com/privacy/
Vendor Cost: <= $25,000

Comment 1

[Justin Crawford [:hoosteeno] [:jcrawford]]

Bug 1194730 - Security Review: Implement Akismet for spam blocking on MDN
Bug 1194731 - Legal Review: Implement Akismet for spam blocking on MDN
Bug 1194732 - Data Compliance Review: Implement Akismet for spam blocking on MDN

Comment 2

[Eric Shepherd [:sheppy]]

Where are we on this project? I haven't heard any status updates on it in a while now, and the goal had been to go live with it in Q3.

Comment 3

[Justin Crawford [:hoosteeno] [:jcrawford]]

(In reply to Eric Shepherd [:sheppy] from comment #2)
> Where are we on this project? I haven't heard any status updates on it in a
> while now, and the goal had been to go live with it in Q3.

We didn't agree about the spec or the project management team in time to launch it in Q3 (we didn't finalize the latter until 8/31). The current status is...
* Legal conversation between Akismet and Mozilla mature and ongoing
* Code well underway: https://bugzilla.mozilla.org/show_bug.cgi?id=1188029#c4

Those are the two major items blocking launch.

Comment 4

[John Whitlock [:jwhitlock]]

The blockers have been dealt with, and the project as specified shipped. Closing this bug.

Akismet and Mozilla Legal reached an agreement around 11/16/15, and an API key was generated on 11/20/15.  The bill for "Akismet Enterprise Vip Annual Subscription" arrived on 3/1/2016.

The Akismet code was shipped to production in February 2016. Further improvements have been shipped to the site in February and March, including sending a training set of historical "ham" content (bug 1248476).  However, the site continues to accept spam, and appears to be under attack from an automated, human-assisted spam machine.

There is an "out of scope" section in the project specification [1] that includes future work items. Some of these have been completed. Others should be tracked in new bugs.

* Users who attempt TKTKTK spam posts in TKTKTK minutes are automatically banned
* An editor should be able to see all the posts that were flagged as spam in a dedicated dashboard (the “quarantine”), and to click a button that says, “not spam”. When the editor clicks this button…
  * notify the Akismet API so it can learn from our content
  * do not try to automatically merge the revision into the article
  * jezdez is opposed to this, because of the resources required to store spam data
* An editor should be able to click a button next to an accepted post in the revision dashboard called, “spam”. When the editor clicks this button…
  * notify the Akismet API so it can learn from our content
  * do whatever we might normally do in that circumstance (revert? ban? delete?)
    * For newly-created articles, delete the article and ban the contributor
    * For spam added to or replacing existing content, revert the change and ban the user
  * tracked in bug 1221967, shipped to site
* Apply Akismet to additions/updates on user profiles
* If an Akismet error occurs, put a flag on the revision record that indicates an Akismet error occurred, with that information noted in the revision dashboard. 
* Every time a Trusted Writer makes an edit, send it to Akismet as ham.

[1] https://docs.google.com/document/d/1RrUvVmISM1GHW81su3idAH92q4bTR_rSeyhKNwXWRxc/edit?ts=56ddcca4#heading=h.kf3wowhlkh9l

Comment 5

[Jannis Leidel [:jezdez]]

FTR, the training set sent to Akismet via bug 1248476 was not "ham" but "spam" by looking up the logically deleted wiki pages that were deleted due to spam.