1194771 - Use Akismet's "comment check" API on incoming revisions

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: Editing, enhancement)

Description

[Justin Crawford [:hoosteeno] [:jcrawford]]

When a new revision is posted, let's send all fields to Akismet's comment check API (http://akismet.com/development/api/#comment-check). The actual list of fields may change subject to bug 1194732, but we can probably instrument this without knowing the final set of fields. 

Specifics:

* concatenate text fields (title, body, tags, summary, revision comment, keywords) into the “comment” attribute of the Akismet payload.
* skip Akismet if the user is in the Trusted Writers group
* send a form validation error if akismet returns "spam" (see bug 1194768)
* capture the timestamp and userid of spam posts in a spam log
* make sure the form validation error repopulates the form with content attempted
* if akismet fails to respond in 3 seconds or returns an error, allow the post to be saved -- e.g. default to "ham"

Comment 1

[Jannis Leidel [:jezdez]]

I'd like to volunteer working on this given my involvement in the spec.

Comment 2

[Jannis Leidel [:jezdez]]

I've read through the whole create and edit form code this morning (haven't gotten to the translate worflow yet) and decided that there is a new blocker for this to ship: A refactor of the editing workflow.

There is simply no way around this and I will veto any new feature to the wiki editing like spam detection before we've created a code base that doesn't fall apart when we add stuff like this.

Comment 3

[Eric Shepherd [:sheppy]]

(In reply to Jannis Leidel [:jezdez] from comment #2)
> I've read through the whole create and edit form code this morning (haven't
> gotten to the translate worflow yet) and decided that there is a new blocker
> for this to ship: A refactor of the editing workflow.
> 
> There is simply no way around this and I will veto any new feature to the
> wiki editing like spam detection before we've created a code base that
> doesn't fall apart when we add stuff like this.

I don't outright reject this notion, although I have many questions, including:

* When you say "refactor," are you talking a revamp or a rewrite?

* How long will this take to do?

* Is this all low-level work or is it going to span the entire editing codebase?

* Can the refactor be implemented taking into consideration any outstanding feature requests or logged bugs? Even if they're not immediately fixed, if the refactor is implemented to be easier to implement those changes later, that would be an excellent thing.

Comment 4

[Jannis Leidel [:jezdez]]

(In reply to Eric Shepherd [:sheppy] from comment #3)
> (In reply to Jannis Leidel [:jezdez] from comment #2)
> > I've read through the whole create and edit form code this morning (haven't
> > gotten to the translate worflow yet) and decided that there is a new blocker
> > for this to ship: A refactor of the editing workflow.
> > 
> > There is simply no way around this and I will veto any new feature to the
> > wiki editing like spam detection before we've created a code base that
> > doesn't fall apart when we add stuff like this.
> 
> I don't outright reject this notion, although I have many questions,
> including:
> 
> * When you say "refactor," are you talking a revamp or a rewrite?

Sorry, "refactor" is an ambiguous term - I think "revamp" fits best. I'm going through the existing code and make it less fragile and more fitting best practices for complex forms such as we have. I won't change the application logic itself.

I just had trouble knowing where to add the form based spam detection into our workflow since it's so convoluted. The first goal is to make it not convoluted anymore. I would also guess that it will have a positive impact on the issues we've had with race conditions in this part of kuma.

> * How long will this take to do?

It's non-trivial so it's hard to estimate. Please give me a few days to continue my probing.

> * Is this all low-level work or is it going to span the entire editing
> codebase?

This is localized in the form validation code and the adjunct view code that hands the form data validation. It won't have a visible impact on frontend, ckeditor or data models.

> * Can the refactor be implemented taking into consideration any outstanding
> feature requests or logged bugs? Even if they're not immediately fixed, if
> the refactor is implemented to be easier to implement those changes later,
> that would be an excellent thing.

The refactor is focused on the form validation in the create/edit/translate workflow so I would assume that other enhancements would be straightforward. The lowered implementation complexity and increased readability is a major improvement to this particular part of the code base as it's essentially been untouched for years ever since the first product version. We can't do everything at once though.

Comment 5

[Eric Shepherd [:sheppy]]

:jezdez -- Thanks for your very clear reply. You told me exactly what I needed to know. I appreciate it!