428535 - Auto-approve reviews?

Status: VERIFIED FIXED

(addons.mozilla.org Graveyard :: Public Pages, defect)

Description

[Fred Wenzel [:wenzel]]

In bug 398065, morgamic suggested to auto-approve reviews in order not to waste editors' cycles on approving reviews anymore.

The rationale is that most of the reviews we get are are perfectly fine and only a minority are inappropriate.

We could automatically publish the reviews we get, thus making them immediately available, and allowing logged-in users (or maybe the devs only) to flag a review as inappropriate. That'd greatly reduce the number of reviews to require editor attention, giving them more time to review add-on updates etc., and reducing their overall work load.

This is up for discussion so please comment.

Comment 1

[Wil Clouser [:clouserw]]

I wrote an email to amo-editors suggesting this very thing earlier this week on the basis that our policy says if you don't know the language wait for someone that does.  

I thought that this would be a pretty big bottleneck for less common languages, particularly for language packs and dictionaries that are trying to get out of the sandbox.

I didn't send the email because there were no reviews over 11 days old in the queue so I was waiting to see if the ones in Tamil and Arabic would get approved or not. :)

Either way, I'm in favor of this.

Comment 2

[Basil Hashem [:baz]]

I'm very much in favor of this as it'll help editors reduce their review load and we can get more policing from the community.

How would it work? We add a "Report a problem" link (or similar) to every review and then it automatically kicks it into the review queue? Or, is there a voting threshold?

Comment 3

[Wil Clouser [:clouserw]]

(In reply to comment #2)
> I'm very much in favor of this as it'll help editors reduce their review load
> and we can get more policing from the community.
> 
> How would it work? We add a "Report a problem" link (or similar) to every
> review and then it automatically kicks it into the review queue? Or, is there a
> voting threshold?
> 

Could be just for add-on authors for add-ons they wrote?

Comment 4

[Andrew Williamson [:eviljeff]]

I'm in favour of this from a workload point of view but in my experience its far less than most of the reviews I approve.  Perhaps I'm especially harsh but taking out all the blanks, duplicates and reviews saying "where's the fx3 version", "greAT plugin!!" and "I'm getting a hash error when I try to download the file" means I approve about 50%.  
It'll lead to a quality reduction but with over 2000 in the queue (and over 400 add-on updates/approvals) I suppose its unfortunately a price worth paying.

Comment 5

[Andrew Williamson [:eviljeff]]

(In reply to comment #3)
> (In reply to comment #2)
> > I'm very much in favor of this as it'll help editors reduce their review load
> > and we can get more policing from the community.
> > 
> > How would it work? We add a "Report a problem" link (or similar) to every
> > review and then it automatically kicks it into the review queue? Or, is there a
> > voting threshold?
> > 
> 
> Could be just for add-on authors for add-ons they wrote?
> 
If we're leaving retroactive moderation to the authors we probably need to notify them via email there's something they might need to do something about.  

Comment 7

[Basil Hashem [:baz]]

I'd like to nominate this for consideration for AMO 3.4.3 since I see the moderation queue not getting smaller anytime soon. We still need to address the UI aspects of thumbs up/down, report a problem.

Comment 8

[Fred Wenzel [:wenzel]]

I second that -- it's not going to get better any time in the future (in particular after the Fx3 release). Which is awesome. But we need auto-approval for that to be feasible.

I just went through like 200 reviews myself but there are still more left. Basil, should we maybe have our new interns go through a few of them? It won't take long but I think it'd help a lot.

Comment 9

[Fred Wenzel [:wenzel]]

I just filed bug 434048 on how to mark auto-approved reviews as "to be reviewed". Some comments over there would be greatly appreciated. 

Comment 10

[Dave Garrett]

See also bug 379048.  (little old, but the point is still valid)  We should have the review system disabled on public pages for extensions owned by a logged in user.  (same as if not logged in)  This would also fix bug 433501.

Comment 11

[Fred Wenzel [:wenzel]]

Attachment: Auto-approve reviews

This change is quite straightforward: It publishes a review automatically instead of moderating it.

Comment 12

[Michel Gutierrez]

As an extension developer, i find very dangerous to have auto-approved reviews. Back to the previous amo versions, when reviews were not moderated, you often got stupid and unfair comments, like "this extension is ****, check mine at xxx". Things improved a lot on amo 3 since all those mis-leading comments are filtered.

I agree that the review can be flagged, but it will take several days before the comment is removed. By that time, it will have been seen by hundred thousands people, affecting the number of installs if the review says "it's a spyware".

However, i understand the interest of removing workload from the reviewers.

Could we do something like this:

• someone writes a review, it is immediately displayed
• the addon author flags the review, it is immediately removed from public view
• a reviewer may then decide to re-enable the review (if it's a really spyware, the world must know)

Comment 13

[Seth Wagoner]

As another extension developer, I agree with comment #12 and comment #5 - but I would suggest that the email sent to developers provides some guidance on what sort of reviews can/should be reported. Also, when they flag a bad review they should be encouraged/forced to give the reason that they are flagging it. 

Comment 14

[Fred Wenzel [:wenzel]]

The auto-approval is checked into SVN r13425. We need ays to flags reviews (thanks for your comments so far, everybody!), that's bug 434048.

Comment 15

[Stephen Donner [:stephend] Not actively reading bugmail]

Verified FIXED as designed up to this point; my review of the Selenium IDE came through instantaneously:

https://preview.addons.mozilla.org/en-US/firefox/reviews/display/2079.

As we're still working through this functionality, please file new bugs to we can better-scope the fixes/discussions.