Closed Bug 375649 Opened 17 years ago Closed 16 years ago

Comment/Review policy for Remora

Categories

(addons.mozilla.org Graveyard :: Policy, defect)

Product:
addons.mozilla.org Graveyard
Component:
Policy
Type:
defect
Priority:
Not set
Severity:
blocker

Tracking

(Not tracked)

Status:
VERIFIED FIXED
Milestone:
4.x (triaged)

People

(Reporter: fligtar, Assigned: baz)

References

(
URL
)

Details

Comments/reviews are moderated by default in Remora. We need to establish a policy for both users and editors that explains what a good review contains and what we consider a bad review (one that should not be approved), with specific criteria for denying reviews.
Justin, when you say "moderated by default," do you mean "approved by default"? That's what I'm led to believe by some of the reviews I've read.  The one that caused me to come looking for a bug related to review moderation or the ability to flag a review is the "Macstripe = ****" gem at https://addons.mozilla.org/en-US/firefox/reviews/display/4784  The author of that review gives the theme a zero rating based on 2 or 3 small bugs and manages to include the URL to his personal theme development site twice.  Definitely not what I'd consider to be a helpful review; at best, it belongs in the discussion forum.

Reviews don't appear until they are approved by an editor. Reviews aren't supposed to be approved at all until this policy is created, but they are.
I guess the question is then, who is approving them?
This probably should be a different bug but shouldn't we have a log of who approved/rejected what when?  Like we had under v2.  
(In reply to comment #5)
> This probably should be a different bug but shouldn't we have a log of who
> approved/rejected what when?  Like we had under v2.  
> 

There wasn't a log of comments in v2, but we do log all of this information in Remora - it's just not accessible to editors. I filed bug 377572 for this.
Would anyone have any objections with me attempting to write a policy for us to use so there's something for us to discuss?  If its not my place to do so I won't bother.
I was under the impression from shaver that there was one written, it's just not on amo yet.
There's a draft on shaver's wiki page that I'm sure he can link to - but it's not official as far as I know.
Severity: normal → major
There are currently 1500 reviews in the queue.
Severity: major → blocker
Not to mention the 100+ add-ons in the public nomination queue that depends on reviews for approval.  For new add-ons it's pretty much ground to a halt.
Assignee: nobody → shaver
Target Milestone: --- → 3.1
Coming to this a bit late but IMO, since editors are still being limited to a small number of involved and trusted people, we should be able to start moderating the obviously good reviews and the obviously bad reviews.  Not only would this get some movement going, it would show what the grey areas are that need clarification in an official policy.
I was looking at some the other night to do just that, but then I realized that a lot of people are treating them like comments on the old site, and I'm not really sure how we want to deal with them.  There are a lot of empty ones though that can be deleted to reduce the queue size.
It's my understanding that "Discussion" is the replacement for "comments" and that "Reviews" are meant to be real "Performance Evaluations" (as I used the term in an editors email.) So something that you'd expect to see on epinions or cnet. That is, it shouldn't just contain an opinion, it should also have at least one reason for that opinion included.  Some sites (like epinions) have mandatory Pros and Cons fields where the user must enter 3 of each to compose a review.  While suggesting that for Remora would be another bug, I think that's sort of the idea of a "review" here.  It provides useful information that helps other users decide if it is a good extension for them.

Do we email the reviewer like we do when approval requests are denied? If so, the obvious ones we could let them know that their comment should go in the Discussions section and to watch for a Review policy that will be posted soon to explain what makes a good review.
I'm gonna try to publish the draft so we can close this.  Will work w/ Shaver on it.  :)
Assignee: shaver → morgamic
Status: NEW → ASSIGNED
(In reply to comment #15)
> While suggesting that for Remora would be another bug, I think that's
> sort of the idea of a "review" here.  It provides useful information that
> helps other users decide if it is a good extension for them.
This is a good idea, so I filed bug 384023.

> Do we email the reviewer like we do when approval requests are denied? If so,
> the obvious ones we could let them know that their comment should go in the
> Discussions section and to watch for a Review policy that will be posted soon
> to explain what makes a good review.
We don't currently notify people when their review is deleted, but the action is logged in the addon history.  Will look into adding this.


We'll need a review policy for language packs, too. As the security risk of a language pack should be zero, should we track that policy here, or in a separate bug?
No, separate bug, and I don't know if I agree that a malicious lang pack couldn't run privileged script...
Filed bug 391705 on a policy for language packs.
Assignee: morgamic → basil
Status: ASSIGNED → NEW
Target Milestone: 3.1 → 3.2
This doesn't block 3.2, so changing TM, but Baz did some work here:
http://wiki.mozilla.org/Update:Editors/PolicyRevisionsFeb08
Target Milestone: 3.2 → 3.x (triaged)
Closing this bug, we now have a policy at http://wiki.mozilla.org/Update:Editors/ReviewingGuide#Moderating_Reviews.
Status: NEW → RESOLVED
Closed: 16 years ago
Resolution: --- → FIXED
Product: addons.mozilla.org → addons.mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.