780119 - Developer manifest changes are being counted as Reviews

Status: RESOLVED WONTFIX

(Marketplace Graveyard :: Reviewer Tools, defect, P4)

Description

[Andrew Williamson [:eviljeff]]

Since the 2012-08-02 push Developer changes to the manifest (that trigger a re-review) are being counted as Reviewer reviews on both the stats table and the review log.

Its similar to bug 775900, which was due to the cron job adding manifest changes as Reviews, but this time it applies to Developers instead.

This breaks reviewer stats (including the points incentive scheme) and pollutes the app review log.  (It fills up the individual app histories also but I'm not sure if this is fixable without losing information we might need)

I think the fix would be to exclude all Manifest Change type reviews from all the stats and the Review log.

Comment 1

[Andrew Williamson [:eviljeff]]

Attachment: Review log screenshot

Comment 2

[Andrew Williamson [:eviljeff]]

Attachment: Stats table

Comment 3

[Wil Clouser [:clouserw]]

Rob is rummaging around in this code currently

Comment 4

[Rob Hudson [:robhudson]]

Is this when the developer pushes the "refresh manifest" link?
Are these because the manifest refresh then triggered a re-review b/c something changed or broke in the manifest?
Do you want me to look into clearing historical stats? (e.g. I might be able to update the user ID of the reviewer logs to be the "Mozilla" user if the user isn't a reviewer)

Comment 5

[Andrew Williamson [:eviljeff]]

(In reply to Rob Hudson [:robhudson] from comment #4)
> Is this when the developer pushes the "refresh manifest" link?

Yes.

> Are these because the manifest refresh then triggered a re-review b/c
> something changed or broke in the manifest?

It seems to be, which isn't really something that's particularly useful - the developer can't see the entry or the validation link and we would get notified during the daily cron job anyway.  But if its easier, then the re-review trigger can stay - it just shouldn't be counting it as a review for anyone.

> Do you want me to look into clearing historical stats? (e.g. I might be able
> to update the user ID of the reviewer logs to be the "Mozilla" user if the
> user isn't a reviewer)

even if they are a reviewer then pressing refresh shouldn't be a 'review'.  Can you update the log (just for these manifest change type reviews) for everyone to the Mozilla?