626452 - Review queues are showing wildly incorrect waiting times

Status: VERIFIED FIXED

(addons.mozilla.org Graveyard :: Admin/Editor Tools, defect, P2)

Description

[Jorge Villalobos [:jorgev] (he/him)]

Both the nomination and update queues are showing some incredibly bloated waiting times, many over a year.

Nomination waiting times should be based on the date when the add-on was last nominated, and update times should be based on the date when the last version non-beta version was uploaded.

Comment 1

[Jeff Balogh (:jbalogh)]

Can you point to a specific review item that doesn't look right? It may be better to do this on allizom so the data doesn't change.

Comment 2

[Jorge Villalobos [:jorgev] (he/him)]

(In reply to comment #1)
> Can you point to a specific review item that doesn't look right? It may be
> better to do this on allizom so the data doesn't change.

Irony: bug 627513.

On the live site you can look pretty much at any nomination except the ones on the last pages. They have multi-year waiting times.

Comment 3

[Jorge Villalobos [:jorgev] (he/him)]

Looks like this was lost in time...
Bumping to 5.12.8

Comment 4

[Wil Clouser [:clouserw]]

assuming this is just a math problem, kumar is fixing this in the new queues.

Comment 5

[Kumar McMillan [:kumar]]

Is this still happening on PAMO?  If so, could someone point to a specific addon ID, version ID, or file ID that it's happening for?

I see some wild dates on PAMO but they look legit.  For example, the stephen3000 addon was indeed nominated on 2008-11-13 so perhaps it is old and just never got cleared out.

I'm suspicious of production data because there were a few bugs with nomination date in the new devhub that have been fixed but probably only pushed recently.

Comment 6

[Jorge Villalobos [:jorgev] (he/him)]

The Preliminary Queue shows a few at the top: https://addons.allizom.org/en-US/editors/queue/preliminary

Comment 7

[Kumar McMillan [:kumar]]

(In reply to comment #6)
> The Preliminary Queue shows a few at the top:
> https://addons.allizom.org/en-US/editors/queue/preliminary

Do you mean the Review Policy manager addon?
https://addons.allizom.org/en-US/editors/review/83145?num=1

From the db, this was created on 2008-04-20 23:31:04, the latest version was created on 2009-11-05 02:13:13 and the addon was nominated on 2009-03-16 16:58:38.  Do any of those dates seem wrong?  If not, the wait time of 454 days seems accurate to me.  If the dates do seem wrong then let me know because it could be a bug in the new review process code.

Comment 8

[Jorge Villalobos [:jorgev] (he/him)]

(In reply to comment #7)
> If the dates do seem wrong then let me know because it
> could be a bug in the new review process code.

Yes, that is most likely the problem. The new review code should be setting nomination dates when add-ons are nominated. Apparently, it isn't.

Comment 9

[Kumar McMillan [:kumar]]

On closer inspection and discussion with Jorge, here are the two things that need fixing:

1. When a version is nominated/re-nominated for preliminary review, the nomination date should be set since this is only way to calculate the proper wait time.  Currently it is not set at all.

2. The preliminary queue should calculate wait time based on the greater value of a) nomination date b) version creation date.  This will adjust the wait time accordingly when an update is submitted to a previously nominated version.

Comment 10

[Kumar McMillan [:kumar]]

Forget everything you read in Comment 9!

Here is the real fix: https://github.com/jbalogh/zamboni/commit/6d6016f61df8a1766d523aabd81f8fdf06f4440c

This should fix the scenario we were seeing in the Full Review queue in production after devhub went live:
- Author nominates an an add-on
- It gets rejected
- Author creates/uploads new version and re-nominates the same add-on

NOTE: the above patch does not fix historic data (addons in the current queue).

The prelim review queue, contrary to comment #9, is not a problem because in case a version gets rejected, author uploads a new version and re-nominates for prelim review.  The wait times shouldn't be skewed here because they are based on version creation date.

Comment 11

[krupa raj[:krupa]]

STR:

* Picked an add-on from full review queue with wait time = 15 days
* Rejected the add-on
* Uploaded a new version for that add-on
* Nominated it for full review
* Check the wait time

Wait time is reset. Marking this as verified.

Comment 12

[Jorge Villalobos [:jorgev] (he/him)]

Reclassifying editor bugs and changing to a new whiteboard flag. Spam, spam, spam, spam...