656122 - Time/Position in queue should not be lost when uploading a new version

Status: RESOLVED DUPLICATE of bug 717495

(addons.mozilla.org Graveyard :: Admin/Editor Tools, defect, P1)

Description

[Erik Vold]

It seems that many developers actually hold off on uploading new versions of add-ons when a prev version is still in a review queue, because that would mean that they would lose their time & position in the queue.

Actual Result:
Uploading a new version of a add-on which is in a review queue causes the add-on to be moved to the end of the queue.

Expected Result:
Uploading a new version of a add-on which is in a review queue should not cause the add-on to lose it's position in the queue.

The way things are a developer will upload a version to be reviewed, and the review could take 30+ days, at which point they have a new version fixing a number of issues that are in the version that is waiting to be reviewed, but the developer won't upload the new version because that would mean that they lose their spot in the queue. The downsides to this story are:

- The reviewer is reviewing code that they don't need to. Thus wasting their own time.
- The overall time it takes to clear a queue is greatly increased.
- The overall time it takes to review a single addon is increased.
- The overall time it takes for users to receive addon updates is increased.
- Avg review time is decreased artificially.

Comment 1

[Jorge Villalobos [:jorgev] (he/him)]

I'm not sure if this is true in the current system. Before we switched to having a nomination date field in every version, only updates would have this behavior. Nominations could have new versions uploaded without being moved to the back of the queue. The new system should have this same behavior, and I remember discussing it with someone (Andy, maybe?), but I'm not sure if this is how it works now.

Comment 2

[Wil Clouser [:clouserw]]

-> gkoberger to verify

Comment 3

[Gregory Koberger (:gkoberger)]

This is the exact opposite of another bug I have this milestone, bug 654096.

Currently, the Full Review queue doesn't change with a new version, however the rest do.

Wil and Jorge: which should I do?  This or bug 654096 (which is basically done, and is waiting on a decision as well).

Comment 4

[Jorge Villalobos [:jorgev] (he/him)]

Bug 654096 is about uploading a new file to an already existing version, and applies to updates only (see bug 654096 comment #6).

This bug is about uploading a new version altogether when your version is pending review. I think updates should be moved back if a new version is uploaded, but nominations should maintain their current position. So, if I'm waiting for a nomination to be reviewed and upload a new version, the new nomination should inherit the nomination date of the one before it.

Comment 5

[Gregory Koberger (:gkoberger)]

I know that one is about version and one is about files, but they're essentially the same-- it's still checking the newest file of the newest version, no matter if it's a new version or not.

So, is this (the current workflow on preview) correct?

- For Prelim and Pending, any new file or new version will bump the add-on to the end of the list.
- For Full Review, the nomination date and waiting time will never change.  A user can upload a new version or a new file to an existing version, and the add-on will keep its place in the queue.

Comment 6

[Jorge Villalobos [:jorgev] (he/him)]

- For Prelim *updates* and Pending, any new file or new version will bump the add-on to the end of the list.
- For Full Review *and prelim nominations*, the nomination date and waiting time will never change.  A user can upload a new version or a new file to an existing version, and the add-on will keep its place in the queue.

Comment 7

[Gregory Koberger (:gkoberger)]

I might get something wrong along the way; I don't completely understand how this whole review queue works.  So let me know if I confuse something.

Prelim updates and prelim nominations are both in the "Prelim" queue, and for all intents and purposes there is no difference between the two other than the fact that the former is on the site and the latter is new.

This would mean we would be ordering things in the same queue by different things (nomination date v file creation date), depending on the status?  That seems at best confusing and at worse not even possible via our current setup.

Additionally, prelim nominations don't even have a nomination date.  Only full reviews do.

Naturally, I'm biased because changing this means more work for me.  However, my vote is to leave it the way it is.  By changing this, we're adding in a lot of extra SQL and complexity for something that is what I'd imagine to be a fairly small edge case.

Comment 8

[Jorge Villalobos [:jorgev] (he/him)]

I think we can postpone this fix if it is too complex in the current setup.

A few editors have requested to split the prelim queue into nominations and updates anyway. I initially requested it to be a single queue because I thought that having 4 queues would be too much to keep track of. However, it appears to cause more problems than those it solves.

I'll file a new bug to split the preliminary queue and make it block this bug.

Comment 9

[Erik Vold]

(In reply to comment #6)
> - For Prelim *updates* and Pending, any new file or new version will bump
> the add-on to the end of the list.

Why? my request is that the position in the queue would not be lost in this case too.

Comment 10

[Kris Maglione [:kmag]]

For what it's worth, I don't think this is a small edge case. We get quite a lot of complaints about this, and a lot of requests for expedited reviews after authors realize that uploading multiple new versions have bumped them to the end of the queue n times. It's especially problematic when wait times are longer than a month, in which case many add-ons have significant updates of bug fixes long before they hit the head of the queue.

Comment 11

[Jorge Villalobos [:jorgev] (he/him)]

Now that developers can see their position in the queue, they are more aware of this bug and it is more evident that this is a serious problem. See https://forums.mozilla.org/addons/viewtopic.php?f=21&t=3805 and https://forums.mozilla.org/addons/viewtopic.php?f=21&t=3739 for some complaints about this.

While there are reasons we decided updates should be pushed back in the queue when new versions were uploaded, it looks like the harm is more than the benefit, so I agree with Erik that we should just maintain the position in the queue for all cases.

What we need:
1) Set a date when a version is added to the queue (any queue). I believe every version row has a nomination date column, so that could be used. Otherwise, use a new one.
2) If a new version is submitted and there's a previous version in the queue, the new version should inherit that date so that its position in the queue is preserved.
3) If a new file is uploaded to a version that already has approved files, the date should be reset. This is meant to preserve what we wanted fixed in bug 654096.

Thoughts?

Comment 12

[silverwav]

Hi Guys here is an example and some additional info that should help:

I have just realised that on the page Manage My Add-ons that the information " Last updated: August 8, 2011" is reporting correctly.

To expand on that, I have updated as shown below and the field "Last updated:" did not get updated incorrectly.

So this page is recording the fact that I have been waiting for preliminary review for a month correctly.

So my point being, that the code must be looking at the wrong date but that, surprisingly the correct information is available, hopefully this is a simple fix?

Also perhaps a simple query could be run based on "Last updated:" & "Status: Awaiting Preliminary Review" and this will help identify how may add-ons are affected?

_________
Version 1.0.9 August 29, 2011
Version 1.0.8 August 28, 2011
Version 1.0.7 August 21, 2011
Version 1.0.6 August 20, 2011
Version 1.0.5 August 8, 2011

Latest version: 1.0.9
Last updated: August 8, 2011
Status: Awaiting Preliminary Review
_________

Comment 13

[Gregory Koberger (:gkoberger)]

Jorge, it's been a while and I don't think we came up with a final decision on this. There are conflicting bugs for and against this (originally it worked this way and I "fixed" it to work the way it currently works, and now this bug wants it reverted), so we just need to make a decision and go with it.

So, can you decide what we want to do and sum it up? Additionally, you might want to close this bug and file a separate bug, since long bugs like this are confusing to developers.

Comment 15

[Jorge Villalobos [:jorgev] (he/him)]

Reclassifying editor bugs and changing to a new whiteboard flag. Spam, spam, spam, spam...