1160593 - Sign beta versions of add-ons

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Code Quality, defect)

Description

[Wil Clouser [:clouserw]]

We are skipping beta versions of add-ons, but we shouldn't be.  

For v1:  This bug is to put any incoming beta add-ons into the preliminary review queues and, as they are reviewed, sign them as preliminary add-ons.

Once automatic validation lands:  We should scan these and if they pass validation we'll automatically sign them.  If this bug is closed before automatic validation let's close this bug and file a new one for this part.

Comment 1

[Mathieu Agopian [:magopian]]

If we need to review beta addons, we need a new status for unreviewed beta addons, something like STATUS_BETA_PENDING, or STATUS_UNREVIEWED_BETA?

Another possibility would be to put any STATUS_BETA file that is not signed (File.is_signed == False) in the review queue, but that might be incoherent with what we're doing with other files/statuses.

Thoughts?

Comment 2

[Mathieu Agopian [:magopian]]

Also, we don't want to review all the existing beta files, right? Only the newly submitted ones?

There's currently 4665 beta files on production.

Newly submitted beta files would thus have the STATUS_UNREVIEWED_BETA status, and go through the prelim review queue (or get automatically reviewed/signed once that's working).

Once reviewed/signed, they'd get the STATUS_BETA status.

Comment 3

[Jorge Villalobos [:jorgev] (he/him)]

The plan is to treat beta versions like unlisted versions, which should be automatically validated and signed. If they don't pass automatic validation then the dev gets the option to send them for manual review, in which case I suppose it makes sense to have a new pending status for it. I'm not sure if sending beta versions to the queue is worth the effort, though. It's possible that it would be a very rare occurrence that we could deal with manually...

Comment 4

[Mathieu Agopian [:magopian]]

:jorgev, are you saying that we could simply not allow the beta add-on to be submitted if it doesn't pass automatic validation?

This means that we wouldn't have to deal with making beta versions reviewable (and thus won't need to be in the review queues).

Comment 5

[Jorge Villalobos [:jorgev] (he/him)]

I think that if there aren't any objections to this, we can try it that way, yes. We should keep in mind that we might need to implement this queuing in the future, if it turns out to be a common problem.

Comment 6

[Rob Hudson [:robhudson]]

Ok, if I understand correctly, this bug is now to do the following:

* If validation passes, sign the beta version with the prelim cert.
* If validation fails, report to user and do nothing further.

In which case, it is making sense to work on bug 1159720 first since it will use similar code/UI patterns.

Comment 7

[Mathieu Agopian [:magopian]]

Attachment: Screen Shot 2015-05-06 at 14.57.12.png

There's something to take into account: when a new version is detected as beta, there's a checkbox (see the screeshot attached) to confirm that it's indeed to be treated as beta.

If this checkbox is unchecked, the file isn't going to be processed as beta, but as a normal file, and the automatic signing/reviewing should not be done. Instead, the file should go through the normal flow.

Comment 8

[Mathieu Agopian [:magopian]]

PR: https://github.com/mozilla/olympia/pull/555

Fixed in https://github.com/mozilla/olympia/commit/17808672a60e5d37f44152a850b5de0696922923