1190335 - Automation for reviewing add-ons based on the open extension API

Status: RESOLVED INVALID

(addons.mozilla.org Graveyard :: Add-on Validation, defect, P3)

Description

[Gabor Krizsanits (INACTIVE)]

This is quite a challenging task probably. This bug probably should be a meta. But because of the urgency I would prefer to have something landed relatively early, and then do the rest of the work in follow-ups.

We need a script that analyzes Add-ons and in some cases r+ them in other cases requires manual review for them while giving useful hints for the reviewer why it needs special attention. Or in some case just r- them while giving some useful reason.

There should be probably both static and dynamic checks. 

Even after r+, continuous negative feedback from users should raise a red flag and attract manual review even after the Add-on got an r+ already and are in use. Probably same for performance issues, but not sure how to get there.

- it should check if permissions are really needed
- it should check for possible add injection
- it should check for possible data fishing
- it should do some performance tests
- it should check for dangerous evals

Comment 1

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

This bug is about forming a plan for what to do. Implementation will happen separately and maybe a little later.

Comment 2

[Andy McKay]

We probably need to expand this bug in relation to the validator.

Comment 3

[Andy McKay]

Jorgev, I think we should probably close this bug and start a PRD for this for 2017. Chances are that all this work will be done in the linter or new reviewer tools and not in Bugzilla anyway.

I've added this to the Trello board: https://trello.com/c/oylgAz2L/66-review-queue-evolution so it can go into the PMs queue. All status in this case suck, so picking invalid.