1396009 - Display badges that indicate the trust/maturity status of an add-on

Status: RESOLVED WONTFIX

(addons.mozilla.org :: Security, enhancement)

Description

[Julien Vehent]

It would be interesting to evaluate the use of badges add-ons authors can obtain to represent the security, trust and overall maturity of their add-on.

For example, badges could represent the fact that:
* an add-on has been reviewed by an experienced add-on reviewer
* an add-on has received external security reviews [1]
* an add-on has been maintained for X numbers of years and received regular updates during that time
* an add-on only uses secure resources (https over http)
* etc.

The thinking is add-ons author would be interested in obtaining these badges to increase user adoption, which would drive the quality of add-ons up over time.

[1] https://github.com/mailvelope/mailvelope/wiki/Security

Comment 1

[Jorge Villalobos [:jorgev] (he/him)]

I'm generally in favor of giving users more data they can use to make an informed decision, but I'm also wary of flags that could give certain add-ons an unfair advantage, or give other add-ons an unwarranted reputation that they are less safe. For example, if a flag says a particular add-on passed a security review, then what about all the others that haven't?

> * an add-on has been reviewed by an experienced add-on reviewer

We're moving to a post-review process for add-ons that use WebExtension APIs, which means a significant portion of future add-ons won't be reviewed by anyone. It's also likely that many add-ons will have past manual reviews but the latest N versions haven't been manually reviewed.

Add-ons will be post-reviewed based on a number of metrics that will tell reviewers how risky they are, so most add-ons that are considered harmless are unlikely to be reviewed.

Filtering it down further to experienced reviewers would make this badge even less common and less useful.

> * an add-on has received external security reviews [1]

Similar to the previous point, only add-ons that are risky or are closer to us would ever have the opportunity of such a review and such a badge.

> * an add-on has been maintained for X numbers of years and received regular updates during that time

We sort of expose this by saying when an add-on was last updated. However, it's also not a very fair metric since some add-ons require little to no maintenance in order to work, and this will only be more true with WebExtension add-ons.

Exposing how long an add-on has existed is a good idea. And maybe instead of showing when an add-on was last updated, we could show if the developer has been active (logged in, uploaded a version) for the past couple of months.

> * an add-on only uses secure resources (https over http)

We plan to validate for this (if we don't already), but we decided we weren't quite there yet to be able to automatically reject add-ons using plain HTTP. In some cases the developer doesn't have a choice because the add-on depends on an external service that doesn't use HTTPS.

Not sure if this is something that an end user will understand, though.

Comment 2

[Daniel Veditz [:dveditz]]

We do show how many users and ratings an add-on has had, which is some help. If you read an article about some great extension and your search finds two with a similar name it's more likely to be the one with 100K users and not 236. You can also somewhat judge by the age of an extension, though it may not be obvious to a lot of users (there's a link in the collapsed "Versions" section).

None of this helps in the very real case when a well-established add-on gets sold or the author's account gets hacked. In that case all these signals _help_ the attacker (or adware purveyor). A badge system would have the same problem. We could reset some badges if we knew of an ownership transfer, but many times the AMO account itself is handed over and we don't know until users complain about the sudden crapification. Chrome Extensions have this problem as well.

Comment 3

[Julien Vehent]

This is very exploratory, and I agree with both of your comments. I actually think badges can be a healthy way to re-balance the ecosystem by showing that a less popular add-on is actually more secure than its more popular competitor. If we only expose popularity, discovery of younger add-ons is very difficult.

Popularity and ratings are only useful for moderately popular add-ons. I often find myself looking for an obscure features that only few add-ons provide. In those cases, popularity and ratings are non-existent, and the user has no indication of the safety of the add-on. Web extension permissions help immensely with that already, and perhaps badges could provide another level of confidence that developers are following best practices, or that the author also wrote some other extension that's very popular, etc.

I suspect it would drive the quality of all add-ons upward because, generally speaking, everyone loves collecting badges. We've seen this happen on observatory.mozilla.org, where people who previously didn't care much for security implemented complex controls like CSP to get an A+.

Comment 4

[Jorge Villalobos [:jorgev] (he/him)]

AMO is moving in a different direction where this sort of distinction won't be necessary anymore, so I'm closing this bug.