516710 - Improve experimental/sandbox add-on installation confirmation process

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Public Pages, defect, P2)

Description

[Dave Garrett]

There was some discussion in the AMO group about experimental/sandboxed add-ons recently, and I'm beginning to wonder if there isn't something that can be done to improve a few things with the current system. (complete overhauls of the submission process, aside)

Couple of problems:
#1 The experimentalness isn't stressed enough. A user just needs to click through a little checkbox to install something that could be outright malware for all they know.
#2 Reason for experiemntalness isn't stressed enough. Because of #1, the checkbox looks odd and just annoying to many users. I doubt that many users actually click and read the "What's this?" page.

Here is what I'm suggesting:
1. Ditch the checkbox. Instead, on click of the button a lightbox would pop up with a simple explanation of the risks involved here. Have a nice little warning icon and something to the effect of "You're on your own here, be careful", possibly even mentioning the potential risk of malware or bugginess breaking things. The goal is not to scare them completely away, but to just make sure they're aware that they are taking a known risk here. In addition to OK/install and cancel buttons, it could have a "Don't ask me again" checkbox to skip this for the rest of the session, though to some degree we DO wish to annoy users into actually paying attention here, so I'd actually prefer that be left to logged in users.
2. Implement bug 491446. It may be subtle, but simply changing the install button color from green to orange will indicate clearly that this add-on is different from others. Adding an "experimental" label underneath inside the red/pink box, similar to the recommended box, would also help to simply distinguish the difference.

Posting this to request suggestions, criticisms of this and the current system, and any other related discussion.

Comment 1

[Daniel Veditz [:dveditz]]

I heartily agree that the current system is not a clear enough warning for average users who generally don't go around reading our documentation. At the very very least we need to add "unreviewed" to the "experimental" label.

The trust built up in the Addons brand waters down the "experimental" label. If you go into your favorite shop and get an "experimental" muffin you wouldn't be surprised if it tasted bad, but you wouldn't expect there to be a bomb inside because the shop was accepting muffins from any bum who came to the back door.

Personally I would prefer experimental addons to hosted on a completely different domain. It could reuse the same code and database, but should look different and convey "experimental, unreviewed, untested" on every page. People who use our nightlies and beta versions of Firefox will know about the beta-addons site, and average users won't AND THAT'S OK. If the addon author themselves builds up awareness they could link to directly to the beta-addon page and their users could just get it.

If average users were less likely to get these we could even think about reenabling updates. True beta versions of addons probably do have to go through several rounds of testing just as we have several rounds of Firefox betas to get it right. The updates should probably require a light review, though. If we do it well we'd reduce some of the pressure from authors to get their addons out of the sandbox prematurely and let beta addons stay beta.

But failing a separate beta/experimental site we definitely need to expand on the current text which is not scary enough for end-users who don't know that their real-world understanding of "experimental" does not match our use of that word. I'd keep the checkbox though, to make sure they read it.

  "[] I understand that this addon is unreviewed and experimental and
   may be harmful. <a>What are the risks?</a>"

Comment 2

[Eric Jung [:ericjung]]

There's a bug out there which has to do with the Extension Manager supporting multiple channels (sort of like Synaptic Package Manager's support of multiple repositories/software sources). Perhaps it's enough to offer a beta/experimental channel and put the addons in question there.

Comment 3

[Dave Garrett]

(In reply to comment #2)
> beta/experimental channel and put the addons in question there.

A beta channel is a separate but related issue. See bug 526077.

Comment 4

[Jorge Villalobos [:jorgev] (he/him)]

This is something we should consider to fix sooner. I'm referring to changing the wording in the Experimental box so that it is clearer that the add-on hasn't been reviewed in any way.
-> 5.5

The rest are separate issues.

Comment 5

[Justin Scott [:fligtar]]

I'm very much against a separate website. It won't stay a secret -- experimental add-ons are linked to all the time from tech sites and blogs like Lifehacker, and then we'll just be creating more hassle for users who will have to search 2 sites to find out if what they want exists.

However, I am in favor of getting rid of the checkbox in favor of a stronger warning and different color button, and have been for some time. I am thinking something similar to the one we use for self-hosted add-ons: https://addons.mozilla.org/en-US/firefox/addon/5042

I think we'll have to target this for Q1; our remaining scheduled release this year is not a normal release for us, and I wouldn't want to do something major like removing the checkbox in that release. Plus, we should have enough time to get some designs made for what a sandbox add-on's page should look like.

Comment 6

[Wil Clouser [:clouserw]]

-> fligtar for designs/wording/colors/whatever

Comment 7

[Justin Scott [:fligtar]]

The latest status of this bug is that we proposed some changes to experimental add-ons in the Zamboni rewrite that will be designed and implemented over the coming months.

Those include:
* changing "experimental" add-ons to "unreviewed"
* changing the entire style of add-on display pages and search results of unreviewed add-ons to be different from reviewed add-ons
* changing the button color of unreviewed add-ons
* removing the checkbox and having a stronger warning similar to what we have for self-hosted add-ons (possibly in an overlay requiring a second click/acceptance) - see https://addons.mozilla.org/en-US/firefox/addon/6111 for an example of this

Our design firm is starting on this and a few other designs today and it's scheduled to be implemented in AMO 5.8 (launching March 16). I'll post the designs in this bug when I get them.

We've also talked about a few other ideas for sandbox process changes, but those will be in separate bugs/discussions.

Comment 8

[Justin Scott [:fligtar]]

Attachment: mockup of unreviewed add-on display page, v1

This is a mockup of what the add-on display page might look like with the new styling. Note that clicking this button would pop up another warning that would have to be clicked through. (The text will differ between what's shown on the page and what is in the pop-up -- still working that out.)

See bug 544508 for the other install button formats.

Comment 9

[Justin Scott [:fligtar]]

As noted above, I'm going to consider this fixed with the major install button changes described in comment #7 and implemented in bug 544508 for 5.8 in the AMO rewrite. We are still discussing other large changes to the sandbox system, but any ideas that we have for that will be separate bugs and blog posts.

Comment 10

[Jeff Balogh (:jbalogh)]

[bonus] via bug 544508.