Closed Bug 571038 Opened 14 years ago Closed 11 years ago

Assess effectiveness of blocklist level 0 for Adobe Flash

Categories

(Toolkit :: Blocklist Policy Requests, defect)

Product:
Toolkit
Component:
Blocklist Policy Requests
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED

People

(Reporter: rdoherty, Unassigned)

References

Details

Due to the severity of Adobe's Flash exploit we should update the blocklist to message Firefox users their version of Flash is out of date. 

Waiting on an fixed version of Flash we can point users to before doing this.
I don't think we should do this because it is a negative user experience.
(In reply to comment #1)
> I don't think we should do this because it is a negative user experience.

I think I need more reasoning on why it's a negative user experience.
Have you tried it?
I think we should wait on this and see how pushing warnings on first-run/what's new does as far as driving adoption.  The issue with the blocklist is that installing Flash is too hard, and I wonder if users can adequately deal with that warning.
I'm getting a series of screenshots for mac and linux.  If someone can get win 7 that'd help.
I can't even trigger this behavior on mac with a hard-coded blocklist.xml.  :\
Working on Mac and Windows.
Thanks Mike! I'll try on Mac, I know I had problems unless it was actually hosted on a website, not sure if that's what you are doing.
Yeah, I agree the user experience is sucky. I also think that when all 400 million of our users are at risk we sometimes need to use all avenues available to update them. I'm glad you put the screenshots together, it does help.
Thank you for explaining that to me.
OSX screenshot flow is the same after the warning bar http://grab.by/4QLp

Current plan is to utilize firstrun and whatsnew pages for a week and if we don't see a significant number of updates (~30%? I'm following up with metrics to get hard data and to measure this setup) we'll evaluate using the blocklist.
(In reply to comment #12)
> OSX screenshot flow is the same after the warning bar http://grab.by/4QLp

No it's not.  Not even close.

Here is the mac workflow: http://www.flickr.com/photos/morgamic/sets/72157624241445444/
Close as in sucky :)
People won't get the firstrun warning en masse until 3.6.4 (which is Tuesday).  This may affect the marketing of that release (since we doing strong promo on that page for OOPP) so we should make sure to coordinate with that team.
Some thoughts before I head home -- reason why it's important is each step down the funnel destroys conversion.  Each time we take a hit on conversion, the less users actually update.  The less users who actually update because of the difficulty, the less ROI we have on the user disruption caused by the warning itself.

Keep on going down the line and we can get a better handle on how effective we'll be on each platform.  So I'm trying to accurately depict what each step is going to be for that reason -- each click and each level is another barrier -- which is what we covered in the meeting a couple weeks ago (Boriss mentioned it).

So if we put it the decision like this:
* send warning out to 400 million and get 400 million updated -- yes, of course we'd do that.  Do it a million times kthx!
* send warning out to 400 million and get 200 million updated -- yes, but we can achieve roughly the same conversion via product pages (and we'll see about those, which is great)
* send warning out to 400 million and get 50 million updated -- that's hard to justify to me from a product standpoint because your hurt more than you help (debatable of course, but that's how I see it)

Anyway, I'll do Win7 when I get home and we can do some fuzzy math using the last flash check on in-product pages as a control.  Ken also has numbers on roughly how many users we'd lose per step and/or user decision in the funnel and we can mess with that to try to estimate impact.
Windows case: http://www.flickr.com/photos/morgamic/sets/72157624117952339/

Things I found odd:
- have to download an .xpi
- have to opt-out of a mcafee install
- ended up on a random success page
Oh, and:
- have to install the adobe download manager add-on :\
(In reply to comment #15)
> People won't get the firstrun warning en masse until 3.6.4 (which is Tuesday). 
> This may affect the marketing of that release (since we doing strong promo on
> that page for OOPP) so we should make sure to coordinate with that team.

We're adding the update messaging to the 3.6.3/4 pages in bug 570777.
(In reply to comment #16)
> * send warning out to 400 million and get 200 million updated -- yes, but we
> can achieve roughly the same conversion via product pages (and we'll see about
> those, which is great)
> Anyway, I'll do Win7 when I get home and we can do some fuzzy math using the
> last flash check on in-product pages as a control.  Ken also has numbers on
> roughly how many users we'd lose per step and/or user decision in the funnel
> and we can mess with that to try to estimate impact.

http://blog.mozilla.com/metrics/2009/09/16/helping-people-upgrade-flash/

10 million clicks in a week. Even with an awesome upgrade % that is fairly small. Ken, are you able to get estimates on upgrade %'s based on the # of steps in the upgrade path for Flash?
Summary: Blocklist Flash versions <= 10.0.45.2 with outdated severity → Assess effectiveness of blocklist level 0 for Adobe Flash
Blocks: 571295
No longer depends on: 571037
(In reply to comment #20)
> http://blog.mozilla.com/metrics/2009/09/16/helping-people-upgrade-flash/
> 
> 10 million clicks in a week. Even with an awesome upgrade % that is fairly
> small. Ken, are you able to get estimates on upgrade %'s based on the # of
> steps in the upgrade path for Flash?

That warning on the whatsnew page generated a click through rate of >30%, which is spectacular.

Once a Fx user went to the get.adobe.com/flashplayer, we don't have any data how they ultimately converted.  That said, we do know a lot about funnel drop off rates with people downloading/installing Firefox, so based on that, we might guess that somewhere in the ballpark of 25%-50% of people landing at get.adobe.com/flashplayer ultimately update their version of flash.

Keep in mind that conversion rates will vary depending on how the notice/warning is presented within the user experience.  People clicking on the whatsnew warning are making the conscious decision to update flash and it's clearer to them what's going on, so I would expect their flash update conversion rate to be significantly greater than that which would be achieved via the blocklist notice.
That's based on a warning that is flash-specific.  Some variables would be different with the blocklist:
- they'd be on youtube or some other site in the middle of doing something
- the blocklist warning doesn't say anything about flash
- the plugincheck doesn't highlight Flash, it shows all plugins and presents a lot of choices
Here's some ideas for improvements:
- make the blocklist warning plugin-specific
- make a better connection between the initial warning and plugincheck (when we get to plugincheck, make the plugin that prompted the warning way more poignant/highlighted so users don't have to think)
- consider using the same mechanism used in PFS (installerHash/installerLocation) to eliminate 4 steps for users
I guess this is fixed. We already have a blocklisting plan in place for Flash, using the new Click-to-Play experience.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Product: addons.mozilla.org → Toolkit
You need to log in before you can comment on or make changes to this bug.