692974 - Interface to ACR reports that allows for easy blocklisting

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Admin/Editor Tools, defect, P2)

Description

[Jeff Balogh (:jbalogh)]

"We should have an interface to the Add-on Compatibility Reporter reports that allows for easy compatibility blacklisting of top incompatible add-ons. "
https://wiki.mozilla.org/Features/Add-ons/Add-ons_Default_to_Compatible

This sounds simple, a sketch on a whiteboard will probably suffice.

Comment 1

[Justin Scott [:fligtar]]

I drew this on a whiteboard but I can't turn on the lights to take a picture, soooo:

Compatibility Reports for Firefox 10
---
Incompatible Add-on #1 - 7.*
- 300/315 (95%) incompatible reports [view version] [view reports] [override compat]

==============

Basically, we list the add-ons that we've received more than 10 incompatible reports on with a percentage higher than 80, ordered by that percentage desc.

If an add-on is hosted, we show the latest compatibility info we have for it. If it's not, we say it's not hosted.

Clicking override will go to the block tool with the correct info filled in.

Comment 2

[Justin Scott [:fligtar]]

[6:04 PM] <fligtar> that means the default values when you hit the button
[6:04 PM] <fligtar> maxVersion should be the version of the add-on
[6:04 PM] <fligtar> minVersion should be 0
[6:04 PM] <fligtar> app maxVersion should be *
[6:05 PM] <fligtar> app minVersion should be 10.0a1
[6:05 PM] <fligtar> or whatever the app version we're looking at + a1
[6:05 PM] <fligtar> so add-on version 0 -> current version are blocked in 10.0a1 - *

Comment 3

[Christopher Van Wiemeersch [:cvan]]

https://github.com/mozilla/zamboni/commit/d9e152e

The compatibility index gets updated every three hours. Within the hour the index should be up to date, and the hotness will be viewable:

https://addons-dev.allizom.org/z/en-US/admin/compat

It's also accessible from the admin navigation: Manage Add-ons > ACR Reports

Comment 4

[Justin Scott [:fligtar]]

I'm getting a 500 error on that page. Dunno if I haven't waited long enough or what.

I'm going to be traveling tomorrow morning; Jorge, can you please test this before the push and make sure it does what we expect it to?

Thanks.

Comment 5

[Wil Clouser [:clouserw]]

Unless something critical pops up the push tomorrow is going to be skipped.  Let me know if that's not gonna work for you.

Comment 6

[Christopher Van Wiemeersch [:cvan]]

(In reply to Justin Scott [:fligtar] from comment #4)
> I'm getting a 500 error on that page. Dunno if I haven't waited long enough
> or what.

It's up now: https://addons-dev.allizom.org/z/en-US/admin/compat

Currently, there aren't that many add-ons with both > 10 incompatible reports and > 80% incompatibility - but there are a few.

Comment 7

[Jorge Villalobos [:jorgev] (he/him)]

1) The Override Compat option fills the override form with very aggressive values and saves the override without any confirmation. At the very least, the changes should not be saved immediately; we should have the chance to modify the min/max ranges before the block applies. I also think the ranges should be as conservative as possible: min version and max version should equal the add-on version being blocked, and min app version and max app version should be 7.0a1 - 7.* for 7, 8.0a1 - 8.* for 8, etc.

2) The dropdown currently goes up only to Firefox 10. That's just because the newer appversions haven't been added on dev-, right?

3) If possible, I'd like the dropdown to have an "All apps" option, since some add-ons may go unnoticed because they only get a few reports for each version.

#1 is the only one that worries me, but it shouldn't block the push. Having the dashboard to look at the data is what matters most right now.

Comment 8

[Christopher Van Wiemeersch [:cvan]]

https://github.com/mozilla/zamboni/commit/7035ead 

Items 1 and 2 have been addressed. Item 3 will be addressed in bug 714112.