460309 - Create add-ons compatibility dashboard

Status: VERIFIED FIXED

(addons.mozilla.org Graveyard :: Compatibility Tools, defect)

Description

[Michael Morgan [:morgamic]]

We need to create a public-facing page that is responsible for this type of output:
http://people.mozilla.com/~polvi/threedom/status-bars.html

Ideally we could choose which version we wanted to report on, and show the top 95% of add-ons in terms of active users in a similar format.

This page would be instrumental in helping us understand how many add-ons are compatible with new versions of Firefox.

Comment 1

[Justin Scott [:fligtar]]

Attachment: compat main

While I'm taking a break waiting for my local copy of the AMO database from May to update to today, thought I'd post a couple screenshots. Still have a lot left to do but making good progress.

(the graph is using data from may, so no it's not that bad currently. that's why I'm updating my db.)

Comment 2

[Justin Scott [:fligtar]]

Attachment: compat graph

Comment 3

[Alex Polvi]

Attachment: threedom.py

This is the script I used to create that table. 

Usage: 

  python threedom.py < 3dom-latest.txt > status-bars.html

You will need to manage BETA_VERSIONS, BETA_LATEST_VERSIONS, and ALPHA_VERSIONS accordingly. I updated it for 3.1.

Comment 4

[Alex Polvi]

Attachment: get-latest.sh

this was the shell script I ran when I wanted to update the status-bars.html page. I did it by hand ... lame, I know.

Comment 5

[Michael Morgan [:morgamic]]

Thanks, Alex!

Comment 6

[Justin Scott [:fligtar]]

http://fligtar.khan.mozilla.org/addons-compat/site/en-US/firefox/compatibility

Comment 7

[Justin Scott [:fligtar]]

Attachment: Compat Dashboard Main

Comment 8

[Justin Scott [:fligtar]]

Attachment: Compat Dashboard Overall Progress

Comment 9

[Justin Scott [:fligtar]]

Attachment: Compat Dashboard Add-on Details

Comment 10

[Dave Garrett]

Looks nice. I don't suppose you could sub-divide the current versions percentage into Firefox 3, 2, 1.5, and older? It would be useful to show the full picture here. For example, in attachment 345260 [details] the 3rd one from the top (Skype) is still listed as 2.0.* but gets lumped in with the 3.0.* stats. Maybe blue for 3.0.* and different shades of red for the old (soon to be) unsupported versions.

Comment 11

[chris hofmann]

for the script that filters some addons

https://bugzilla.mozilla.org/attachment.cgi?id=344378

I think we still want to use that, but we might also want to make them visible somewhere...    for example  skype was in the orginal report, but then we looked at it, developed a strategy for how to handle (including contacting owners, sometimes getting info in relase notes and support articles, and other stuff...), then filtered it for 3.0.

if we have a growing list of filtered extensions it might lead to us getting surprised by some top extensions that we don't know much about, and won't have had done anything to prepare. 

Maybe we can just put these extensions down at the bottom of the report and an anchor tag at the top of the page to get quickly to that list at the bottom.

Comment 12

[Justin Scott [:fligtar]]

Attachment: patch, v1

This patch adds support for the overall compatibility report and detailed report. I still have a bit of work to do on the other sections and l10n, but we wanted to get this part out in tonight's push. l10n and the other sections should be ready for a push next week.

To test:
1. Add new variables from config.php to your existing config.php or copy over. make sure to define NETAPP_STORAGE to a writable location.
2. Run bin/compatibility_report.php. This will create compatibility.serialized in NETAPP_STORAGE location.
3. Going to /compatibility in your AMO install should now load.

The images used are already checked in.

Comment 13

[Michael Morgan [:morgamic]]

-r=morgamic - it's too sexy

Comment 14

[Justin Scott [:fligtar]]

Hmm, there is a workaround if you disable CSS.

Comment 15

[Michael Morgan [:morgamic]]

I'll try it.

Comment 16

[Michael Morgan [:morgamic]]

Works for me.  Let's get this patch in and we'll slate the rest of this (polish, 2 other report panels, l10n) for 4.0.3.

Comment 17

[Justin Scott [:fligtar]]

r19427

Comment 18

[Justin Scott [:fligtar]]

The first part of this is now live @ https://addons.mozilla.org/en-US/firefox/compatibility

Comment 19

[chris hofmann]

I like it!

It would also be cool if you could add a link next to each addon in the detailed list

so it looks something like

Addon_Name  [link to open bugs]  [version]  [histogram]

Take the name of the addon and plug it into this bugzilla query like it is shown for "Video Download Helper"


https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=Video+DownloadHelper&long_desc_type=substring&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=&resolution=DUPLICATE&resolution=---&emailassigned_to1=1&emailtype1=exact&email1=&emailassigned_to2=1&emailreporter2=1&emailqa_contact2=1&emailtype2=exact&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&field0-0-0=noop&type0-0-0=noop&value0-0-0=

Comment 20

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

This will be tremendously useful for Thunderbird, Seamonkey and Sunbird, other apps based at AMO. Any plans to support them e.g. at:

https://addons.mozilla.org/en-US/thunderbird/compatibility

for Thunderbird for example?

Comment 21

[Justin Scott [:fligtar]]

I looked into adding other applications yesterday and it should be doable after we're done with all of the parts of the Firefox version.

It would help a lot if other applications could get me their official wordmarks with each version they want in the dashboard in the same form as https://addons.mozilla.org/img/wordmarks/firefox-3.0.png

Comment 22

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

(In reply to comment #21)
> It would help a lot if other applications could get me their official wordmarks
> with each version they want in the dashboard in the same form as
> https://addons.mozilla.org/img/wordmarks/firefox-3.0.png

CC'ing Bryan for the TB logo. Simon/KaiRo, you know how to get SM/SB's official wordmarks?

Comment 23

[Robert Kaiser]

Thanks for CCing... I don't think I have created any SeaMonkey artwork with the version numbers in it yet, I guess I need to finally do this for that AMO thing...

Comment 24

[Simon Paquet [:sipaq]]

We don't have any Sunbird artwork with version numbers right now, but could surely create it. At what time would you need it?

Comment 25

[chris hofmann]

polvi also had another report that he produced so reviewers could see most poplular addons in the review queue that had compat version updates.

here is a link to one of those reports
http://people.mozilla.com/~polvi/threedom/3.0-in-queue.csv

and some backround into how it was used...

> Subject: Re: [amo-editors] Add-on Review Push
> > > 
> > > Is that list still being updated daily?
> > > 
> > > Cesar Oliveira
> > > Basil Hashem wrote:
>> > > > As you may have heard, Firefox 3 is getting released this
> Tuesday,
>> > > > June 17th
>> > > >
> > >
> (http://developer.mozilla.org/devnews/index.php/2008/06/11/coming-tuesday-june-17th-firefox-3/).
>> > > >
>> > > >
>> > > > In order to help provide the best first upgrade experience for
> our
>> > > > existing users, I'm requesting that all editors focus on clearing
> > > the
>> > > > "pending updates" queue.
>> > > >
>> > > > To help us prioritize that queue, Alex Polvi has graciously
> > > generated
>> > > > a database dump (which is updated only once a day) with the
> pending
>> > > > update queue sorted by number of active users. Let's try to work
> > > down
>> > > > this list. Understand that you might find the first ones already
>> > > > reviewed since.
>> > > >
>> > > > http://people.mozilla.com/~polvi/threedom/3.0-in-queue.csv
>> > > > Columns: AMO ID, Add-on Name, Add-on Version, MaxVersion, link to
>> > > > review page
>> > > >
>> > > > I understand that this list will be out of date the minute that I
> > > send
>> > > > it out...unfortunately, we don't have a live database lookup at
> the
>> > > > moment. Once we go thru the Editor Tools rework, we'll add these
> > > (and
>> > > > other) types of sorts.
>> > > >
>> > > > Thanks!
>> > > > -- 
>> > > > Basil Hashem
>> > > > [e-mail]
>> > > >
>> > > >
>> > > >

Comment 26

[Michael Morgan [:morgamic]]

->4.0.4 for polish and l10n.

Comment 27

[chris hofmann]

sounds like https://developer.mozilla.org/En/Updating_extensions_for_Firefox_3.1 could be plugged into the "Information for Developers" which now just says "Coming soon!"

Comment 28

[chris hofmann]

anyone know what's up with https://addons.mozilla.org/en-US/firefox/compatibility/

latest version info gone blank.   Is this a problem as we switch over from b2pre to b2?

Archaeopteryx commented that latest version is now maybe 3.2a1pre;  but none of the addons in the compatibility report seem to be reporting that version yet.

if that version info is now available maybe that is used in the calculations as the "latest version"

either way we should figure out how we will handle these conditions and get the report updated.

Comment 29

[Justin Scott [:fligtar]]

Looks like 3.1b2 was added today, and as I mentioned in emails before, as soon as it is added, our compatibility will drop off completely because it's the new standard and no one is compatible with it yet.

To me, this is expected behavior, and I don't really know what else we can do... once we allow developers to use 3.1b2, they should be using 3.1b2 and anything lower is not compatible with the latest.

Whether or not 3.1b2 should have been added today is another story, but it's not undoable now.

Comment 30

[chris hofmann]

when I loaded the page earlier today I saw about 32% of the bar chart colored white.   now that has been replace with 28% light green and 4% white so it looks as expected now.

Comment 31

[chris hofmann]

meant to comment on this before but just getting around to doing it.

what does the histogram on the right side mean and how should it be read?

current dashboard shows increasing size of the bars for each addon with decreasing usage.

java console      --
adblock           ----
video downloader  ------
ie tab            -------


polvi's dashboard showed rough pct. of overall use when compared to other addons in the top 95%.

java console      -----     = 5% of total use
adblock plus      ----      = 4% of total use
video dl helper   ----      = 3.9% of total use
ie tab            ---       - 3% of total use

see http://people.mozilla.com/~polvi/threedom/status-bars.html

to me the povli histogram makes more sense, but maybe I'm just miss reading or not understanding what the current dashboard says.

Comment 32

[Dave Garrett]

(In reply to comment #31)
> what does the histogram on the right side mean and how should it be read?
...
> to me the povli histogram makes more sense, but maybe I'm just miss reading or
> not understanding what the current dashboard says.

Thus bug 463077

Comment 33

[Justin Scott [:fligtar]]

Attachment: Information for Developers

screenshot of new info for developers section

Comment 34

[Justin Scott [:fligtar]]

Attachment: add-on status check

screenshot of status check of my add-ons

Comment 35

[Justin Scott [:fligtar]]

Attachment: Information for End Users

screenshot of new information for end users section

Comment 36

[Dave Garrett]

Comment on attachment 352006 [details]
add-on status check

Looks very nice and I like the idea for the quick adjust maxVersion link. However, I'd be a bit worried that some developers would just click it without actually testing if it will work first. Make sure it yells at them to really test before taking them to the manage version page. ;)

Comment 37

[Justin Scott [:fligtar]]

Ok, new dashboard up for testing @ http://fligtar.khan.mozilla.org/addons-compat/site/en-US/firefox/compatibility

Everything is done except l10n, which I will try to do later tonight.

Note that on khan, it will report 0 active daily users because this week update pings were only parsed for Firefox versions < 3.0 because of the versioncheck.amo log mishap.

Comment 38

[chris hofmann]

the link on khan seems to get 404 for me.

Comment 39

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to comment #38)
> the link on khan seems to get 404 for me.

You have to VPN in to MPT.

Comment 40

[Michael Morgan [:morgamic]]

Any plans on creating a legend?

Comment 41

[Justin Scott [:fligtar]]

(In reply to comment #40)
> Any plans on creating a legend?

A legend for what?

Comment 42

[Michael Morgan [:morgamic]]

For the detailed report.

Comment 43

[Fred Wenzel [:wenzel]]

(In reply to comment #40)
> Any plans on creating a legend?

No need. Fligtar's a legend already.

Comment 44

[Robert Kaiser]

This looks really nice, I'm looking forward to that :)

(In reply to comment #21)
> It would help a lot if other applications could get me their official wordmarks
> with each version they want in the dashboard in the same form as
> https://addons.mozilla.org/img/wordmarks/firefox-3.0.png

For this, I guess it's best to file followups for our apps and get us a clue of what images exactly are needed (e.g. a link to all the Firefox images used, I see you have different sizes there).

I guess we'd be good to go with doing this once the Firefox side has landed or is ready to land, so we know exactly what will be used.

Comment 45

[chris hofmann]

I wonder if it would be more useful and easier to change the horizontal bar chart to show the specific versions rather than the abstract list of "Other Versions", "Alpha Versions", "Beta Versions", and "Latest Version".

Maybe something like


[-3.2a1pre-][3.1b3pre][---3.1b2---][3.1b1][-----3.0.*------][2.0.0.*]
  12%         3%         15%        4%         43%            7%


or maybe histogram format.

Links to a search that showed lists of addons at each specific level would be great too.


And despite fligtar's legendary status we should fix the detail report bug 463077 or add a legend like morgamic suggests in comment 40. ;-)

Comment 46

[Dave Garrett]

(In reply to comment #45)
> I wonder if it would be more useful and easier to change the horizontal bar
> chart to show the specific versions rather than the abstract list of "Other
> Versions", "Alpha Versions", "Beta Versions", and "Latest Version".

I agree. That's why I made a similar suggestion way back in comment 10. ;)

(In reply to comment #45)
> or maybe histogram format.

I'd go with a pie chart, personally.

Comment 47

[Nick Nguyen [:osunick]]

One more request- limit scope to 3.0 and above (addons currently 2.0 only are 20% of usage)

Comment 48

[Justin Scott [:fligtar]]

Attachment: l10n patch, v1

This patch adds l10n support to the Compatibility Dashboard. I was careful to make it so that if we add support for non-Firefox applications in the future, the l10n would not need to be changed.

Nick, please file a separate bug for your change request. It is not a simple change, because the dashboard is currently dynamic so that when Firefox 3.2, 4, or 100 comes out, all the versioning is automatically determined and no code changes are required. We will have to figure out a formula to make the "above 3.0" request apply to all future and past versions.

Comment 49

[Justin Scott [:fligtar]]

Wil - I applied the patch on khan, fixed the .po errors, compiled, and restarted apache, and everything works fine for me.

http://fligtar.khan.mozilla.org/addons-compat/site/en-US/firefox/compatibility

Not sure why you're having problems with it?

Comment 50

[Wil Clouser [:clouserw]]

Comment on attachment 358509 [details] [diff] [review]
l10n patch, v1

I reapplied and it's working now.  vsprintf() complains when the number of elements in the array doesn't match the %s's in the string and somehow my patch didn't apply write to the .thtmls.

Comment 51

[Justin Scott [:fligtar]]

Checked in the new l10n and am posting to the localizers newsgroup now. Thanks.

Comment 52

[Stephen Donner [:stephend] Not actively reading bugmail]

What's the best way to verify this?  Specifically, what am I looking for on a not-yet-fully-localized site such as https://preview.addons.mozilla.org/de/firefox/compatibility?  Thanks.

Comment 53

[Wil Clouser [:clouserw]]

Primary verification is to make sure en-US hasn't degraded.  That'll keep us on par with where we are.  After that, none of the localizers have had a chance to look at this yet so we can't really verify that.

For what it's worth, I've filed bug 475475 to fix the COMPAT_DEFAULT_VERSION you see on there now.

Comment 54

[Stephen Donner [:stephend] Not actively reading bugmail]

Verified FIXED, per my work with comment 53 as a guideline; if we (localizers) encounter problems localizing this, we'll file bugs.

Comment 55

[Robert Kaiser]

Justin: what about comment #44? Any plans regarding non-Firefox releases?