483835 - Create public stats dashboard

Status: VERIFIED FIXED

(addons.mozilla.org Graveyard :: Public Pages, defect, P2)

Description

[Nick Nguyen [:osunick]]

Create a simple dashboard with the following info:
Total number of add-ons
Total downloads today
New add-ons created today
Queue lengths
New nominations
New file updates

Would also be cool if there was a timeplot graph of total downloads (cumulative) for the past few days so we can see where we're at.

This dashboard should be linked from the header as a simple link above "Register | Log In" that shows 

8200 Add-ons
1.6B Total Downloads

Comment 1

[Nick Nguyen [:osunick]]

BTw- new nominations and new file updates should just be a count, we shouldn't expose this data publicly

Comment 2

[Nick Nguyen [:osunick]]

BTW, when I said header, I meant home page header, not other places.

Comment 4

[Michael Morgan [:morgamic]]

Can I get another BTW please?

Comment 5

[Dave Garrett]

BTW, there's also bug 427104 for showing queue info.

Comment 6

[Michael Morgan [:morgamic]]

Going to merge them.  We'll just keep this one since it is more recent.  Thanks, Dave.

Comment 8

[Michael Morgan [:morgamic]]

Attachment: v1, first crack sans l10n for now

Comment 9

[Wil Clouser [:clouserw]]

Comment on attachment 370359 [details] [diff] [review]
v1, first crack sans l10n for now

This is a good start and probably good enough for v1.  A couple thoughts:

1) "Daily Totals for 2009-03-31 00:00:00 thru 2009-03-31 23:59:59" is annoying.  Let's just call it yesterday when we do L10n.

2) As a developer, I still have a lot of questions.  What I most want to know is how long will my add-on be in the queue.  Since we can't say specifically maybe saying "300 add-ons in the queue, the oldest of which is _____ weeks."  That probably won't help though since a few are pretty old....

3) As a developer, it's not clear to me what "107 files pending review" means - are those add-ons?  files within add-ons? versions of change-arounds?

Comment 10

[Nick Nguyen [:osunick]]

Agree with wil's points.  For 5.0.5 we can rev this more- also currently showing the number of add-ons in the queue isn't really that indicative of review time because of the way add-ons are sorted by original submission date.  As a result, a very old add-on getting nominated gets reviewed right away, whereas a newer add-on may linger for a very long time on page 3 before getting looked at.  Once we fix this sort, it'll be much easier to determine the average length.

Comment 11

[Michael Morgan [:morgamic]]

r24192

TODO:

1. polish some stats - we should probably productize the summary tables fligtar is using for his kev kovash cron job and re-use them for this -- deltas for downloads and other things are cool numbers but expensive queries

2. ask people what they want to see?  have we done this?

3. update the stats themselves

4. l10n when we're satisfied

5. add linkage elsewhere in the app to publicize/promote

Comment 12

[Nick Nguyen [:osunick]]

Chris-

let's talk about this one- could be an interesting short project for you.

Comment 13

[Chris Howse [:chowse]]

Initial designs:

http://people.mozilla.com/~chowse/drop/statistics_dashboard/v1/

- Show statistics prominently on the left size, to provide users a quick overview.
- Show current/cumulative statistics first to emphasize the _size_ of the community and selection.
- Provide historic data going back a week/month/year/etc. from today, showing the _growth_ of AMO.
- Use the chart to display a single stat+timeframe at a time.
  - Use different colors to distinguish statistics.
  - Emphasize longer timeframes where possible (e.g. if showing stats for one day, still show one week's data in the chart).
- In the Review Queue visualization, use two separate data series: one showing the queue's current size, and another showing the queue size w/o editor reviews. This will provide a clear indication of how fast new things are entering the queue, as well as how quickly editors are reviewing them.

Comment 14

[Nick Nguyen [:osunick]]

Great work- for the queue chart I'd also like to see a table below the chart with the following columns
add-ons reviewed|file updates|add-ons nominated

rows will be days|weeks|months which are user selectable.

Comment 15

[Nick Nguyen [:osunick]]

Chris, if you can put in one more iteration with my suggestion from #14 we can kick it to scott for impl in 5.0.8

Comment 16

[Chris Howse [:chowse]]

Updated design w/ tables:

http://people.mozilla.com/~chowse/drop/statistics_dashboard/v2/

Same principles mentioned above still apply.

Comment 17

[Nick Nguyen [:osunick]]

Almost right- I think we can put the table and the charts on the same page and eliminate the modal switch from one to the other.  Also- the aggregations are not quite right- instead of 'this week' it's more like 'group by day|week|month'.  See our existing dashboard for an example:

http://skitch.com/osunick/bwpnk/delicious-bookmarks-statistics-dashboard-add-ons-for-firefox

Comment 19

[Chris Howse [:chowse]]

Take 3:

http://people.mozilla.com/~chowse/drop/statistics_dashboard/v3/01_unified_chart_table.png

Comment 20

[Wil Clouser [:clouserw]]

(In reply to comment #19)
> Take 3:
> 
> http://people.mozilla.com/~chowse/drop/statistics_dashboard/v3/01_unified_chart_table.png

-> mccammos for implementation of this design.

Comment 21

[Scott McCammon [:mccammos]]

Do we want to keep the drop-down of Add-ons with public stats (or any of the other elements on the current stats dashboard page)?

Comment 22

[fcp]

(In reply to comment #21)
> Do we want to keep the drop-down of Add-ons with public stats (or any of the
> other elements on the current stats dashboard page)?

Please keep the drop-down list or some other way to navigate through public statistics of add-ons.  It is useful to compare the stats of one’s own add-on to the stats of other add-ons.

Comment 24

[Scott McCammon [:mccammos]]

Attachment: Timeplot + Datatable, v1

This adds some new metrics to the global_stats table, so you need to run `php bin/run_once/site_stats_history.php` to fill in the data history.

I slightly deviated from the mockup with regard to dropdown menu placement. This patch makes the UI consistent with individual addon stat pages. I can easily move those components around if the mockup layout is preferred.

Comment 25

[Wil Clouser [:clouserw]]

Comment on attachment 394167 [details] [diff] [review]
Timeplot + Datatable, v1

I'm r+ing because I don't want to read through a giant patch again ;) and I only have minor fixes:

- I got this when running the maintenance script the first time:  PHP Notice:  MySQL Error 1048: Column 'count' cannot be null
Query was: [REPLACE INTO global_stats (name, count, date) VALUES ('addon_downloads_new', (SELECT SUM(count) FROM download_counts WHERE date = '2009-08-12'), '2009-08-12')] in /bin/database.class.php on line 118

- Is there a way to make this column be a little wider?  Like, a min-width? http://www.pict.com/view/1459074/0/statistics2520dashboard25203a3a2

- Changing the drowdown for "Group by" to anything other than days results in an alert() that says:  Caught exception: undefined Details: TypeError: v2 is null

Other than that, great work!

Comment 26

[Scott McCammon [:mccammos]]

(In reply to comment #25)

Thanks Wil. The MySQL error is a simple fix.

> - Is there a way to make this column be a little wider?  Like, a min-width?
> http://www.pict.com/view/1459074/0/statistics2520dashboard25203a3a2

I can't pull up your screen shot, but I'm sure you're talking about the date column. A min-width on that column clips the table as its width expands beyond the container. I'll try to recover some space elsewhere to squeeze it all in - even with a long German localization! 

> - Changing the drowdown for "Group by" to anything other than days results in
> an alert() that says:  Caught exception: undefined Details: TypeError: v2 is
> null

I see the alert code in the Timeplot library, but I can't trigger it. (I tried FF3, FF3.5, Safari, and IE 8)

Comment 27

[Scott McCammon [:mccammos]]

Committed in r49095.

Comment 28

[Wil Clouser [:clouserw]]

Thanks.  I'm crunching historical stats on a prod dump right now (took 58 minutes last time) and then I'll import them into preview.amo and this should be verifiable.

Comment 29

[Scott McCammon [:mccammos]]

Attachment: Date Column Styling

This patch shrinks the table content and expands the date column so each row fits nicely into a single line.

Comment 30

[Wil Clouser [:clouserw]]

(In reply to comment #28)
> Thanks.  I'm crunching historical stats on a prod dump right now (took 58
> minutes last time) and then I'll import them into preview.amo and this should
> be verifiable.

Stats are imported on preview.amo

Comment 31

[Scott McCammon [:mccammos]]

Table styling fixed in r49114.

Closing this bug since the alert issue in comment 25 can't be reproduced on preview.

Comment 32

[Stephen Donner [:stephend] Not actively reading bugmail]

If you load https://preview.addons.mozilla.org/en-US/statistics and click on "15", the number doesn't get styled as being "selected," as the other pagination numbers do.

Comment 33

[Scott McCammon [:mccammos]]

Attachment: Last Page Selected Styling

Good catch, Stephen. Here is the one-liner fix.

Comment 34

[Scott McCammon [:mccammos]]

Table pagination styling fixed in r49283

Comment 35

[Stephen Donner [:stephend] Not actively reading bugmail]

Verified FIXED on https://preview.addons.mozilla.org/en-US/statistics.