476660 - Create new script to update stats totals

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Statistics, defect)

Description

[Wil Clouser [:clouserw]]

Once bug 475919 is fixed we'll be getting our stats directly from metrics.mozilla.com.  They will be filling in the following tables:

`update_counts` 
`download_counts`
`addons_collections`

We need a script that takes the data loaded into those tables and fills in:

`addons`.`weeklydownloads`
`addons`.`totaldownloads`
`collections`.`downloads` (this is a running total)

If the script doesn't total all the numbers every time it will need to accept a flag that tells it to (we'll be running back logs and will be replacing already counted numbers).

Comment 1

[Justin Scott [:fligtar]]

Our maintenance script (http://svn.mozilla.org/addons/trunk/bin/maintenance.php) already updates weeklydownloads and totaldownloads from the data in download_counts.

Can probably just add something for collections.

Also, I've been wanting to have an addons.latestupdatepings field for a while, might be a good time to add that in. Will improve performance in a number of places where we have to figure out the latest date's update ping count, and also if we decide to sort popular add-ons based on update pings at some point.

Comment 2

[Wil Clouser [:clouserw]]

maintenance.php is a good idea.  We need to add the following:

1) Code to update `addons_collections`.`downloads` by totaling the data in stats_addons_collections_counts (similar to how the add-on download totals are currently calculated).  Call this section "addons_collections_total"

2) Code to update `collections`.`downloads` by totaling the data in stats_collections_counts.  Call this section "collections_total"

Comment 3

[Jeff Balogh (:jbalogh)]

Attachment: scripts to update collections download totals from the metrics tables

The existing download stats will be lost the first time this runs, but clouserw says we're ok with that.

The deleted comment is a copy/paste error.

Comment 4

[Jeff Balogh (:jbalogh)]

r22770