938879 - Consume crazy new stats from GA

Status: RESOLVED WONTFIX

(Marketplace Graveyard :: Statistics, defect, P1)

Description

[Wil Clouser [:clouserw]]

Over in bug 938855 we started recording some cool new statistics about searches and downloads.  We'd like to be able to share those with developers and e.me, but to do that we'll need to get the numbers into monolith.  This bug is about figuring out how to pull those numbers out of GA (copy the code we have that already grabs stats from GA) and put them in monolith.  To help figure out the best way to store this data in monolith I'll add the two shapes we're interested in for this data.

Firstly, I'd like to be able to send in a region or list of regions and see a breakdown of the top n keywords.  For example:

>  [
>    "region": "de",
>    "terms": [
>      "world cup": {
>        "total_queries": 1000,
>        "apps": [
>          "espn-app-slug": {
>             "clicks": 100,
>             "installs": 20
>          },
>          "twitter-app-slug": {
>              "clicks": 50,
>              "installs": 15
>          }
>          ...
>        ]    
>      },   
>      "second term": {
>        "total_queries": 500,
>        "apps": [
>          "some-app-slug": {
>            "clicks": 100,
>            "installs": 20
>          },
>          "som2-app2-slug": {
>            "clicks": 50,
>            "installs": 15
>          }
>          ...
>        ]    
>      }
>      ...    
>  ]

This will grow like crazy so if there are concerns about scaling (# of keywords, # of apps per keyword) put in some reasonably high limits.

The second shape is the inverse of the first which I think would be more useful to developers.  We pass in an app slug and get back the top n queries, broken down by region, along with clicks/downloads.  If implementing this second shape is technically significant or time consuming we can skip it - it's not a requirement.

This bug is about:

1) Making a cron which consumes the data from GA daily
2) Adding APIs to monolith to show the data, as above

Comment 1

[Wil Clouser [:clouserw]]

Let's chat when you're back; sounds like e.me plans are changing.  Wontfixing this now in case you're reading bugmail over the holiday. :)