1052152 - Make json available for Loop dashboard

Status: RESOLVED FIXED

(Cloud Services :: Operations: Metrics/Monitoring, task)

Description

[Katie Parlante]

Here's a proposal for the loop json:

{
    "total_callers": 284400,
    "cumulative_callers": [
        {"date":"2014-02-13","time_t":1392249600,"total":0,"fxos":0},
        {"date":"2014-02-14","time_t":1392336000,"total":0,"fxos":0}        
    ],
    "daily_unique_callers": [
        {"date":"2014-02-11","time_t":1392076800,"total":0,"fxos":0},
        {"date":"2014-02-12","time_t":1392163200,"total":0,"fxos":0}        {
    ],
    "new_callers": [
        {"date":"2014-02-12","time_t":1392163200,"total":0,"fxos":0},
        {"date":"2014-02-13","time_t":1392249600,"total":0,"fxos":0},
        {"date":"2014-02-14","time_t":1392336000,"total":0,"fxos":0}
    ],
    "weekly_recurring": [
        {"date":"2014-02-12","time_t":1392163200,"total":0,"fxos":0},
        {"date":"2014-02-13","time_t":1392249600,"total":0,"fxos":0},
        {"date":"2014-02-14","time_t":1392336000,"total":0,"fxos":0}
    ],
    "fortnightly_recurring": [
        {"date":"2014-02-12","time_t":1392163200,"total":0,"fxos":0},
        {"date":"2014-02-13","time_t":1392249600,"total":0,"fxos":0},
        {"date":"2014-02-14","time_t":1392336000,"total":0,"fxos":0}
    ],
    "monthly_recurring": [
        {"date":"2014-02-12","time_t":1392163200,"total":0,"fxos":0},
        {"date":"2014-02-13","time_t":1392249600,"total":0,"fxos":0},
        {"date":"2014-02-14","time_t":1392336000,"total":0,"fxos":0}
    ],
    "urls_clicked": [
        {"date":"2014-02-12","time_t":1392163200,"total":0,"fxos":0},
        {"date":"2014-02-13","time_t":1392249600,"total":0,"fxos":0},
        {"date":"2014-02-14","time_t":1392336000,"total":0,"fxos":0}
    ],
    "urls_generated": [
        {"date":"2014-02-12","time_t":1392163200,"total":0,"fxos":0},
        {"date":"2014-02-13","time_t":1392249600,"total":0,"fxos":0},
        {"date":"2014-02-14","time_t":1392336000,"total":0,"fxos":0}
    ]
}

I don't think it matters (to the dashboard) if the data accumulates in one file or multiple files (separate files depending on process generating them).

Comment 1

[Wesley Dawson [:whd]]

I have this mostly implemented. Due to the number of metrics being reported and the output limit of 64KB the metrics will have to be split into multiple outputs. My plan is to feed the results of the offline queries for recurring users into heka similar to how we finalize the firefox accounts data. Once that's implemented we just need to figure out how to do fxos segmentation since the main entry point (call.setup) will not provide us with this information.

Comment 2

[Katie Parlante]

New plan for segmentation data: repeat the array.

OLD:
[
  {"date": "2014-03-03","total": 0,"time_t": 1393804800,"fxos": 0},
  {"date": "2014-03-04","total": 0,"time_t": 1393891200,"fxos": 0},
  {"date": "2014-03-05","total": 0,"time_t": 1393977600,"fxos": 0}
]

NEW:
{
  "total": [
    {"date": "2014-03-03","n": 0,"time_t": 1393804800},
    {"date": "2014-03-04","n": 0,"time_t": 1393891200},
    {"date": "2014-03-05","n": 0,"time_t": 1393977600}
  ],
  "fxos": [
    {"date": "2014-03-03","n": 0,"time_t": 1393804800},
    {"date": "2014-03-04","n": 0,"time_t": 1393891200},
    {"date": "2014-03-05","n": 0,"time_t": 1393977600}
  ]
]

Comment 3

[Katie Parlante]

We have a live dashboard, so I'm calling this fixed: https://metrics.services.mozilla.com/loop-server-dashboard/

Other bugs capture specifics that still need to get tweaked (like associating the recurring calls metrics with a successful websocket setup).