1126776 - Analytics for the public API

Status: RESOLVED FIXED

(Socorro :: Webapp, task)

Description

[[DEACTIVATED] Adrian Gaudebert]

It would be nice to have detailed analytics about our public API. Notably, what endpoint is called, how many times, with what parameters. 

It is difficult to remove public API endpoints if we have no idea if anyone is using them or not.

Comment 1

[Peter Bengtsson [:peterbe]]

Lonnen suggested we insert calls to statsd to record this. Apparently we have a server here at Mozilla so ideally we'd just be able to set a URL in a config somewhere. statsd is UDP so it's very unlikely to cost much performance anyway. 

Lonnen, can you help us get a URL to set and URL to go to to view the numbers?

Comment 2

[Lonnen :lonnen]

We tried this with statsd and it created a ton of unique keys, which was bad. Additionally, I don't know of a good graphite server to point at anymore.

Could we fire requests at GA instead?

Comment 3

[Peter Bengtsson [:peterbe]]

(In reply to Chris Lonnen :lonnen from comment #2)
> We tried this with statsd and it created a ton of unique keys, which was
> bad. Additionally, I don't know of a good graphite server to point at
> anymore.
> 

We can scrub the values but keep the keys to avoid that problem. We just need a destination to post pings to. Can we stand up a quick-n-dirty stats on Heroku or something similar?

> Could we fire requests at GA instead?

Doing server-side GA events is hard. The JWT madness is scary to figure out. 

What about New Relic? We need an account set up (easy) and a one-liner in the wsgi script (easy)
It'll record every request coming in even if it's all server-side stuff.

Comment 4

[Peter Bengtsson [:peterbe]]

I have some promising code that I'm happy about. It sends a requests.post() pageview event to GA in a separate thread that works in django runserver and (more importantly) uwsgi. 

To test I need to try to enable this gently with waffle and/or config on stage, so I can see if it works as we expect it to. 

Set up a blocker on https://bugzilla.mozilla.org/show_bug.cgi?id=1264775 which is about getting me a GA Tracking ID for crash-stats.allizom.org where I intend to launch this first. 

Also, note-to-self: This awesome tool allows me to experiment with setting some of those parameters and see how they appear in GA. https://ga-dev-tools.appspot.com/hit-builder/

Comment 5

[Peter Bengtsson [:peterbe]]

Another note-to-self; here's a great resource for figuring out what all those GA tracking keys are for 
https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters

Comment 6

[Peter Bengtsson [:peterbe]]

Attachment: Link to Github pull-request: https://github.com/mozilla/socorro/pull/3292

Comment 7

[[github robot]]

Commit pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/74178b751a6d8bbbfe7143a7478b43bf56fe854a
fixes bug 1126776 - Analytics for the public API (#3292)

* fixes bug 1126776 - Analytics for the public API

* API endpoint decorator

* don't bother with async option

* renamed decorator

* grammer typo

* dl

* use logger.exception instead

* unbreak test