1411991 - add metrics for symbol uploads through socorro

Status: VERIFIED FIXED

(Socorro :: General, task, P1)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

Peter wrote Tecken which is about to take over symbol uploading from Socorro. One thing we want to know over time to facilitate the switch over is:

1. how many uploads are being done by Socorro?
2. which users are using Socorro?

We want to know this over time.

I was thinking the easiest thing to do here is use the metrics infrastructure I threw together which I think is available in the webapp and do an incr for every symbol upload with the username in the tag. Then we can add a graph to datadog.

This will let us watch people transition and know which people haven't transitioned, yet.

This bug covers seeing if we have anything already that answers our questions and if not, implementing the needed metrics.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Making this a P1. We want to do it asap so that it's in place for Peter's symbol upload transition timeline.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

I thought the metrics junk I threw together covered the webapp, but it totally doesn't. The webapp seems to have no scaffolding or anything for generating statsd metrics. :(

I'm going to make this block on bug #1412590 which involves adding Markus to the mix. After doing that, fixing this would be a three-liner.

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

We want to add an incr for symbol uploads. We can use a different key for web uploads and api uploads. We can add a tag with the user's email address. I think that covers everything.

Something like this for web uploads:

    metrics.incr('web_upload', tags=['user:' + request.user.email])

and this for API uploads:

    metrics.incr('api_upload', tags=['user:' + request.user.email])


Regarding tag values, Datadog has this in an incredibly difficult to find place:

"""
Tags must start with a letter, and after that may contain alphanumerics, underscores, minuses, colons, periods and slashes. Other characters will get converted to underscores. Tags can be up to 200 characters long and support unicode. Tags will be converted to lowercase.
"""

https://help.datadoghq.com/hc/en-us/articles/204312749-Getting-started-with-tags

That's good to know because it means we don't have to deal with sanitizing email addresses--datadog lib will do it for us.

One thing to note is that tags are unique to datadog's statsd, so we won't be able to see tags in Graphite in the local development environment. Since Markus also logs metrics, we will be able to verify tags in the logs.

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

Grabbing this because I'm almost done.

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 4138: fixes bug 1411991 - symbol upload metrics

Comment 6

[[github robot]]

Commits pushed to master at https://github.com/mozilla-services/socorro

https://github.com/mozilla-services/socorro/commit/89b4baab5d7ff2ae031bcfa93f2b256ffc66e430
fixes bug 1411991 - add metrics for symbol upload

This adds metrics generation for symbol uploads. This will let us trace how many
symbol uploads are being done and by which accounts. This will make it easier to
switch from Socorro doing symbol uploads to Tecken doing symbol uploads.

Also, since statsd doesn't support tags, this adjusts the local development
environment to run Markus with the statsd and logging backends configured.

https://github.com/mozilla-services/socorro/commit/eafda620be720d4c5052e59a6ec2afdfb62ce75e
Merge pull request #4138 from willkg/1411991-upload-metrics

fixes bug 1411991 - symbol upload metrics

Comment 7

[Will Kahn-Greene [:willkg] ET needinfo? me]

When this gets to -stage, I'll do a few web uploads and verify that it's generating data. Then I'll add a graph to the Datadog -stage dashboard.

When this gets to -prod, I'll add a graph to the Datadog -prod dashboard.

Comment 8

[Will Kahn-Greene [:willkg] ET needinfo? me]

I did a web_upload on -stage and it sent data and I added a graph to the -stage dashboard. Yay!

I'll do -prod after we deploy to -prod.

Comment 9

[Will Kahn-Greene [:willkg] ET needinfo? me]

We deployed -prod. I added a graph to the Datadog -prod dashboard. We're all set here!