1764570 - implement sensitive data scrubbing for sentry (socorro)

Status: RESOLVED FIXED

(Socorro :: General, task, P2)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

We updated to sentry-sdk. Now that we're using that, we need to make sure we're not sending PII along with Sentry error reports.

The Socorro webapp has some sanitizing code. I think we can generalize that, combine it with the code in socorro/lib/sentry_client.py so we have a single module that we use for setting Sentry up, sanitizing events, and providing some other functionality.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Grabbing this because it's blocking the migration.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

I did a lot of the R&D work in bug #1764569.

There's a Sentry migration doc that contains analysis on what we need to scrub and how here:

https://docs.google.com/document/d/1sIT80jMVa11T3g2GgCSCchHrMksC71fosr9VgOqAYQM/edit#heading=h.vibvf76xo2th

We're going to use the newly developed Fillmore library (https://pypi.org/project/fillmore/) to scrub Sentry events.

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 6121: bug 1764570: implement sentry pii scrubbing

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

willkg merged PR #6121: "bug 1764570: implement sentry pii scrubbing" in 6b528f6.

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

This adds two new metrics we need to add to the dashboard:

• webapp.crashstats.apps.sentry_scrub_error
• processor.sentry_scrub_error

These aren't prefixed with "socorro". Maybe when we move to GCP we can fix that.

Comment 6

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 6122: bug 1764570: fix predicate rule that had wrong arg names

Comment 7

[Will Kahn-Greene [:willkg] ET needinfo? me]

willkg merged PR #6122: "bug 1764570: fix predicate rule that had wrong arg names" in 043eaf6.

Comment 8

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 6123: bug 1764570: fix basedir and host_id for processor app sentry events

Comment 9

[Will Kahn-Greene [:willkg] ET needinfo? me]

willkg merged PR #6123: "bug 1764570: fix basedir and host_id for processor app sentry events" in 03d0f9c.

Comment 10

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 6124: bug 1764570: update to fillmore 0.1.1; drop capture_error

Comment 11

[Will Kahn-Greene [:willkg] ET needinfo? me]

willkg merged PR #6124: "bug 1764570: update to fillmore 0.1.1; drop capture_error" in 683213f.

Comment 12

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 6134: bug 1764570: update fillmore to 0.1.2; fix fillmore logging

Comment 13

[Will Kahn-Greene [:willkg] ET needinfo? me]

willkg merged PR #6134: "bug 1764570: update fillmore to 0.1.2; fix fillmore logging" in 100e2b9.

Comment 14

[Will Kahn-Greene [:willkg] ET needinfo? me]

I deployed this in bug #1782791 just now. I checked Grafana and Sentry and everything looks ok. Any new issues, we'll do new bugs. Marking as FIXED.