1289849 - enable ad-hoc analysis over PII containing crash data

Status: RESOLVED WONTFIX

(Socorro :: General, task)

Description

[Lonnen :lonnen]

We'd like to be enable ad-hoc investigations over all of the crash data, including PII. Analysis will likely include some off-the-shelf machine learning.

We do not want this to block enabling similar features with the public subset, see https://bugzilla.mozilla.org/show_bug.cgi?id=1273657

We may want to do the opposite and ensure the data platform features would meet this need before advancing this one. Needs coordination with the data platform engineering team to figure out the best operational path -- either cloning the infra or setting up multi-tennancy with some auth something/something *handwave*.


filed on behalf of jst

Comment 1

[Lonnen :lonnen]

We need to introduce some schema and validation step for the new PII fields, as well. That may require a dependent bug.

Comment 2

[Lonnen :lonnen]

Spoke with services ops at our work week last week. Our best bet is to spin up parallel infra to telemetry. This doesn't solve access control, but it does help with the analysis.

Comment 3

[Peter Bengtsson [:peterbe]]

(In reply to Chris Lonnen :lonnen from comment #2)
> Spoke with services ops at our work week last week. Our best bet is to spin
> up parallel infra to telemetry. This doesn't solve access control, but it
> does help with the analysis.

What does that really mean? Would analysis (e.g. Presto queries) be able to query non-public fields?

Comment 4

[Lonnen :lonnen]

There would be a separate instance of the same tools at a different domain that would have the non-public fields included. We still need to figure out how our access control would govern access to that set of infra.

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

I don't think we're doing any work on this front anymore. No one's asked me for something like this as far as I can remember, so I'm going to mark this as WONTFIX. We can re-open if there are specific needs to address.