1280655 - Provide a subset of production crash data to FuzzManager for experimentation

Status: RESOLVED WORKSFORME

(Socorro :: General, task)

Description

[(not currently active) Ted Mielczarek]

FuzzManager has interesting clustering techniques. We'd like to experiment with them on production crashes. We should get some production crash data into FuzzManager.

There's a bug somewhere on getting crash data into telemetry, it'd be straightforward to scrape data out of telemetry for this purpose, so lonnen said he'd make that bug block this one.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Pretty sure the "let's send it to telemetry!" bug is bug #1273657. If that's not it, I'll apologize profusely and eat my shorts in penance.

Comment 2

[Peter Bengtsson [:peterbe]]

Ted, Do you want to have some the crashes we sent to telemetry to be copied from telemetry to FuzzManager?
Why not send them straight from socorro?

FYI, the way the socorro-->telemetry is going to work is that we're going to add a processor that is very similar to the one we currently have for sending it to the elasticsearch cluster. Basically, it takes the processed crash, redacts it a bit and then does an HTTP POST to a URL. The telemetry team is going to be responsible of accepting those POSTs and collating them into storage files that are appropriate for presto and spark.

Comment 3

[(not currently active) Ted Mielczarek]

Sending them from telemetry was lonnen's idea, I think to simply avoid adding another data storage target for Socorro and all the complexity that that entails. Since this is just for experimentation, we decided that it'd be fine to just have a separate script or something that exfiltrates a sample of data from telemetry to FuzzManager.

Comment 4

[Lonnen :lonnen]

A public subset of crash data (no url, no email) is published as a data set in analysis.telemetry.m.o and this can be achieved with ipython notebooks. See: https://bugzilla.mozilla.org/show_bug.cgi?id=1280659