1336587 - [meta] Signatures for crash pings

Status: RESOLVED DUPLICATE of bug 1631563

(Data Platform and Tools :: General, defect, P2)

Description

[Adam Gashlin (he/him) [:agashlin] (ex-moco)]

We'd like to generate crash signatures from crash pings, similar to Socorro's crash signatures, and likely using a very similar algorithm. This would allow grouping of the much larger body of crash pings to provide more insight into prevalence of certain kinds of crashes.

This would ideally be done via a Socorro API as proposed by bug 828452, but until that is available we will investigate producing something similar. It cannot be expected to produce identical results to Socorro processing the minidump (unless much of the processing, including symbolication, is done on the client as proposed in bug 1296274), so we will be investigating how to make the most of what is currently submitted with the ping.

Comment 1

[Adam Gashlin (he/him) [:agashlin] (ex-moco)]

I'll also investigate modifying the Socorro processor to ingest crash pings (or massaging them into a shape that the unmodified processor can consume).

Comment 2

[Thomas Huelbert]

Hey David, is this something your team will be taking on?

Comment 3

[David Durst [:ddurst]]

Yes, needed for 1310695 and others.

Comment 4

[Thomas Huelbert]

would you mind adding a priority to avoid this showing up in triage? Thanks!

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

I extracted Socorro's signature generation into a Python library:

https://github.com/willkg/socorro-siggen/

Ben Wu (intern) used that library to generate signatures for incoming crash pings over the summer for his internship project. That worked pretty well as I understand it.

If there's anything I can do here, let me know.

Comment 6

[Will Kahn-Greene [:willkg] ET needinfo? me]

Making this a project tracker bug because there are a bunch of things that need to happen to get here.

Right now, I'm fixing fx-crash-sig so people can use it as a Python library in notebooks and other places for their ad hoc investigations. issue 25

When we did some fx-crash-sig a couple of years ago, Tecken (the Mozilla Symbols Server) fell down and we determined that it was a bad idea for symbolication to be tied to symbols upload. I'm working on spinning off a separate symbolication microservice. That's tracked in bug #1636210. I'll make this bug depend on that one.

Once that's done and we can use a symbolication API that can handle bursts, I think the plan was to run that on a sample of crash pings nightly and capture the symbols and crash signatures. That's covered in bug #1631563. I'll make this bug depend on that one.

cc:ing esmyth so he can add his needs in case this work doesn't cover them, yet.

Comment 7

[Will Kahn-Greene [:willkg] ET needinfo? me]

After some discussion, we decided this bug is like bug #1636210, but that one has work done and more context and technical details for our current platform, so I'm going to dupe this one to that one.