600929 - Function name as a signature is not granular enough

Status: RESOLVED WONTFIX

(Socorro :: General, task)

Description

[LegNeato]

In 3.6.9, we missed catching a new crash because it had the same signature as an existing (small volume) crash. They had the same signatures because they were crashing in the same function but in different places. Because most of the analysis is done on signatures, the 2 crashes looked to most of our tools as one, unless one drilled down into individual reports.

I'm not sure how best to fix this...perhaps having a line# column in the top crasher view? Perhaps appending the line# to the signature (would probably be too much noise, but might work within a release).

I'll add specific links and examples to this bug in a bit, but I wanted to get this filed for the post-mortem.

Comment 1

[Daniel Veditz [:dveditz]]

I, too, want this, but I slightly worry that we might slice too finely and the same bug might get smeared across different buckets and not get the prominence it needs.

Talkback used to give us an interesting report, for purposes of "top crash" it counted signatures by function name as we do now but then also had a report that broke it down into sub-reports that had different stacks (can't remember if it was by different crashing lines or the first 4 or 5 stack frames).

One thing that might be easy would be to add a column to the reports that lists the instruction address. Sure, the same crash would move around in different builds, but in the same build it'd be easy to sort on that column and see if they're all in the same place. I think we've got this value because that's how we get the source line number, and also the linux builds (or was it mac?) used to list this address in the "crashing address" column rather than the memory address we were trying to access.

"filename plus line number" would be equivalent, less compact but more readable. This information is already in that form in the individual crash reports (although we'd want to strip the path to save space).

Comment 2

[chris hofmann]

> Talkback used to give us an interesting report, for purposes of "top crash" it
> counted signatures by function name as we do now but then also had a report
> that broke it down into sub-reports that had different stacks (can't remember
> if it was by different crashing lines or the first 4 or 5 stack frames).

This is, I think, Bug 527304 - provide smart analysis ala talkback

There are a couple of problems.

1) signature morfing from one release to another (which I think is what comment 0 talks about)

2) one signature is actually many bugs

3) many signatures are actually one bug

The first problem is pretty hard to solve without some kind of human inspection right now.  tools used for solving problem 2 might help a bit since its just a case of many bugs stretching over several releases.

Comment 3

[(not currently active) Ted Mielczarek]

Socorro does have a config option to append line numbers to particular signatures, we use it for js_Interpret since it's such a huge function. See bug 514593.

For an individual build, function:line should be fine, since the line numbers won't change. For comparing across builds, it will probably shift and not be terribly usable. On a stable branch you might be ok, except the crashy places are where you probably fix the most bugs, so those signatures would tend to move.

Comment 4

[Lonnen :lonnen]

we are investing in signature generation now, and with crashes in data platform interested parties could experiment in ipython notebooks with alternatives. Socorro is unlikely to implement these in the foreseeable future.