527304 - provide smart analysis ala talkback

Status: RESOLVED WONTFIX

(Socorro :: General, task)

Description

[timeless]

smart analysis did statistical selection and flattening of reports showing groupings by stack.

among many useful features, it was flat, so i could quickly browse through a bunch of crash stacks and pick things that were interesting.

we also archived them on ftp (which thankfully is indexed by google, unlike some other servers...)

Comment 1

[chris hofmann]

the code that operated on top of talkback to do this was developed by mozilla so it could be reused.

Comment 2

[chris hofmann]

jay, do you know if the smart analysis scripts are around in some version control repository?

Comment 3

[chris hofmann]

or shiva?

Comment 4

[Shiva Thirumazhusai]

It was all in the server installations. For the new database it needs to be re-written. Is the DB schema available ?

Comment 5

[Jay Patel [:jay]]

the smart analysis scripts we run daily are not checked into CVS or SVN.  they only survive on the actual servers that run them.

ss would know more.

Comment 6

[Austin King [:ozten]]

(In reply to comment #4)
I can get you a snapshot of the Socorro DB Schema, but it is maintained via 
http://code.google.com/p/socorro/source/browse/trunk/socorro/database/schema.py

Let me know what works for you.

Comment 7

[Frank Griswold [:griswolf] [:fgriswold]]

You can also look at http://code.google.com/p/socorro/wiki/SocorroDatabaseSchema which, as with all documentation, may be a step or two behind reality. As of today, I think it is correct.

Comment 8

[Samuel Sidler (old account; do not CC)]

I can attach the scripts later this week.

Comment 9

[Samuel Sidler (old account; do not CC)]

Attachment: Original Smart Analysis

This is the original smart analysis shell script as created by various people at Netscape and Mozilla over a few years. Pulled from star today.

I'm not sure how much help this will be, but no reason to recreate everything from scratch if we can use this as a starting point.

Comment 10

[chris hofmann]

morgamic,  why is this wontfix?

Comment 11

[chris hofmann]

smart analysis would have helped us to detect stuff like
https://bugzilla.mozilla.org/show_bug.cgi?id=531551#c35 
long ago.   we really need this, and maybe with the new backend things like smart analysis and other forms of static analysis on the stack traces will get a lot easier.

Comment 12

[chris hofmann]

bug 570367 is another example of the kinds of things we need to be doing with smart analysis

Comment 13

[chris hofmann]

I'm are also doing some smart analysis kinds of of stuff at
http://people.mozilla.org/crash_stacks/ for the top 300 signatures for major releases.  the reports are just run periodically until we can get better hardware or get them integrated into socorro.

http://people.mozilla.org/crash_stacks/stack-summary-3.6.13.txt is an example that helps to understand the distribution of stacks under a single signature for each of the top 300 signatures.

Comment 14

[Laura Thomson :laura]

chofmann, priority?

Comment 15

[Robert Kaiser]

As a note, it might be helpful for someone who has never seen the talkback stuff to have a description of what those "smart reports" actually mean and if this is one report or multiple that are requested here.

Comment #13 sounds like a report on different stack profiles for the same signature is what this is about. Is that all or is something else there as well?

Comment 16

[chris hofmann]

a modern version of the "smart analysis report" is up and running in some scripts I can fire by hand in  http://people.mozilla.org/crash_stacks/

http://people.mozilla.org/crash_stacks/stack-summary-3.6.13.txt is an example of such report

It basically takes topcrashes in order of their ranking, and does a futher breakdown of the distribution of crashes that share the same signature, but differ in the contents of the top X frames of the stack.  In the reports I'm producing now I'm using 10 frames, and I think talkback used to use 6 frames.

the value of these kinds of reports is to understand when a single signature might be representing several different bugs and differing stacks, and to find the set of stacks that are most important to diagnosing and killing of the majority of the problem.

the methods I'm using to build the current reports are pretty inefficient and don't look at a large enough sample, so replacing this with something that runs faster, looks at larger samples of data, and runs on a regular (daily) schedule are the keys to solving this bug.

Comment 17

[Laura Thomson :laura]

Still wanted? 

Also, is this a dupe of bug 512910?

Comment 18

[chris hofmann]

(In reply to Laura Thomson :laura from comment #17)
> Still wanted? 
> 
> Also, is this a dupe of bug 512910?

Yeah, still needed by anyone that is trying to analyze crash data in any kind of efficient way.  The are either spending hours doing things that jesse talks about in 512910, or filing a bunch of dups that end up needing to be sorted out by some other engineers time.