Closed Bug 788048 Opened 8 years ago Closed 7 years ago

[socorro-crashstats] implement correlations

Categories

(Socorro :: Webapp, task)

task
Not set

Tracking

(Not tracked)

RESOLVED DUPLICATE of bug 875990

People

(Reporter: rhelmer, Assigned: selenamarie)

References

Details

(Whiteboard: https://github.com/mozilla/socorro-crashstats/issues/8)

Corellation reports are not working anywhere, and this may be blocked until we need to put correlation stuff into the DB.
Blocks: 788049
Depends on: 798153
OK I am ready to implement the UI for correlations, but I am having trouble finding working examples of the UI on the existing site.

Looking at a source file like https://crash-analysis.mozilla.com/crash_analysis/20121004/20121004_Firefox_15.0.1-core-counts.txt.gz I think that the "Correlations" tab on this page should work:
https://crash-stats.mozilla.com/report/list?product=Firefox&version=Firefox%3A15.0.1&query_search=signature&query_type=contains&query=TouchBadMemory&reason_type=contains&date=10%2F05%2F2012%2006%3A50%3A25&range_value=1&range_unit=weeks&hang_type=any&process_type=any&do_query=1&signature=TouchBadMemory%20|%20mozalloc_abort%20|%20NS_DebugBreak_P%20|%20X11Error

It does not seem to finish loading though.

Does anyone have examples of what this should look like? I am still unconvinced that the current implementation really works as described, especially after reading through the code a while back (more detail in bug 650904)...

However I just want it to look like it should. I suspect that this particular feature is not going to be useful to compare directly with against the old site since it seems to barely work if at all.
Status: NEW → ASSIGNED
Flags: needinfo?
Hey Rob,

So, I have been down this road before and to be honest, I do not think much of the correlations tab actually works. When clicking on the 'load' buttons I always get "Loaded Correlation Data, but none available for this signature TouchBadMemory | mozalloc_abort | NS_DebugBreak_P | X11Error"

This one at least generates a different kind of error ;)

https://crash-stats.mozilla.com/report/list?product=FennecAndroid&query_search=signature&query_type=contains&reason_type=contains&date=10%2F05%2F2012%2011%3A57%3A22&range_value=2&range_unit=weeks&hang_type=any&process_type=any&do_query=1&signature=libxul.so%400xe94218#modver

Here is one that actually works (click on any of the two load buttons):

https://crash-stats.mozilla.com/report/list?range_value=7&range_unit=days&date=2012-10-05&signature=mozilla%3A%3AFrameLayerBuilder%3A%3AClearManagerData%28nsIFrame*%2C%20mozilla%3A%3ALayerManagerData*%29&version=Firefox%3A18.0a1

The display is basically just an ugly unformatted table it seems.
Flags: needinfo?
That the current implementation is so wonky is the reason why bug 642325 and bug 650904 exist in the first place. Right now, all it does is load the fitting text file from crash-analysis, cut out the section matching this signature and display that in a <pre> element. Those text files are huge and it's pretty easy to see why this has to be wonky when you look at what kind of hack this is.
See e.g. https://crash-analysis.mozilla.com/crash_analysis/20121005/ for those text files, e.g. https://crash-analysis.mozilla.com/crash_analysis/20121005/20121005_Firefox_15.0.1-interesting-modules.txt.gz is the file for module correlations on Firefox 15.0.1, AFAIK for yesterday's data (and yes, in "today's" directory as it's what is shown today, IIUC). Just look at it and think about how we'd need to cut out of there what to display for a given signature, and you're in a state between sighing and banging your head against the keyboard, depending on how much you can take. ;-)

Schalk, it's not even a table, it's just plain fixed-width text. We should make this nicely formatted tables in the new implementation, though. ;-)
No longer depends on: 650904
OK so instead of the front-end parsing the output of the correlations script, I am planning to just have the correlations script dump CSV and get that into the DB, so it'll be available to the front-end as a middleware service.

In the interest of getting this out there in time to stage and not make too many user-visible changes, I am going to leave the current UI in place. I think we really need a spec for what it should look like.
Sounds good to me Rob. I will speak to Kairo and put some mocks together for his review as soon as v1 is out the door.
Thanks kairo and sneethling, I didn't realize this was quite so busted. To summarize:

* "correlations" tab on /report/list has spinners that never stop
** the "load" buttons for the addons/modules-with-versions do seem to work
* topcrashers never seems to load anything (I think it may be doing the signature lookup wrong, or the logic the correlation report script uses to identify topcrashers is broken)
* the "correlations" tab on /report/index does actually complete, not sure why this is different than the /report/list one

(In reply to Schalk Neethling [:espressive] from comment #5)
> Sounds good to me Rob. I will speak to Kairo and put some mocks together for
> his review as soon as v1 is out the door.

I think this is definitely the way to go. I have an improved backend almost ready (bug 798153) so we will have a dependable service and not need to do all this lazy-loading (unless we want to, and it works etc).

I'll put in a django correlations view and model that uses this service in this bug.
I am going to unblock this from staging, we are better off just sending traffic to the old URLs until the new service and new UI is ready.
No longer blocks: 788003
No longer blocks: 788045, 788049
Unblocking per comment 7
No longer blocks: 788003
No longer depends on: 798153
Depends on: 749359
Blocks: 749359
No longer depends on: 749359
Assignee: rhelmer → sdeckelmann
I'll have a look at getting something stubbed in.
There have been enough false starts in this bug I think we should start over in bug 875990.
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 875990
You need to log in before you can comment on or make changes to this bug.