1306891 - Integrate correlation results from https://mozilla.github.io/stab-crashes/correlations.html on crash-stats

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[Marco Castelluccio [:marco]]

The correlation results are sometimes useful to analyze signatures, but
they aren't very visible now at https://mozilla.github.io/stab-crashes/correlations.html.

Moving them on crash-stats will make them useful to a wider audience.

Currently the tool is not analyzing the modules, so we can't replace
the old results just yet. In the future, hopefully, we will be able
to add modules as well and thus replace them completely.

My plan would be to add them to the "Correlations" tab on crash-stats.

Comment 1

[Peter Bengtsson [:peterbe]]

Note-to-self, here is my response from IRC:

<peterbe> marco: /me looking
09:00 marco: there’s a lot to unpack here. First of all, your tool looks cool. I don’t really understand it but that’s because I’m the worlds worst c++ coder. 
09:02 Generally I don’t think we’re going to want to put this into crash-stats. In general we want to reduce the scope of crash-stats. For example, we’re going to see if we can move all "aggregate-thinking” away from SuperSearch and into re-dash etc. 
09:03 And instead of building more built-in reports (as django endpoints) we want to instead encourage use of our API to build your own tools. Like you’ve done, but we data from telemetry. 
09:03 The existing Correlations tab we display on Top Crashers and the Report Index page, is going to be removed soon. 
09:04 It’s use is near miniscule and even its primary author who wrote the code many many years ago says the algorithm isn’t any good. 
09:05 But I can understand how it’s frustrating that all sorts of amazing platform coders make all sorts of cool tools spread all over. One solution I’d be very open to is making a link to your tool. 
09:06 That’s what we do with the link to the Support docs. Instead of trying to integrate sumo into crash-stats we just link to it. 
09:07 Lastly, you should have a word with lonnen too about this. He’s got a much bigger broader picture of how crash-stats can and should evolve. Also, he has a much better understanding of the world of tooling around telemetry data.

Comment 2

[Sylvestre Ledru [:Sylvestre]]

I understand you have plans for crash-stats future. However, for now, this is the primary source of information and tool to manage crashes for Firefox releases.
Until we have a better alternative, I don't think we should stop investing on this tool.
Marco's tool is a big improvement for everyone dealing with crashes. Having it leaving outside of crash-stats is a big downside for his discoverability while having it presented in the crash view would be a huge gain for the developers.

Comment 3

[Nicholas Nethercote [inactive]]

I agree that the existing Correlations tab doesn't work well, and removing it is reasonable.

In contrast, Marco's new tool works extremely well so we want to make it as visible as possible.

Comment 4

[Peter Bengtsson [:peterbe]]

What do you suggest we do? Make a link to it from a Signature Report? 
Alternatively we could simply copy the code and put into github.com/mozilla/socorro and Marco you can be responsible for maintaining it. It could simply replace what's on the Correlations tab on the Signature Report. 

Also, we mustn't rush through this. It's a neat tool but is it a mature success? Like, does it work really well and are a lot of people depending on it? I'm hesitant to add more moving parts to Socorro. We're a small team and busy keeping things running whilst at the same time trying to use the crash data to make an impact in the "telemetry data analysis world".

Comment 5

[Peter Bengtsson [:peterbe]]

What do you think Calixte? You've become quite a seasoned stand-alone-dashboard-with-socorro-data hacker.

Comment 6

[Calixte Denizet (:calixte)]

I'm pretty sure that's a good idea to integrate Marco's data in Socorro and probably better to have them here rather than to have a link. It's not a success for now, but the more you have the more you want.

Comment 7

[Nicholas Nethercote [inactive]]

I am regularly using https://mozilla.github.io/stab-crashes/correlations.html while triaging Nightly crashes, and I believe the other Nightly crash triagers are too.

Comment 8

[Peter Bengtsson [:peterbe]]

So if we make a link to this, where should it appear? When you're on a Signature Report you have a signature but no specific release channel (unless you've done some filtering down to 1 version). When you're on a Report Index page, you have a signature and a release channel (deduced from the version). 

I'd be curious what Lonnen has in store for other general ideas about collecting dashboards and tooling around telemetry data.

Comment 9

[Marco Castelluccio [:marco]]

Attachment: screen.png

What about something like this?

Some heuristics will try to automatically fill the product and channel fields,
but the user has the option to change them.

Clicking on 'Get Correlations' would open a new tab to https://mozilla.github.io/stab-crashes/correlations.html.

Comment 10

[Marco Castelluccio [:marco]]

(In reply to Marco Castelluccio [:marco] from comment #9)
> Clicking on 'Get Correlations' would open a new tab to
> https://mozilla.github.io/stab-crashes/correlations.html.

Or, print the results right after the button.

Comment 11

[Marco Castelluccio [:marco]]

Attachment: screen2.png

This is what I have for now. I think the code is almost complete,
but I need to clean it up to avoid using too many new JS features
(since on Socorro you mostly use ES5).

The heuristics for choosing the initial channel and product are really
simple.

Comment 12

[Marco Castelluccio [:marco]]

(In reply to Marco Castelluccio [:marco] from comment #11)
> The heuristics for choosing the initial channel and product are really
> simple.

The sentence was meant to continue. They are really simple, but should be
OK for the most common use cases.

Comment 13

[Sylvestre Ledru [:Sylvestre]]

Comment on attachment 8800901 [details]
screen2.png

This is great work, bravo.
Two questions:
* As discussed yesterday, I don't think we should have the button, it should be directly loaded when arriving on the tab
* Are you going to also include the graphic?
merci

Comment 14

[Nicholas Nethercote [inactive]]

Nice! I agree with Sylvestre's comments. Also, I think it's safe to remove the old correlations stuff in that tab. Your new system works much better.

Comment 15

[Marco Castelluccio [:marco]]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/socorro/pull/3531

Comment 16

[Marco Castelluccio [:marco]]

(In reply to Sylvestre Ledru [:sylvestre] from comment #13)
> * As discussed yesterday, I don't think we should have the button, it should
> be directly loaded when arriving on the tab

The results are directly loaded, but the button is needed in the signature pages because in those pages we don't know for certain which channel the user is interested in.
I'm initially showing the results based on some heuristics, but sometimes they might fail.

> * Are you going to also include the graphic?

I'll start with the textual results, which are the most useful (because you can directly copy & paste them to Bugzilla), but we can add the graphic later.

(In reply to Nicholas Nethercote [:njn] from comment #14)
> Nice! I agree with Sylvestre's comments. Also, I think it's safe to remove
> the old correlations stuff in that tab. Your new system works much better.

The only problem is that my tool can't find correlations in modules yet. Perhaps we can keep them both for a while and remove the old one once my tool supports modules.

Comment 17

[[github robot]]

Commit pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/6cbe84b79e659ca0b90c8a17d4ef98fc4ed0ba20
Fixes bug 1306891 - Integrate new correlations results in Socorro (#3531)

* Fixes bug 1306891 - Integrate new correlations results in Socorro

* Refactored correlations code, added polyfills for fetch and Promise.

* Use the sha1 lib (from https://github.com/Caligatio/jsSHA) instead of WebCrypto, which isn't available in some browsers

* Fix selection of default channel / product

* Move channel/product guessing back to the signature_correlations handler