1371574 - Crash correlation broken since May 5th

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[Hector Zhao [:hectorz]]

(In reply to Hector Zhao [:hectorz] from bug 1257531 comment #18)
> 
> We at Beijing office scraps these reports to find out if any of our
> extensions have unusual high correlation with high volumn crashes, it helped
> a few times in the past.
> 
> We just noticed we've been gotten empty results since 5 May, and it turns
> out since that day only the first character of the extension id was left.
> [1][2]
> 
> [1]:
> https://crash-analysis.mozilla.com/crash_analysis/20170504/
> 20170504_Firefox_52.0.2-interesting-addons.txt
> [2]:
> https://crash-analysis.mozilla.com/crash_analysis/20170505/
> 20170505_Firefox_52.0.2-interesting-addons.txt

We might be able to find out about bug 1370087 sooner if this still works.

Mark as blocking bug 1250132 per:

(In reply to Adrian Gaudebert [:adrian] from bug 1257531 comment #20)
> Hector, that problem very likely comes from us changing the format of the
> addons field. See bug 1250132 for context.

Comment 1

[Peter Bengtsson [:peterbe]]

Here's a long shot...
Marco do you know Hector? Hector do you know Marco?

Hector, 
What Marco did was, a query in ipython notebook, that calculates correlations but using the data that Socorro sends to the Telemetry platform. His work on correlations was also "returned to Socorro" in that on any Signature Report (e.g. https://crash-stats.mozilla.com/signature/?date=%3C2017-06-09T12%3A31%3A21%2B00%3A00&date=%3E%3D2017-06-02T12%3A31%3A21%2B00%3A00&product=Firefox&version=54.0b&signature=CDevice%3A%3ADriverInternalErrorCB#correlations), the Correlations tab is ultimately done by fetching data from https://analysis-output.telemetry.mozilla.org/top-signatures-correlations/data/... 
I think this is the file to start with, if that helps:
https://github.com/mozilla-services/socorro/blob/master/webapp-django/crashstats/crashstats/static/crashstats/js/socorro/correlation.js

The data on that tab *used* to come from fetching from https://crash-analysis.mozilla.com/crash_analysis/...

Pardon me if you already knew all of this and this is not news to you. Perhaps you're perfectly aware of the two sources of correlations analysis. Perhaps you're aware and confident that the old un-maintained scripts on crash-analysis is "better" for you. 

Marco, 
Perhaps there's something you can help out with in this. 

My hope of making this communication connection is that we can progress in deprecating the old crash-analysis infra.

Comment 2

[Hector Zhao [:hectorz]]

Oops, looks like I wrote a reply in my phone but didn't send it.

(In reply to Peter Bengtsson [:peterbe] from comment #1)
> Here's a long shot...
> Marco do you know Hector? Hector do you know Marco?

No, I don't believe we've met.

> 
> Hector, 
> What Marco did was, a query in ipython notebook, that calculates
> correlations but using the data that Socorro sends to the Telemetry
> platform. His work on correlations was also "returned to Socorro" in that on
> any Signature Report (e.g.
> https://crash-stats.mozilla.com/signature/?date=%3C2017-06-
> 09T12%3A31%3A21%2B00%3A00&date=%3E%3D2017-06-
> 02T12%3A31%3A21%2B00%3A00&product=Firefox&version=54.
> 0b&signature=CDevice%3A%3ADriverInternalErrorCB#correlations), the
> Correlations tab is ultimately done by fetching data from
> https://analysis-output.telemetry.mozilla.org/top-signatures-correlations/
> data/... 
> I think this is the file to start with, if that helps:
> https://github.com/mozilla-services/socorro/blob/master/webapp-django/
> crashstats/crashstats/static/crashstats/js/socorro/correlation.js

Thanks for all the details.

> 
> The data on that tab *used* to come from fetching from
> https://crash-analysis.mozilla.com/crash_analysis/...
> 
> Pardon me if you already knew all of this and this is not news to you.
> Perhaps you're perfectly aware of the two sources of correlations analysis.
> Perhaps you're aware and confident that the old un-maintained scripts on
> crash-analysis is "better" for you. 

Since this script is more like an one time assignment which works w/o incidents for quite some time, no, I'm not familiar with all these recent changes.

> 
> Marco, 
> Perhaps there's something you can help out with in this. 
> 
> My hope of making this communication connection is that we can progress in
> deprecating the old crash-analysis infra.

Thanks, I won't have time to work on major changes to this script in the mean time. I'll ask around and see if anyone else in Beijing office is willing to take this.

Also, I think you should know that we're in the process of transition to WebExtension API and still figuring out what will happen to our more complex (and fragile) features.

Comment 3

[Marco Castelluccio [:marco]]

Other than the correlations tab on Socorro, available in crash report and signature pages, there's also https://mozilla.github.io/stab-crashes/addon_related_signatures.html, which shows the signatures that might be correlated with addons.

I'm not maintaining the latter as well as the former, but it should work.

If there's any improvement you need for them, feel free to file bugs on Bugzilla (for the correlations on Socorro) or issues on https://github.com/mozilla/stab-crashes (for the addon signatures page).

Comment 4

[Hector Zhao [:hectorz]]

(In reply to Marco Castelluccio [:marco] from comment #3)
> Other than the correlations tab on Socorro, available in crash report and
> signature pages, there's also
> https://mozilla.github.io/stab-crashes/addon_related_signatures.html, which
> shows the signatures that might be correlated with addons.
> 
> I'm not maintaining the latter as well as the former, but it should work.
> 
> If there's any improvement you need for them, feel free to file bugs on
> Bugzilla (for the correlations on Socorro) or issues on
> https://github.com/mozilla/stab-crashes (for the addon signatures page).

Thanks, I'll have a closer look later.

Comment 5

[Peter Bengtsson [:peterbe]]

In conclusion, I think there's now a much better way to query correlations and it's Marco's excellent work. 

There's no actionable work to do on the old un-maintained correlations code.