1361396 - Stop running correlations script

Status: RESOLVED FIXED

(Socorro :: Infra, task)

Description

[[DEACTIVATED] Adrian Gaudebert]

Old correlations are still being generated at the moment on our crash-analysis host. We should stop running that job, as no one should be using it anymore. 

Note that we want to just stop it, not remove it yet, because we are not sure that nobody is using it. Hopefully, by stopping it, we will quickly get complaints or confirmation that it was unused.

Comment 1

[Peter Bengtsson [:peterbe]]

The cron_libraries.sh is kicked off from /etc/cron.d/socorro
So to stop things we just need to comment out that first line in there.

Do we not want to send an email first before this bug becomes actionable?

Comment 2

[Marco Castelluccio [:marco]]

They are sometimes still useful, as the new correlations don't include the version number of the DLL for now (see bug 1339159).

The version numbers can be obtained via Telemetry, but Telemetry is not accessible by Philipp.

There are several options:
1) Wait for bug 1339159 to be fixed;
2) Ignore bug 1339159, as the worst that can happen is that Philipp can't access the Telemtry dataset and so he will ask me to do it;
3) Ignore bug 1339159, as with the new blocklisting policy we're usually just blocking all DLL versions anyway, without caring which versions are affected.

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

cc:ing Hector to this because he has questions.

Comment 4

[[DEACTIVATED] Adrian Gaudebert]

Here's what we want to do here: 

1. Add one or a few files that explain that the crash-analysis server / scripts are going down soon. Link to this bug for context. Maybe link to different data sources (like the public API, Telemetry...)? Put that file at the root of the box, or in the latest directory that people are likely to use.

2. Send an email to the stability mailing list, informing them that we plan on stopping the correlations script on some given date (Friday next week? ).

3. Stop running the correlations script.

4. Wait for feed-back, complaints, bugs, etc. Redirect people to other potential solutions to their problems.

Comment 5

[Peter Bengtsson [:peterbe]]

Marco, in https://bugzilla.mozilla.org/show_bug.cgi?id=1339159#c1 you wrote: 
"There was a bug in the framework (Apache Spark) I'm using to generate correlations; it is fixed now so this should now be feasible."
So that was the last blocker for this. Can you review your comment above (https://bugzilla.mozilla.org/show_bug.cgi?id=1361396#c2) based on that?

Comment 6

[Marco Castelluccio [:marco]]

The situation is still the same, the blockers for bug 1339159 were fixed, but I still haven't done the work to fix the bug itself. I can probably look into it in a few days.

Comment 7

[Will Kahn-Greene [:willkg] ET needinfo? me]

I'm pretty sure all this code is gone from Socorro now, though I don't recall what bug it was under. At a bare minimum, we no longer have this script.

Going to mark this as FIXED.

If that's not correct, please reopen.