1289466 - reify stage submitter

Status: RESOLVED WONTFIX

(Socorro :: Infra, task, P1)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

(Moved from https://github.com/mozilla/socorro-infra/issues/238 )

We have a unicorn node that runs the -stage submitter socorro app which looks at a specific rabbitmq queue in production for crash ids to resubmit to our -stage environment. This app goes down periodically.

https://bugzilla.mozilla.org/show_bug.cgi?id=1236918
https://bugzilla.mozilla.org/show_bug.cgi?id=1288170

There are other instances, but they were reported via email and IRC and there aren't bugs.

This issue covers reifying that node.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Assigning it to JP since I think he's working on it already.

Marking this as critical because every time the stage submitter goes down, we can't push out code. It happens semi-frequently and affects development.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

Marking infra bugs that are important to get fixed asap as P1.

Comment 3

[JP Schneider [:jp]]

I'm tracking my commits for this here:  https://github.com/mozilla/socorro-infra/pull/249/files

Comment 4

[JP Schneider [:jp]]

The PR has been merged.  We are holding off in favor of the new statsd and python updates for the moment.

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

JP did a ton of work reifying the stage submitter over the last few days but then hit problems with consul. After talking to Lonnen and then JP, I decided the stage submitter isn't long for this world and it doesn't make sense to keep fiddling with it.

Given that, I'm nixing this project. We're going to keep it as a long-running node and I'm going to spend some quality time with it over the next few days to get it into a better state so it's less flaky.

Closing this out as WONTFIX.