1089700 - Raven on middleware isn't reporting

Status: RESOLVED FIXED

(Socorro Graveyard :: Middleware, defect)

Description

[Peter Bengtsson [:peterbe]]

I know it used to work. The webapp is getting lots of 500 errors when trying to fetch things from http://socorro-mware-zlb.webapp.phx1.mozilla.com/bpapi/crash_data/ and clearly there's nothing in sentry [0].

Either the key is wrong, or we need to punch a hole in the firewall. 

[0] https://errormill.mozilla.org/webtools/socorro-prod/

Comment 1

[Peter Bengtsson [:peterbe]]

I'm going to check it locally first to make sure it can work at all.

Comment 2

[Peter Bengtsson [:peterbe]]

It clearly works locally: https://errormill.mozilla.org/webtools/socorro-prod/group/173822/
I took the API key from the prod middleware_secrets.ini

Comment 3

[Peter Bengtsson [:peterbe]]

Hmm... It all looks like it's working. 
I found this::

[[e-mail]1 httpd]$ grep Sentry /var/log/httpd/error_log
[Mon Oct 27 03:12:22 2014] [error] 2014-10-27 03:12:22,559 INFO  - MainThread - Error captured in Sentry. Reference: 8c102aeae0ca4167b95ff0ccb6f5c82c

That error can be found. It's part of this: https://errormill.mozilla.org/webtools/socorro-prod/group/165987/events/925276/

But it doesn't add up. I've received so many errors that the middleware is responding with a 500. 

Perhaps it's not a python exception but a Zeus timeout that never reaches middleware_app.py. We should perhaps include the output on 500 errors from the middleware.

Comment 4

[Peter Bengtsson [:peterbe]]

A baby-step https://github.com/mozilla/socorro/pull/2454 to see why the middleware fails so often from the perspective of the webapp.

Comment 5

[[github robot]]

Commits pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/45b164f390028d502de3003ea2a34ac400ac2c03
bug 1089700 - Raven on middleware isn't reporting

https://github.com/mozilla/socorro/commit/438116ce47ba09bcb6db8a78549bc4f553b0babe
Merge pull request #2454 from peterbe/bug-1089700-raven-on-middleware-isnt-reporting

bug 1089700 - Raven on middleware isn't reporting

Comment 6

[Peter Bengtsson [:peterbe]]

I want to punt on this bug until the changeset in comment #5 has been put into production and actually started reporting some errors (which I check by email by the way).

Comment 7

[Peter Bengtsson [:peterbe]]

Fixed in another bug when we moved into AWS and I'm too lazy/busy to find the right duplicate bug.